FIPS mode Windows macOS VPN client ML-KEM: what works?

Running a VPN client in FIPS mode on Windows or macOS means the client may only use cryptography from a module validated under FIPS 140-3 by the Cryptographic Module Validation Program (CMVP). The question for ML-KEM is whether hybrid post-quantum key exchange still works under that restriction, and which hybrid groups remain usable.

Operators of OT and critical infrastructure often face two requirements at once. One is to protect remote access against “harvest now, decrypt later” collection. The other is a contractual or regulatory requirement to stay within FIPS-validated cryptography. This article explains where these two requirements conflict, and where the conflict only looks real.

Does enabling FIPS mode on Windows block hybrid ML-KEM key exchange?

Not automatically. Windows FIPS mode is a policy flag that cryptographic consumers read. Whether a hybrid ML-KEM handshake survives depends on how the VPN client interprets that flag and which group it negotiates.

Windows provides a group (or local) security policy setting, “System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing”, and Microsoft’s own validation documentation says to use the FIPS Local/Group Security Policy setting or a Mobile Device Management (MDM) to enable FIPS-Approved mode. The flag does not filter every algorithm call on the system. Components that honour it, such as the OS TLS stack and well-behaved applications, restrict themselves to approved algorithms. Components that ignore it carry on unchanged.

Which component a client relies on is therefore what decides the outcome. If the VPN client calls into the Windows cryptographic modules, what matters is whether ML-KEM is available there and covered by the module’s current CMVP certificate. If the client ships its own crypto library, the Windows flag has no effect on it, and FIPS compliance depends on that library’s own validation and configuration.

Why X25519 is the usual casualty

Under NIST rules, a hybrid key exchange can stay FIPS-compliant even when one of its parts is not approved. A NIST workshop presentation summarises the rule: approval can be achieved by combining Z’ = Z || T, where Z is a shared secret from a FIPS 140 approved mode and T is a shared secret distributed through other schemes. The same presentation notes that X25519 is not an approved mode. A FIPS-strict client that drops X25519 entirely will lose X25519MLKEM768, unless it recognises that the ML-KEM half can carry the approval.

Is the macOS cryptographic module FIPS 140-3 validated for ML-KEM?

Apple’s corecrypto modules hold FIPS 140-3 validations. Whether ML-KEM falls within a validated boundary depends on the specific certificate and security policy for the macOS release you run. You have to check this per version rather than assume it.

The published policies describe modules such as Apple corecrypto Module v14.1 [Apple silicon, User, Software, SL1], validated as a Security Level 1 module. The security policies reviewed for this article describe classical key establishment: the module provides SP800-56ARev3 compliant key establishment. They do not list FIPS 203 ML-KEM. Before you claim validated ML-KEM on a Mac, look up the module version that ships with your macOS build in the CMVP database and check its approved-algorithm table.

macOS does not offer a system-wide FIPS toggle comparable to the Windows policy. Apple instead points administrators to its documentation: the Apple Platform Certifications guide and Apple Platform Security guide provide the information needed to deploy systems in a FIPS 140-3 compliant way. In practice, a macOS VPN client’s FIPS posture depends on whether it uses the system’s corecrypto services or its own library.

Which hybrid groups are allowed when a remote access client must be FIPS compliant?

All three standardised ECDHE-MLKEM hybrids can be used in a FIPS-approved way. Which one you can use depends on which component sits in your validated module. NIST SP 800-56C requires the approved shared secret to come first in the combiner.

The IETF specification states that all groups defined in this document permit FIPS-approved key derivation as per [NIST-SP-800-56C] and [NIST-SP-800-135]. The ordering explains why. The authors put the ML-KEM shared secret first in X25519MLKEM768, and the ECDH shared secret first in SecP256r1MLKEM768 and SecP384r1MLKEM1024. The IETF TLS Working Group draws the practical consequence: “for SecP256r1MLKEM768 and SecP384r1MLKEM1024, the ECDH implementation must be certified whereas the ML-KEM implementation does not require certification.” For the X25519 variant, the ML-KEM implementation must be certified.

Not everyone is happy with this asymmetry. A commenter in NIST’s 2025 review of the SP 800-56 series asked: “Please standardise X25519 and X25519MLKEM768 as approved KEM.” Until NIST changes the rules, design to the current ones.

Where CNSA 2.0 narrows the choice further

For systems that follow NSA guidance, the NSA authors of the CNSA 2.0 TLS profile state: “CNSA 2.0 requires the use of ML-KEM-1024 for key establishment in TLS.” The NSA’s IPsec profile similarly selects ML-DSA-87 [FIPS204] for signing and ML-KEM-1024. A hybrid built on ML-KEM-768 is therefore an interim step for these environments. SecP384r1MLKEM1024 is the hybrid that matches the CNSA 2.0 strength.

How do I enforce FIPS-approved post-quantum settings on managed Windows and macOS endpoints?

Enforce at three layers: the OS policy, the client’s own group list, and verification on the gateway. The OS flag alone proves nothing about what a tunnel negotiated.

1. Set the OS baseline. On Windows, deploy the FIPS policy through Group Policy or your MDM so users cannot remove it. On macOS, deploy the VPN client and its settings through configuration profiles, and record which corecrypto module version each managed OS build carries.
2. Pin the group list in the client profile. Configure the client to offer only the groups your validation supports, for example SecP256r1MLKEM768 and SecP384r1MLKEM1024. Keep X25519MLKEM768 out unless ML-KEM is validated. Lock the profile against local edits.
3. Disable silent classical fallback. If the gateway accepts a classical-only group, a misconfigured endpoint will still connect, but without post-quantum protection. Reject classical-only proposals on the gateway for OT access policies.
4. Verify on the wire. RFC 10024 registers X25519MLKEM768 as value 4588 (0x11EC) and SecP256r1MLKEM768 as value 4587 (0x11EB). Log the negotiated codepoint at the gateway and alert on anything outside the allow-list.
5. Check the whole tunnel, not just the handshake. FIPS compliance also covers the payload cipher, the KDF and the signature algorithms. ChaCha20-Poly1305, for example, is not a FIPS-approved cipher. A tunnel can have a well-chosen key exchange and still fail a FIPS requirement because of its data-plane cipher.

That last point affects how you assess products. Quantum Network’s QPN, for instance, uses a hybrid key agreement with ML-KEM (FIPS 203) as its post-quantum part, but encrypts payload traffic with ChaCha20-Poly1305. Organisations with a hard FIPS-only mandate should weigh the full algorithm set of any tunnel, including this one, against their requirement.

What breaks when remote access clients are switched to FIPS mode for critical infrastructure access?

The usual failures are interoperability failures, not cryptographic ones. Peers or tools that only speak non-approved algorithms stop connecting, and larger post-quantum messages expose network-path limits that classical handshakes never hit.

Typical failure points in OT remote access

• X25519-only peers. Gateways or jump hosts that only offer X25519 or X25519MLKEM768 fail against a FIPS-strict client that has dropped Curve25519.
• Legacy engineering tools. Vendor maintenance software on engineering workstations may depend on older TLS versions or non-approved ciphers and stop working when the Windows flag is enforced on the same machine.
• Fragmentation on constrained links. Post-quantum keys are much larger than elliptic-curve keys. The NSA’s IPsec profile discusses what happens if ML-KEM-1024 were used in the IKE_SA_INIT exchange, because the message sizes increase. Cellular, satellite and VPN-in-VPN paths with a reduced MTU are where this shows up first.
• CPU cost on small endpoints. FIPS 140-3 adds mandatory self-tests. Pairwise consistency checks are required for all public key cryptography systems implemented within a FIPS 140-3 validated module, which matters for rugged field laptops and thin clients.

Figures to plan with

Source excerpts: ML-KEM-768 has encapsulation keys of size 1184 bytes and ML-KEM-1024 has encapsulation keys of size 1568 bytes. The shared secret is 64 bytes (32 bytes for each part). The workshop slide on ML-KEM key generation records a ~3x performance regression.

How should OT operators sequence the change?

Inventory first, then pilot, then enforce. The order matters because a failed remote access session to a substation or pumping station is an operational incident, not just an IT ticket.

Start by listing every remote access path into OT zones. For each path, record the client, the crypto library it uses, the CMVP certificate that covers it, and the gateway’s group list. Next, choose a hybrid group per path using the table above: a P-256 or P-384 hybrid where only the ECDH component is validated, and SecP384r1MLKEM1024 where CNSA 2.0 applies. Finally, enforce through MDM and verify on the gateway logs. When a vendor updates a module or a new CMVP certificate adds ML-KEM, revisit the group allow-list instead of leaving the interim choice in place indefinitely.