Passkeys with Spring Security 6.5 and MongoDB Sessions: A Practical Guide

Spring Security 6.5 shipped with built-in WebAuthn/passkey support — a genuinely exciting addition that makes passwordless authentication a first-class citizen of the framework. We recently integrated it into Badger Commerce's central auth server and hit a series of problems that, as far as we can tell, nobody else has written about yet.

This post covers what we learned: the undocumented gotchas, the serialization wall you'll hit with non-default session stores, and the clean DTO-based solution we landed on.

The Setup

Our auth server is a Spring Boot application using Spring Authorization Server. Sessions are stored in MongoDB (via spring-session-data-mongodb), and we use Valkey for caching. The server handles form login, social login (Google/Apple), magic links, and — as of this work — passkeys.

Spring Security's WebAuthn support requires you to implement two repository interfaces for credential persistence:

• PublicKeyCredentialUserEntityRepository — maps WebAuthn user handles to your user model
• UserCredentialRepository — stores the actual passkey credentials (public key, sign count, transports, etc.)

The official docs and every tutorial out there show JDBC or in-memory implementations. We use MongoDB, so we wrote our own backed by Spring Data.

That part was straightforward. The problems started with the WebAuthn ceremony — the multi-step challenge-response flow that happens during registration and authentication.

Problem 1: NotSerializableException

During a WebAuthn ceremony, Spring Security generates a PublicKeyCredentialCreationOptions object (for registration) or PublicKeyCredentialRequestOptions object (for authentication). These contain the challenge, relying party info, allowed credential parameters, and other ceremony state. They need to survive between the two HTTP requests of the ceremony.

By default, Spring Security stores these in the HTTP session via HttpSessionPublicKeyCredentialCreationOptionsRepository and HttpSessionPublicKeyCredentialRequestOptionsRepository.

With an in-memory session store, this works fine. With MongoDB-backed sessions, you get:

java.io.NotSerializableException:
  org.springframework.security.web.webauthn.api.PublicKeyCredentialCreationOptions

PublicKeyCredentialCreationOptions doesn't implement Serializable. Neither do several of its nested types (PublicKeyCredentialRpEntity, AuthenticatorSelectionCriteria, AttestationConveyancePreference, etc.). The PublicKeyCredentialRequestOptions class does implement Serializable, but CreationOptions does not.

This affects anyone using a distributed session store — MongoDB, Redis, JDBC, Hazelcast — basically anyone running more than one instance in production.

The Valkey Detour (and Why We Abandoned It)

Our first instinct was to serialize the options to JSON and store them in Redis, bypassing the session entirely. We registered Spring Security's WebauthnJackson2Module (which provides Jackson mixins for WebAuthn types) and wrote Redis-backed repositories using StringRedisTemplate.

Serialization worked perfectly. Deserialization did not:

InvalidDefinitionException: Cannot construct instance of
  `PublicKeyCredentialUserEntity` (no Creators, like default constructor, exist):
  abstract types either need to be mapped to concrete types, have custom
  deserializer, or contain additional type information

The WebauthnJackson2Module only handles serialization. It registers @JsonSerialize annotations and custom serializers, but provides no corresponding deserialization support. The PublicKeyCredentialUserEntity is an interface with a single implementation (ImmutablePublicKeyCredentialUserEntity) that has a private constructor and no @JsonCreator. Same problem for PublicKeyCredentialRpEntity.

You could write custom deserializers for every abstract type, but you'd be fighting private constructors, sealed hierarchies, and types like COSEAlgorithmIdentifier that have no valueOf(long) method. It's a lot of fragile, version-coupled code.

The Solution: Session-Stored DTOs

The approach we landed on is simple and robust: convert the Spring Security options objects into plain serializable DTOs, store those in the session, and reconstruct the originals via builders on load.

The DTO

Here's the shape of the creation options DTO (simplified):

@Data
static class CreationOptionsDto implements Serializable {
    private String rpId;
    private String rpName;
    private byte[] userId;
    private String userName;
    private String userDisplayName;
    private byte[] challenge;
    private Long timeoutMillis;
    private List<CredParamDto> pubKeyCredParams;
    private List<CredDescriptorDto> excludeCredentials;
    private String authenticatorAttachment; // stored as string value
    private String residentKey;
    private String userVerification;
    private String attestation;
}

Every field is a primitive, byte[], String, or a list of simple DTOs. All trivially serializable. No abstract types, no sealed interfaces, no private constructors.

Save: Options to DTO

@Override
public void save(HttpServletRequest request, HttpServletResponse response,
                 PublicKeyCredentialCreationOptions options) {
    if (options == null) {
        request.getSession(false).removeAttribute(SESSION_ATTR);
    } else {
        request.getSession().setAttribute(SESSION_ATTR, toDto(options));
    }
}

The toDto() method walks the options object and extracts primitives. Enum-like types (which in Spring Security's WebAuthn API are final class singletons, not actual Java enums) are stored via their getValue() string.

Load: DTO to Options

@Override
public PublicKeyCredentialCreationOptions load(HttpServletRequest request) {
    CreationOptionsDto dto = (CreationOptionsDto)
        request.getSession(false).getAttribute(SESSION_ATTR);
    if (dto == null) return null;
    request.getSession().removeAttribute(SESSION_ATTR);
    return fromDto(dto);
}

The fromDto() method reconstructs everything via builders:

var rp = PublicKeyCredentialRpEntity.builder()
    .id(dto.getRpId()).name(dto.getRpName()).build();

var user = ImmutablePublicKeyCredentialUserEntity.builder()
    .id(new Bytes(dto.getUserId()))
    .name(dto.getUserName())
    .displayName(dto.getUserDisplayName())
    .build();

return PublicKeyCredentialCreationOptions.builder()
    .rp(rp).user(user)
    .challenge(new Bytes(dto.getChallenge()))
    .timeout(Duration.ofMillis(dto.getTimeoutMillis()))
    .pubKeyCredParams(params)
    .authenticatorSelection(selBuilder.build())
    .attestation(AttestationConveyancePreference.valueOf(dto.getAttestation()))
    .build();

The same pattern applies to PublicKeyCredentialRequestOptions, which is simpler (challenge, rpId, timeout, allowCredentials, userVerification).

Mapping the "Enum-like" Types

Spring Security's WebAuthn types look like enums but aren't. COSEAlgorithmIdentifier, AttestationConveyancePreference, ResidentKeyRequirement, and UserVerificationRequirement are all final class with static constants. Some have valueOf(String), some don't.

For PublicKeyCredentialParameters (which pairs a credential type with a COSE algorithm), the constructors are private and there's no valueOf. We store the COSE algorithm's long value and match it back against the static constants:

private static final PublicKeyCredentialParameters[] ALL_CRED_PARAMS = {
    PublicKeyCredentialParameters.EdDSA, PublicKeyCredentialParameters.ES256,
    PublicKeyCredentialParameters.ES384, PublicKeyCredentialParameters.ES512,
    PublicKeyCredentialParameters.RS256, PublicKeyCredentialParameters.RS384,
    PublicKeyCredentialParameters.RS512, PublicKeyCredentialParameters.RS1,
};

static PublicKeyCredentialParameters findCredParam(long alg) {
    for (var p : ALL_CRED_PARAMS) {
        if (p.getAlg().getValue() == alg) return p;
    }
    return null;
}

Not elegant, but bulletproof.

Problem 2: Wiring the Custom Repositories

Spring Security's WebAuthnConfigurer picks up PublicKeyCredentialCreationOptionsRepository beans automatically via getBeanProvider(). But the request options repositories (used during authentication) are set directly on two internal filter instances — PublicKeyCredentialRequestOptionsFilter and WebAuthnAuthenticationFilter — which are not Spring beans.

The solution is ObjectPostProcessor, wired through the .webAuthn() configurer:

.webAuthn(webauthn -> webauthn
    .rpId(rpId)
    .rpName("Badger Commerce")
    .allowedOrigins(issuerUrl)
    .disableDefaultRegistrationPage(true)
    .withObjectPostProcessor(
        new ObjectPostProcessor<PublicKeyCredentialRequestOptionsFilter>() {
            @Override
            public PublicKeyCredentialRequestOptionsFilter postProcess(
                    PublicKeyCredentialRequestOptionsFilter filter) {
                filter.setRequestOptionsRepository(sessionRequestOptionsRepository);
                return filter;
            }
        })
    .withObjectPostProcessor(
        new ObjectPostProcessor<WebAuthnAuthenticationFilter>() {
            @Override
            public WebAuthnAuthenticationFilter postProcess(
                    WebAuthnAuthenticationFilter filter) {
                filter.setRequestOptionsRepository(sessionRequestOptionsRepository);
                return filter;
            }
        }))

You need both post-processors. The options filter generates the challenge; the authentication filter consumes it. If they use different repositories, authentication silently fails because the challenge can't be found.

Problem 3: The JavaScript API

Spring Security auto-serves a spring-security-webauthn.js file that handles the browser-side WebAuthn ceremony. If you're writing your own UI (we have a custom Thymeleaf login page), you need to match its exact API format.

A few things the docs don't make obvious:

• The authentication endpoint is /login/webauthn, not /webauthn/authenticate
• The authentication request body uses credType (not type) and includes authenticatorAttachment
• The registration options response is flat JSON — not wrapped in a publicKey property
• AuthenticatorTransport.valueOf() expects lowercase strings ("internal", "usb") — "INTERNAL" throws

We only discovered these by decompiling spring-security-webauthn.js from the JAR. The format is stable but entirely undocumented outside the source.

The Confirm-Identity Pattern

One bonus pattern we implemented: when a known user returns (identified by a cookie on the tenant site), we send an encrypted login_hint (JWE token, RSA-OAEP-256) to the auth portal. The portal decrypts it, looks up the user's registered auth methods, and shows a streamlined "Confirm your identity" page with only the methods they've actually set up — password field, Google button, passkey prompt, or magic link.

This avoids showing a full login form when we already know who the user is, and the JWE encryption prevents email enumeration via the authorization URL.

Takeaways

1. Spring Security's WebAuthn support is production-ready but session-store-naive. If you're not using in-memory sessions, plan for custom options repositories from the start.

2. The WebauthnJackson2Module is serialization-only. Don't count on it for round-tripping through JSON. The DTO approach is cleaner than fighting abstract types with custom deserializers.

3. ObjectPostProcessor is your friend for customizing internal filters that aren't exposed as beans. It's documented in Spring Security's architecture section but rarely mentioned in feature-specific docs.

4. Decompile the JS. If you're building a custom WebAuthn UI, the source of truth is spring-security-webauthn.js inside the Spring Security JAR, not the docs.

5. The ceremony data is short-lived. Options objects are consumed on the next request and have a natural timeout (we use 5 minutes). Whether you store them in the session, Redis, or a database, keep the TTL short and clean up on consumption.

---

*We're building Badger Commerce, an enterprise e-commerce platform. If you're working with Spring Security's WebAuthn support and hit similar issues, we'd love to hear about it below!

Like 0