Shipping a Control That Can Lock Out Your Own Users

{"error":{"code":"PLAY_INTEGRITY_APP_NOT_RECOGNIZED",
"message":"appRecognitionVerdict UNRECOGNIZED_VERSION not allowed in
env=production (allowed: PLAY_RECOGNIZED)",
"request_id":"<redacted>"}}

That response is the control working correctly. Earlier this month, a smoke run against the production backend of a mobile app I build returned that body on every authenticated write from the automation phone in my office. Google Play Integrity had assessed the installed binary and declined to vouch for it, the backend had honored the verdict, and the device being refused was mine.

Device attestation belongs to a small class of controls where a correct implementation and a botched rollout produce an identical user-visible outcome: a 403 that nobody in the support queue can explain. The cryptography is a solved problem with vendor documentation and test vectors. The rollout is not.

What I expected to be the hard part

I expected the verifier to be the work. Per-request App Attest assertions mean reconstructing a hash of the method, path, body and device token exactly the way the client computed it, and verifying an ECDSA P-256 signature over Apple's authenticator data. The replay protection rides on a counter: the Secure Enclave increments a per-key value on every assertion it generates, so the server persists the last value it saw and rejects anything not strictly greater.

That part behaved: the middleware and its verifier service landed as 794 lines across two files in the backend repository, with 1,059 lines of unit tests alongside them. Every hour I spent after that went somewhere else: into whether a denial would be visible, whether a denial could be reversed, and whether the people about to be denied would understand what had happened to them.

Soft mode is the measurement, not a half-measure

The middleware was mounted on three routes chosen for blast radius rather than volume: DELETE /users/me, POST /users/me/export, and POST /tshirt-ideas/:id/vote. It shipped behind ASSERTION_ENFORCE_MODE, which takes three values. off returns before any header read or database query. soft runs the full verification, logs the specific failure code, and calls the route handler anyway. hard returns 403. Staging went to soft on the day the code merged; production stayed at off.

Soft mode is often described as a cautious half-step toward the real control. That is the wrong frame. Soft mode is the only instrument that answers the question you actually need answered before you enforce, which is: how many requests would this have rejected, and for which reason? A separate rollout document merged the same day turned that into gates. Two of them read:

• ASSERTION_SIG_INVALID and ASSERTION_MALFORMED at zero for ≥ 24h on the 3 mounted routes (DELETE /users/me, POST /users/me/export, POST /tshirt-ideas/:id/vote). Either indicates clientDataHash recipe drift → block the flip.
• ASSERTION_MISSING trending to zero as TestFlight rolls forward (>= 95% of device_attestation.provider='app-attest' traffic carrying the header).

The load-bearing word in the first gate is "either". A signature mismatch and a malformed header have different causes, but both mean the client and server disagree about how to build the hash, and a hash disagreement in hard mode is a blanket 403 on all three routes for every iOS user at once. The second gate is load-bearing in a different way: it converts "wait for adoption" into a number a person can look at and a threshold that stops the flip.

A denial nobody can see is not a control

The rollout document closed with six open questions held for a human to decide before production. The first one was the expensive one. Soft mode emitted [assertion:soft] <CODE>; pass-through ... on every failure. Hard mode emitted nothing at all, because the failure code lived only in the response body, and Cloudflare's Logpush does not capture response bodies.

Read that as an operational sequence. You build per-code dashboards during the staging soak, you use them to decide the flip is safe, and the flip turns them off. An iOS regression that started producing ASSERTION_SIG_INVALID in production would have been invisible until customers reported it. The fix, merged the following day, was six lines:

if (mode === "hard") {
  console.warn(
    `[assertion:hard] ${failure.code}; reject request ${requestId} ...`,
  );
}

Six lines of code, 83 lines of test. The prefix is byte-symmetric with the soft-mode line so a single Logpush pattern covers both modes, which is what makes reject rates comparable across the flip rather than merely present on both sides of it.

The revoked column nobody read

A multi-model adversarial security review of the backend, run a few days before that, found something better than a bug.

The device_attestation table has a revoked column. Account deletion wrote to it, with a statement that is still in the codebase: UPDATE device_attestation SET revoked = 1 WHERE user_id = ?1. The middleware that gates every mutation route verified the device token's signature, checked its expiry, and matched its subject claim against the authenticated user. It never queried the table. The column was written and never read.

Write-only security state. A control persists a flag that expresses a security decision, nothing consults the flag, and the code reads as though the decision is enforced. It happens because writing the state is the visible half of the feature and the read is somebody else's file. The practical consequence here was bounded, since a deleted user is tombstoned and rejected by an earlier layer, so replaying their 24-hour device token also required a still-valid session token for an account that no longer exists. The architectural consequence was not bounded at all: there was no working device revocation path. The "I lost my phone, revoke that device" feature that the column's existence implies would have shipped, passed review, and done nothing.

The fix was a single indexed read added to the middleware, plus a new DEVICE_ATTESTATION_REVOKED code that both clients now handle by clearing local attestation state, re-attesting, and retrying once. It also carries a documented limit: the device token holds only a subject claim, so revocation granularity is per user, not per device. Per device needs a token format change.

The test rig the control correctly denied

The plan for validating hard mode on Android was reasonable. Add a productionDebug build flavor pointed at the production identity tenant, install it on the automation Pixel, and drive the real backend end to end. The flavor merged, and the next smoke run produced the 403 at the top of this post.

Google's Play Integrity documentation defines the verdict that came back, UNRECOGNIZED_VERSION, as "The certificate or package name does not match Google Play records." The build was debug-keystore signed and had never been through Play, so the verdict was accurate. Attestation is a statement about distribution provenance, and a locally signed APK has none. Apple's production App Attest service applies the same rule to developer-signed binaries.

What I changed: the automation build is now distributed through a Play internal testing track so that it is Play signed, which is slower to iterate on and is the only version of the test that proves anything. The alternative under consideration was a backend allowlist of tester account IDs that relaxed the verdict requirement, and I would still not rule it out for fast daily runs. But an allowlist tests a code path that production users never take, and the entire purpose of this exercise was to find out what production would do.

The same week produced the client-side version of the same lesson. Repeated automated sign-in cycles tripped bot protection on the identity provider, and both apps reported the result to the user as an incorrect password. The password was correct. Test automation is the first traffic any enforcement control sees, it looks like an attack because structurally it is one, and it is the cheapest available preview of how the control will fail for a real person.

The strongest objection

The strongest objection to this ladder is that soft mode leaves the vulnerability open for exactly as long as it takes to feel comfortable. The whole point of the assertion middleware is that a captured device token can otherwise be replayed against a different request body for up to 24 hours, and the threat table in the original design says so plainly. Every day production sat at off while staging soaked was a day that surface stayed open by choice. Soft mode is not a security posture; it is a scheduling preference wearing one.

That is correct, and I do not have a clean answer to it. What I would defend is narrower: the soak length should be set by the size of the exposure you are leaving open, not by the size of the change you are shipping. This assertion layer sits on three routes and closes a replay window that requires an attacker to already hold both a captured device token and a live session, which is why weeks were tolerable. The Phase 1 control underneath it, which is what stops an unattested client entirely, went to hard in production first and stayed there. If I had inverted those two, the objection would be unanswerable.

What I would gate on before the next flip

Three things have to exist before a blocking control blocks anything, and all three are cheap compared to the verifier. An enforcement mode that is one config value away from off, in both directions. A log line on every denial that carries the specific reason code and is symmetric across modes, so the dashboards you used to decide survive the decision. And a revocation path that has been tested by revoking one real device and watching the next request fail.

Production still reads ASSERTION_ENFORCE_MODE = "off" as I write this, four days after the verifier merged into staging soft mode, and the gates above are why it stays there until the soak numbers say otherwise. Almost none of the work between the code landing and the flip has been cryptography. The 403 from my automation Pixel was the control's first correct denial of a legitimate user, and everything left to do is making sure the next one comes with an explanation.