Release checklists that catch privacy, signing, and metadata drift

The release failures I have seen rarely came from ignorance. The person shipping the build knew about privacy manifests, knew that entitlements have to match the profile, and knew the App Store listing should describe the app accurately.

The build still failed, because the app changed in a way that nobody connected to a rule they already understood.

A permission string was added for a new feature and the App Store privacy answers were not updated. A widget target gained an App Group and its profile was regenerated months later. A dependency update brought a new SDK that collects data, and the privacy labels still describe last spring's app. None of these are knowledge gaps. They are drift between what shipped last time and what is shipping now.

A checklist that only asks "did we do privacy?" catches nothing. A checklist that compares this release against the last approved one catches most of it.

1. Treat the last release as the baseline

The useful question is not "is this app compliant". It is "what is different about this app compared to the version Apple already approved".

That framing changes what you inspect. You stop reading the whole privacy manifest and start diffing it, and the listing review narrows from "is this accurate" to "which fields should this code change have moved".

So the first artifact to keep is a snapshot of the last shipped build: entitlements per target, embedded privacy manifests per bundle, version numbers, the framework list, and the App Store metadata that was live at submission time. Commit it next to the release notes. It costs a few kilobytes and turns compliance review into a diff.

2. Fingerprint the artifact, not the project

Project files describe intent. The archive describes what Apple will receive. Every drift check should read the archive.

#!/usr/bin/env bash
set -euo pipefail
 
APP="build/Packerly.xcarchive/Products/Applications/Packerly.app"
 
for bundle in "$APP" "$APP"/PlugIns/*.appex "$APP"/Watch/*.app; do
  [ -d "$bundle" ] || continue
  id=$(/usr/libexec/PlistBuddy -c "Print :CFBundleIdentifier" "$bundle/Info.plist")
  short=$(/usr/libexec/PlistBuddy -c "Print :CFBundleShortVersionString" "$bundle/Info.plist")
  build=$(/usr/libexec/PlistBuddy -c "Print :CFBundleVersion" "$bundle/Info.plist")
  echo "== $id $short ($build)"
 
  # entitlements actually embedded in the signed binary
  codesign -d --entitlements - --xml "$bundle" 2>/dev/null \
    | plutil -convert json -o - - | LC_ALL=C sort
 
  # privacy manifests shipped in this bundle
  find "$bundle" -name PrivacyInfo.xcprivacy -maxdepth 3 | LC_ALL=C sort
done > build/release-fingerprint.txt

The entitlements come from codesign, not from the .entitlements file, because the signed binary is what ships. The fingerprint also covers every embedded bundle. Extensions are where most drift hides, since adding a capability to the main app is a single checkbox and adding it to three extension targets is three separate pieces of work that are easy to do once.

Keep build/release-fingerprint.txt from the last approved release in the repo as release/last-approved-fingerprint.txt and diff against it:

diff -u release/last-approved-fingerprint.txt build/release-fingerprint.txt

An empty diff means the release surface did not move. A non-empty diff is the review queue for this release.

3. Signing drift looks like a working build

Signing problems usually announce themselves with an error message. The dangerous subset does not.

Three cases pass a local build and fail later:

• a development profile that matches a distribution archive, so aps-environment ends up as development or get-task-allow stays true
• one extension signed with a profile from a previous release while the main app moved on, which uploads fine and then behaves inconsistently on device
• a capability added to the App ID but not to the entitlements file, which uploads and then fails at runtime for the feature that needed it

The fingerprint catches the first two directly, because entitlements and bundle identity show up in the diff. The third needs the reverse comparison: entitlements present in the profile but absent from the binary, which usually means someone enabled a capability in the portal and stopped there.

Keep the profile UUID in the fingerprint as well. If the profile changed between releases, that is worth a sentence in the release notes, because it usually means a capability or a certificate changed too.

4. Privacy drift is a two-sided artifact

The privacy manifest and the App Store privacy answers describe the same behavior in two different places, and nothing in the toolchain keeps them aligned.

Both sides drift for ordinary reasons. A new SDK is added and ships its own manifest, so the app's manifest is unchanged and the answers are stale. A new feature stores a new data type server-side, so the answers should change and no manifest needs to. A permission string is added for a feature you already had, and a careful person notices the usage description but not the data type.

The reviewable check is a small file that maps each declaration to the evidence for it:

declaration                              evidence
NSPrivacyAccessedAPICategoryUserDefaults  SettingsStore, OnboardingState
NSPrivacyCollectedDataTypeCrashData       Sentry SDK, manifest present
NSPrivacyTracking                         false, no ATT prompt in app
App Store: "Identifiers" collected        account id sent to backend
App Store: "Usage Data" collected         analytics events in SessionRecorder

Writing this once takes an hour. Updating it takes five minutes per release, and it is the only artifact that survives staff turnover, because it answers "why is this declared" instead of "what is declared".

When a new feature adds a data type, the row has to be added in both columns. That is exactly the connection that gets missed when the manifest and the listing are reviewed by different people at different times.

5. The listing is part of the build

App Store metadata drifts because it lives outside the repository and changes rarely. The fields that go stale are predictable:

• the privacy policy URL, which 404s after a site restructure
• the support URL, same failure mode, checked less often
• the "What's New" text, which describes the previous version because it was copied as a template
• the age rating questionnaire, which nobody revisits after a feature adds user-generated content or external links
• subscription and in-app purchase products, where the binary references a product identifier that does not exist in App Store Connect or is not approved for the territory
• screenshots, which show a UI from two releases ago

Pull the live version metadata through the App Store Connect API during the release, store it as a JSON file, and diff it against the previous release's copy. The diff is short and readable, and it makes the answer to "did anything change in the listing" a fact instead of a guess.

Check the reverse direction too: fields the new build requires from the listing. If the binary adds a product identifier, a new permission purpose string appears in the archive, or export compliance flags change, the listing side has to be updated before submission. ITSAppUsesNonExemptEncryption in Info.plist is a good example. When it appears for the first time, upload stops asking a question and starts assuming an answer, and getting that answer wrong is a compliance problem rather than an inconvenience.

6. Keep the human checklist short and owned

Automation handles comparisons. It cannot handle judgment, and a checklist that pretends otherwise becomes a list of rows people tick without reading.

The human part should be short enough to take seriously. Mine has eight rows:

1. What new data does this release collect, and where does the user learn about it?
2. What new permission does it request, and is the purpose string specific enough for review?
3. What new paid surface exists, and is it approved in every territory we ship to?
4. What changed in the review notes, and does the reviewer need credentials or steps?
5. Which dependency changed, and did we read its release notes?
6. What is the rollback plan if the release is bad, and who decides?
7. Who signs off, and where is the evidence?
8. What did the last release teach us that should become an automated check?

Each row gets an answer, not a tick. The eighth row is the one that keeps the checklist from rotting, because every release should move one item from human judgment into the script.

7. Fail the build on hard gates, warn on soft ones

Split the checks by consequence.

Hard gates, which should exit non-zero in CI on a release branch:

• version and build numbers are consistent across all targets and higher than the last upload
• every target has a profile whose entitlements include everything embedded in the binary
• get-task-allow is absent or false in the distribution archive
• aps-environment is production
• the fingerprint diff has been reviewed, meaning the file was regenerated and committed with the release notes
• every product identifier referenced in the archive exists in App Store Connect

Soft warnings, which should print and not fail:

• the framework list changed
• the privacy policy URL did not respond 200 in the last check
• screenshots are older than the last two releases

A gate that fails for reasons unrelated to correctness gets disabled within a month, and then it protects nothing. Keep the hard list small enough that a red build always means a real problem.

8. Run it three times, at three different moments

The checks are cheap. Run them where the information is freshest:

• at archive time, so drift is attributed to a commit rather than to a release day
• after App Store Connect validation, so review warnings enter the same record
• immediately before submission, as the last read of the metadata diff

The third run exists because the first two happen hours or days before submission, and the listing is editable the whole time. Someone changing the price or the privacy URL the morning of submission is not malicious. They just did not know there was a gate.

9. What this actually buys

A checklist like this does not make a release safe. It makes the deltas visible, which is the part humans are bad at and machines are good at.

The failure it prevents is specific: a release where every item passed its own check, while the combination of a new SDK, a stale privacy answer, and a regenerated profile produced a rejection, a bug in one extension, or a support thread about a feature that worked in TestFlight.

That class of failure is invisible to a checklist made of questions and obvious to a checklist made of diffs. Build the diff once, keep the last approved fingerprint in the repository, and let the release ask what changed instead of asking whether everyone remembered.