Sync schema: Firestore and Cloud Storage

The cloud format of Vulture’s sync (Phase 8b, decisions §14), written once for both platforms: Android writes it now, iOS adopts it later (ios-changes.md A4). Anything that reads or writes users/{uid}/… in the Firebase project vulture-app-store follows this page.

Where it lives in code: the field names and value encodings are in :domain:sync’s codecs (their tests pin every name), the merge rules in :domain:sync too, the collection names and blob paths in :data:sync (SyncCollection, RemotePath, BlobPaths), and the security rules that enforce it in firebase/ (firestore.rules, storage.rules). If this page and the code disagree, the code’s tests are the truth; fix this page in the same change. Every document shape below is pinned by a golden file in domain/sync/src/test/resources/schema/ (vulture-android/domain/sync/src/test/resources/schema) (flight.json, task.json, mapFile.json, homebase.json, recentCompetition.json, settings.json, profile.json, tombstone.json), checked by :domain:sync’s SchemaGoldenTest; when iOS joins, its encoder should produce the same JSON. Setting up the Firebase project itself: vulture-android/docs/firebase-setup.md.

For an iOS developer: think of Firestore as CloudKit’s private database with JSON-like documents instead of CKRecords, and Cloud Storage as the place CKAssets would live. Unlike NSPersistentCloudKitContainer, nothing is mirrored automatically: each app keeps its own database (Room on Android, Core Data on iOS) as the source of truth and runs a small sync engine that pushes changed rows and pulls changed documents.

1. Design rules

1. The local database is the source of truth; Firestore is a replica. Room is the offline queue (the Firestore SDK’s own offline cache is off, memory cache only).
2. Deterministic ids (decision P8B-D9): the same flight, home base, Soaring Spot task or downloaded map file gets the same document id on every device and platform, so duplicates cannot happen structurally (§4).
3. Platform-neutral values: plain field names, enums as their stable string id (the Swift case names :data:prefs and :data:db already store, never Kotlin ordinals), dates as Firestore Timestamp, calendar days as "yyyy-MM-dd" strings, distances and speeds in SI units, no Android-only JSON strings.
4. Small documents, big bytes in Storage: IGC files and map-file contents are gzipped objects in Cloud Storage.
5. Writes are field-merges (set(…, merge)): a writer only sends the fields it changed and never removes a field it does not know, so a newer app version or the other platform can add fields safely.

2. Layout

users/{uid}                                     profile document
users/{uid}/flights/{flightId}                  flightId = lowercase hex SHA-256 of the IGC file's bytes (64 chars)
users/{uid}/tasks/{taskId}                      see §4
users/{uid}/mapFiles/{fileId}                   see §4
users/{uid}/homebases/{homebaseId}              lowercase hex SHA-256 of name.trim().lowercase()
users/{uid}/recentCompetitions/{competitionId}  the Soaring Spot competition id
users/{uid}/settings/synced                     one document ≈ the iCloud key-value store

Cloud Storage (default bucket vulture-app-store.firebasestorage.app):
users/{uid}/igc/{flightId}.igc.gz               the IGC file, gzip
users/{uid}/files/{contentHash}.gz              a map file's content, gzip, shared by every mapFiles doc with that hash

{uid} is the Firebase Auth user id. Everything outside users/{uid} is closed to clients by the rules. Shared tasks (iOS’s CloudKit sharing) are not part of v1; a later sharedTasks/ tree can be added beside users/.

Common fields on every synced record

Every document in the five record collections and settings/synced carries these, written by the sync engine:

Field	Type	Meaning
schema	int	Document format version, 1. A reader ignores fields it does not know and never deletes them.
updatedAt	Timestamp	Always the server time (FieldValue.serverTimestamp(), the same call in the Kotlin and Swift SDKs); the rules refuse anything else. It is the pull cursor. Never a device clock.
clientUpdatedAt	Timestamp	The writing device’s clock at the edit; only breaks a “both sides changed the same field” tie.
updatedBy	string	A random per-install device id, for debugging and echo detection.
platform	string	"android" or "ios".
deleted	bool	Tombstone (§5.4). A deleted document keeps only the common fields and deletedAt.
deletedAt	Timestamp?	The device’s time of the delete.

users/{uid} (profile)

Field	Type	Meaning
schema	int	1
createdAt	Timestamp	First sign-in.
lastSeenAt	Timestamp	Updated at most once a day.
platforms	array of string	"android", "ios": which platforms have synced.
storageBytes	int	Bytes the user’s Storage objects use (gzipped sizes), changed with FieldValue.increment in the same batch as the metadata write of the blob it counts. The cloud-storage cap reads it (§6).

No personal data beyond what Firebase Auth already holds (no copy of the e-mail address).

flights/{flightId}

Field	Type	Android (Room FlightEntity)	iOS (CoreFlight)
filename	string?	filename	filename
inLogbook	bool	inLogbook	data != nil (logbook) vs recentData only (recents)
createdAt, importedAt, lastOpenedAt	Timestamp?	same	importedAt, lastOpenedAt (createdAt: the local insert time)
pilotName, passengerName, gliderRegistration, gliderType, competitionIdentification, competitionClass	string?	same	the six edited IGC header fields
isRead, isFavorite, isContest, isFieldLanding, isRecordedByVulture	bool	same	same
comments	string? (≤ 10 000 chars, rule)	comments	comments
waterBallast	int	waterBallast	waterBalast (iOS spelling; the cloud name has two l’s)
rating	int	rating	— (iOS keeps the value it reads, ignores it)
taskId	string?	taskId	— (iOS has no assigned task)
weglideFlightId, skylinesFlightId	string?	same	—
takeoffCode, takeoffName, takeoffCountryCode	string?	same	same
isTakeoffProcessed	bool	same	same
summary	map	the cached analysis columns	the cached analysis attributes
trackPreview	Bytes? (≤ 64 KB, rule)	trackPreview	trackPreview (the iOS SimplifiedTrack encoding)
trackPreviewVersion	int	same	same
igc	map { path, sizeBytes, gzBytes, sha256, uploaded }	igcByteCount, igcUploaded	CoreFlightData.rawContent

summary = { analysisVersion, takeoffAt, landingAt, sportKind, isTaskCompleted, taskDeclaredDistance, taskCompletedDistance, taskAverageSpeed, optimizedDistance, optimizedAverageSpeed, fastestAverageSpeed, duration, altitudeGained, thermalAltitudeGained, averageCruisingSpeed, bestThermalVario, rightThermalCount, leftThermalCount }, SI units (m, m/s, s), times as Timestamps. It lets a pulled flight list, count in statistics and show its summary before its IGC is downloaded.

thermalAltitudeGained (int, metres, optional; Android since analysis version 2, 2026-10-20) is the height gained in thermals: Σ altitudeGained of the flight’s thermals that are not attempts, which the statistics’ “Total altitude” sums (Calculation fixes §3). null or absent means not computed yet; an app that does not know the key ignores it. altitudeGained stays the flight’s largest single climb.

Not in the document (derivable or per device): terrain elevations, photos (decision P8B-D3), the search text, and the local bookkeeping columns (dirty, localRevision, syncBase, …). Size ≈ 1.5 KB plus the preview, far under Firestore’s 1 MiB document limit.

tasks/{taskId}

name, importedAt, distance (m), minimumAatDuration (s), turnpoints (array of maps, ≤ 100 by rule), waypointsText, externalName, externalUrl (the iOS relative Soaring Spot format), externalVersion, notes, inactiveAirspace, isStartProcessed, startCountryCode.

A turnpoint map uses the iOS NSKeyedArchiver key names of Waypoint: { name, code, country, latitude, longitude, elevation, style, runwayDirection, runwayLength, frequency, description, order, distance, bearing, zone: { type, isLine, isAAT, R1, A1, R2, A2, A12, minimumAltitude, maximumAltitude } }. The codec (TurnpointCodec in :domain:sync) keeps the cloud format independent of how Room stores turnpoints.

mapFiles/{fileId}

type ("WAYPOINTS" / "AIRSPACE"), source ("USER" / "XCSOAR" / "OPENAIP"), filename, importedAt, itemCount, isVisible, contentUpdatedAt, contentArea, contentHash (lowercase hex SHA-256 of the content bytes), content: { path, sizeBytes, gzBytes, uploaded }.

The content object files/{contentHash}.gz is content-addressed: two documents with the same bytes share one object. A device deleting a map file deletes the object only when no live mapFiles document still has that contentHash (a query on contentHash).

homebases/{homebaseId} and recentCompetitions/{competitionId}

• Home base: name, latitude, longitude, usedAt (Timestamp; newest = the current one). Up to 5 are shown, newest first (a read rule, as on iOS).
• Recent competition: name, location, country, startDate, endDate ("yyyy-MM-dd"), lastOpenedAt.

settings/synced (≈ iCloud key-value store)

{ schema: 1, updatedAt: <server time>, values: {
    "<key id>": { v: <value>, t: <epoch milliseconds, device clock>, d: <device id> },
    … } }

One entry per key; the key ids never change. Values: enums as their string id (the Swift case name), units as the unit id (table below, not iOS’s index), a default zone as { type, isLine, r1, a1, r2, a2, a12 } (type = the CUP style id), the home base as { name, latitude, longitude } or null. About 4 KB for 42 keys.

Key id	iOS key(s)
sport	VultureAnnotationStyle
detectsSportAutomatically	VultureDetectSportAutomatically
windSpeedUnit	VultureSkySightWindSpeedUnit
verticalSpeedUnit	VultureSkySightVerticalSpeedUnit
altitudeUnit	VultureSkySightAltitudeUnit
speedUnit	VultureSpeedUnit
distanceUnit	VultureDistanceUnit
temperatureUnit	VultureTemperatureUnit
homeBase	VultureHomebaseName, VultureHomebaseLatitude, VultureHomebaseLongitude (one value)
mapStyle	VultureMapStyle
isOpenAipAirspaceEnabled	VultureOpenAIPAirspaceEnabled
showHiddenAirspaces	VultureShowHiddenAirspaces
isAutomaticUpdateEnabled	VultureAutomaticUpdateEnabled
showColoredTrack	VultureShowColoredTrack
isPinned	VulturePinned
showWind	VultureShowWind
arePhotosVisible	VultureArePhotosVisible
showThermals	VultureShowThermals
showThermalAttempts	VultureShowThermalAttempts
showFlightEvents	VultureShowAltitudeGained
showOptimizedTrack	VultureShowTrack (the Performance half)
trackOptimization	VultureTrackOptimization
showTask	VultureShowTask
showOptimizedTask	VultureShowOptimizedTask
showAssignedTaskTime	VultureShowAssignedTaskTime
isInFlightTrackVisible	VultureIsFlightPathVisible
isInFlightOptimizedTrackVisible	VultureIsOptimizedPathVisible
isInFlightCurrentTrackVisible	VultureShowTrack (the in-flight half), VultureIsTrackVisible
isInFlightTrackToWaypointVisible	VultureIsTrackToWaypointVisible
startZone, waypointZone, finishZone	VultureWaypointStartZone…, VultureWaypointZone…, VultureWaypointFinishZone… + Type, Line, R1, A1, R2, A2, A12 (one value each)
declaredPilot, declaredPassenger, declaredGliderType, declaredGliderRegistration	VultureDeclaredPilot, VultureDeclaredPassenger, VultureDeclaredGliderType, VultureDeclaredGliderRegistration
declaredCompetitionId, declaredCompetitionClass	VultureDeclaredGliderCompetitionID, VultureDeclaredGliderCompetitionClass
flarmLoggerInterval, isFlarmCompetitionModeEnabled, isFlarmStealthModeEnabled	VultureFlarmLoggerInterval, VultureFlarmCompetitionModeEnabled, VultureFlarmStealthModeEnabled
skySightOverlayOpacity	VultureSkySightOverlayOpacity

Source of this table: SyncedKey in data/prefs/src/main/kotlin/be/fousa/vulture/data/prefs/SyncedKey.kt (id, iosKeys). iOS’s VultureShowTrack is one key for two rows; on iOS write it to both showOptimizedTrack and isInFlightCurrentTrackVisible, and read it from showOptimizedTrack.

Units (iOS stores an index into each setting’s unit list; the cloud stores the unit id):

Key id	iOS index 0	1	2
windSpeedUnit	kph	mph	kt
verticalSpeedUnit	ms	kt	fts (ft/min)
altitudeUnit	m	ft	—
speedUnit	kph	mph	kt
distanceUnit	km	mi	nm
temperatureUnit	celsius	fahrenheit	—

Source: UnitType.units in core/units/…/UnitType.kt (IosUnitIndex in :data:prefs). An unknown id or index falls back to the first unit, as iOS does.

Not synced (as on iOS): list sorts, filters and gallery zoom, the flight panel, play speed, SkySight account state, review counters, the first-launch paywall flag, service credentials, hidden airspaces, photos.

Storage objects

• contentType = application/gzip; the app gunzips itself. No Content-Encoding: gzip, so no client or proxy transcodes the object on the way.
• Size ≤ 32 MB per object (rule); the raw IGC import cap is 64 MiB and IGC compresses about 5×.
• A downloaded IGC is checked: SHA-256 of the gunzipped bytes must equal flightId, else it is discarded.
• Uploads are idempotent (the path is the content’s hash), so a retry overwrites the same object.

3. Limits

Limit	Value	How the schema stays inside it
Document size	1 MiB	A flight < 100 KB with its preview, a task < 50 KB; rule caps on comments, trackPreview, turnpoints.
Writes per batch	500	The engine writes ≤ 400 per batch and ≤ 4 MB of estimated payload.
Writes to one document	~1 per second sustained	Settings are debounced (2 s) and written in one transaction.
Free tier	50 k reads, 20 k writes, 20 k deletes per day	A 500-flight logbook’s first sync ≈ 520 writes; a new phone’s pull ≈ 520 reads. A daily quota error pauses sync until the next day; nothing is lost.

4. Deterministic ids (decision P8B-D9)

All hashes are SHA-256, written as 64 lowercase hex characters; strings are hashed as their UTF-8 bytes.

Record	Id
Flight	SHA-256 of the IGC file’s original bytes (as imported). The same file on two devices = one document.
Home base	SHA-256 of name.trim().lowercase() (root locale).
Task	SHA-256 of externalUrl when the task has one (Soaring Spot, WeGlide, …: two devices saving the same task offline make one document); otherwise a random UUID string.
Map file, downloaded (XCSOAR, OPENAIP)	SHA-256 of `”
Map file, imported by the user (USER)	SHA-256 of `“USER
Recent competition	The Soaring Spot competition id.

The exact strings (separators, case of source / type, trimming) are pinned by :domain:sync’s id tests; iOS reproduces them byte for byte. Android re-keyed existing tasks and map files to these ids in its Room v3 migration.

Risk: flight ids of non-UTF-8 IGC files (plan risk R4)

Android hashes the bytes of the file as it was imported. iOS keeps only CoreFlightData.rawContent (a String), so the closest it can do is hash rawContent.data(using: .utf8). For a plain ASCII or UTF-8 file that gives the same bytes and the same id. For a file whose header holds Latin-1 (or other non-UTF-8) characters, e.g. HFPLTPILOT:Jérôme written by an old logger in ISO-8859-1, iOS decoded the text with a fallback encoding and re-encoding it as UTF-8 changes the bytes, so the two platforms compute two ids and the flight appears twice once iOS joins.

Proposed handling, when iOS adopts this schema:

1. iOS keeps the original file bytes for new imports from then on (e.g. stores the SHA-256 of the original data at import), so new flights always agree.
2. A one-time duplicate check after iOS’s first full pull, on the pulled logbook: flights with the same filename, the same summary.takeoffAt and the same pilotName but different ids are the same flight. Keep the document with the earliest importedAt (that is the Android id when Android imported first), merge the user fields into it with the merge rules of §5, tombstone the other one, and mark the check as done (a flag in the local settings, not synced). This is the only iOS-style dedup left; everything else is structural.

5. Sync rules

5.1 Push (local → cloud)

• Pull first, then push, so the local copy of “what the server has” is current.
• Collections in order: tasks → mapFiles → homebases → recentCompetitions → flights → settings (flights reference tasks).
• Reserve, upload, mark uploaded (the security rules require this order, firebase/README.md): one batch writes the record with its blob map (uploaded = false, gzBytes set) and moves users/{uid}.storageBytes by exactly the change in counted gzBytes, naming the record in the profile’s storageRef: {collection, id} — one blob-size change per batch. Then the gzipped IGC / content is uploaded (a map file’s object carries the custom metadata recordId, since its path holds the content hash); then a later write sets uploaded = true. A reader only downloads a blob whose uploaded is true. A tombstone lowers storageBytes in its own batch.
• Only changed fields: the engine keeps, per row, the fields as last agreed with the server (syncBase on Android) and sends the difference as a field-merge, plus the common fields. Two devices editing different fields of one flight both win.

5.2 Pull (cloud → local)

• Per collection: documents with updatedAt >= cursor − 2 minutes, ordered by updatedAt, in pages of 500. The 2-minute overlap catches writes committed out of order around the cursor; applying a document twice is harmless. The cursor (the largest updatedAt seen) is stored per collection and per uid.
• While the app is in the foreground, snapshot listeners on the same queries apply changes live (decision P8B-D11).
• A pulled flight arrives without its IGC; the file is downloaded when the flight is opened, or in the background by the prefetch job (any network, decision P8B-D5). Recents-only flights are only fetched on open.

5.3 Merge rules (per field, 3-way)

For each field: changed only locally → local; changed only on the server → server; changed on both → the later clientUpdatedAt wins. This is iOS’s NSMergeByPropertyObjectTrumpMergePolicy plus CloudKit’s last-writer-wins. Exceptions:

Field(s)	Rule
inLogbook	local OR remote: adding a flight to the logbook is never lost.
importedAt, createdAt	The earliest non-null.
lastOpenedAt	The latest (Recents are “last opened on any device”); a clear can win too (§5.5).
summary (the whole map)	The higher analysisVersion; a tie keeps the local one.
trackPreview (+ version)	The higher trackPreviewVersion; a -1 (failed) never replaces a positive one.
igc.uploaded, content.uploaded	local OR remote.
taskId	The default rule; if the task is not on the device yet, it is applied once the task arrives.
Settings key	Per key, the larger t wins. A local value never changed by the user counts as t = 0, so the cloud wins over a first-run default (as iCloud KVS does at launch).

After every pull the Recents rule is re-applied locally (at most 50 recents-only flights; older recents-only flights are tombstoned; iOS’s trimRecents including its “stranded” rule).

5.4 Deletes and tombstones (decision P8B-D18)

• A delete is a tombstone: the document stays with deleted: true, deletedAt and the common fields; the other fields are removed. The flight’s IGC object is deleted; a map file’s content object only when no live document shares its contentHash.
• Delete wins over an edit, except a re-import after the delete: a local row created after the remote deletedAt lives again (and re-uploads its file).
• Tombstones are kept 90 days, then purged, locally and in Firestore (the single-field query deletedAt < now − 90 days: only tombstones have deletedAt, so no composite index is needed; deleted in batches of 400). A device that was offline for longer than 90 days can bring a deleted flight back; that is accepted.

5.5 Refinements in the code (:domain:sync)

Where the rules above leave a case open, the code decides it like this (details and tests: vulture-android/domain/sync/README.md); iOS should follow the same choices when it joins.

• Ties go to the server. Both sides changed a field with the same clientUpdatedAt → the document’s value wins, so every device converges on one value. Settings likewise: the same t with different values → the cloud value is applied.
• A clear of lastOpenedAt / usedAt travels. “The latest” applies when both sides changed the date; when one side cleared it (Remove from Recents on a logbook flight) and the other left the agreed value as it was, the clear wins, as CloudKit’s per-property last-writer-wins does on iOS.
• Blob maps are never dropped. A side without igc / content takes the other side’s map; both set → the three-way rule, and uploaded is OR-ed when both name the same object.
• Track preview ranking. The higher trackPreviewVersion wins as a whole, ranked 0 (not made yet) < -1 (failed, no track) < a positive version, so a failed preview never replaces a real one and replaces “not made yet”. A tie keeps the local preview.
• Empty contentHash (a map file whose content is unknown) is written as null, never "".
• Recents trim is iOS’s CoreFlight.trimRecents: at most 50 flights carry lastOpenedAt; beyond that a logbook flight loses its date and a recents-only flight is tombstoned; a recents-only flight that arrived by sync without a date (“stranded”) is tombstoned too. Flights opened at the same instant are trimmed by id, so every device trims the same ones. Only rows that came from the server are swept (a service download never selected looks the same but is left alone).
• A flight pulled before its task is stored without it (the foreign key) and gets the assignment back once the task has arrived, unless the pilot changed the flight in between (Android: :data:sync SyncEngine).

6. Cloud storage cap (decision P8B-D14)

Free users may store 100 MB (100 000 000 bytes, the gzipped sizes) in Cloud Storage; Vulture Pro users have no cap. Metadata (Firestore) is never capped.

• Usage: users/{uid}.storageBytes, changed in the same batch as each blob’s reservation (§5) and lowered by its tombstone. How the rules enforce it, and the gaps they accept: vulture-android/firebase/README.md.
• Pro: the ID token’s custom claim revenueCatEntitlements (a list of entitlement ids, set by RevenueCat’s Firebase extension) contains all. The claim changes when RevenueCat’s webhook fires and reaches the app at the next ID-token refresh (at most an hour, or at once when the app forces a refresh).
• The rules refuse a write that would make storageBytes exceed the cap unless the claim says Pro, and Storage rules check the object’s size against the metadata. A refused reservation means the cap is full. The app checks first and shows “Cloud storage full (100 MB)”; the files that do not fit stay on the device, their metadata still syncs.
• Without the RevenueCat Firebase extension (an owner step, vulture-android/docs/firebase-setup.md step 10) no token has the claim and everyone is treated as free.

7. How iOS maps onto this (for the day iOS joins)

iOS today	Firestore	What iOS adds
CoreFlight + CoreFlightData.rawContent	flights/{sha256} + igc/{sha256}.igc.gz	The id rule of §4 (and the R4 handling); upload the bytes gzipped. No new Core Data attribute needed for flights.
data (logbook) / recentData (recents)	inLogbook	Map the two relationships to and from inLogbook.
Task (waypointsData)	tasks/{id}	A new optional syncID String attribute (an additive, CloudKit-safe change) holding the document id; turnpoints from waypointsData to the array of maps.
File	mapFiles/{id} + files/{sha256}.gz	A syncID attribute; contentHash = SHA-256 of content.data(using: .utf8) (CUB: the Base64-decoded bytes).
RecentHomebase, RecentCompetition	homebases/{sha256(name)}, recentCompetitions/{id}	Only the id rule.
iCloud KVS keys	settings/synced.values.<key id>	The key table and unit table of §2.
Purchases.logIn(<CloudKit record name>)	—	Purchases.logIn(<Firebase uid>) after Firebase sign-in (decision P8-D14), so one subscription unlocks both platforms and the storage claim works.
CloudKit (NSPersistentCloudKitContainer)	—	Keeps running during a transition; Firestore becomes the cross-platform channel. Sign in with Apple joins Google as a provider (P8B-D17).