Cloud Object Pools

← Persistence
STRVCT

Cloud persistence for object pools: one record per row, a version per pool, and an atomic commit protocol.

Overview

Every cloud document—a game session, a character sheet, a campaign catalog—is an Object Pool: a cyclic graph of records reachable from a single root record (objectId == poolId).

Locally, pool rows live in SvLocalRecordStore (IndexedDB). In the cloud, the exact same rows live in a cloud records collection (SvCloudRecordStore). There is no second format: a single mapper converts objects to rows on both sides, making cloud sync a simple operation of reading rows from one store and writing them into the other.

Browser Client SvObjectPool Live object graph in memory SvLocalRecordStore IndexedDB row cache Root row mirrors last acknowledged cloud version SvCloudRecordStore Read Operations open · changes · children asyncScopeRootRows Commit Operation baseVersion → version Idempotent requestId check Cloud Backend (Firestore/DB) records Collection One cloud doc per record row Root row holds pool version blobs/<sha256> Content-addressed storage Lazy fetch on render

The Cloud Model in Five Sentences:

  1. Root Row Identity: A pool's ID is its root object's puuid (objectId == poolId).
  2. Server-Owned State: The root row—and only the root row—carries server-owned columns: version, state, parentId, orderKey, and owner.
  3. Zero-List Folder Pools: Folders are pools whose rows list placed children via root parentId references without storing explicit child lists.
  4. Far References ({ "**": poolId }): The only way to point across pool boundaries. Following a far reference is asynchronous.
  5. Atomic Commit Protocol: Edits reach the cloud through a single commit protocol that compares base versions and applies row deltas atomically.

Scoped Pool Identifiers (`scopedPoolId`)

In the cloud, a pool is identified by its scoped ID:

<scopeId>:<localPoolId>
  • scopeId: The sharing boundary (e.g., user account ID, multiplayer room ID, public catalog ID).
  • localPoolId: The root object's puuid string.

Scoped pool IDs ensure that two accounts cloning the same template document never collide in cloud storage.

Record Row Columns (`SvRecordRow`)

ColumnRoot Row (objectId == poolId)Child Row (objectId != poolId)Written By
poolIdThe root object's puuid (identifies the pool)The enclosing pool's poolIdClient
objectIdThe root object's puuid (objectId == poolId)The child object's own distinct puuid (objectId != poolId)Client
parentId, orderKeyFolder ID & fractional sort keySort key for windowed collectionsClient
payloadJsonRoot object's slot recordChild object's slot recordClient (commit payload)
versionMonotonic content sequence number—Server (mirrored down)
modifiedVersion, isDeletedVersion that last updated rowLast modified version; tombstone flagServer (mirrored down)
scopeId, state, ownerSharing boundary & lifecycle statescopeId onlyServer

Identity Rule: For the Root Row, objectId is identical to poolId (objectId == poolId), identifying the entire document pool and carrying server-owned metadata (version, state, owner). For Child Rows, objectId is the child object's own unique puuid, while poolId identifies the parent pool it belongs to (objectId != poolId).

Local-Only Slots:

A stored slot can be kept exclusively on the device: slot.setIsInCloudRecord(false) keeps it in the local IndexedDB record row but strips it when generating cloud commit payloads (SvObjectPool.cloudPayloadFor). Local view settings, scroll positions, and device sync timestamps use this mechanism to prevent triggering unnecessary cloud commits.

The Record Store Protocol (`SvRecordStoreProtocol`)

Every backing implementation conforms to SvRecordStoreProtocol:

MethodAnswers / Action
asyncOpen(poolId){ root, records, version, state } — the entire pool, or null.
asyncReadChanges(poolId, sinceVersion){ rows, tombstones, version, reloadRequired } — modified rows since version.
asyncChildren(nodeId, range)Rows placed under nodeId ordered by (orderKey, objectId).
asyncResolveFar(poolId)Opens the pool a far reference points to.
asyncCommit({ poolId, baseVersion, requestId, writes, deletes })Atomic commit: returns { status: "committed", version }, "conflict", or "refused".

Backing Implementations:

  • SvLocalRecordStore: The device's IndexedDB store holding every local pool row. Accepts asyncPut / asyncDelete.
  • SvMemoryRecordStore: Reference implementation of both sides (including commit protocol) used for headless testing and in-memory cloud caches.
  • SvCloudRecordStore: The cloud backing executing callFunction(name, args) over HTTPS. Refuses direct writes; modified only via asyncCommit.

Reading a Pool (Opening Decision Tree)

When opening a document, Strvct uses an optimized decision tree to minimize network bandwidth:

               [ User Opens Document ]
                          |
                          v
         Has Local In-Sync Version? (inSyncVersionOfPool)
                 /                 \
            YES /                   \ NO
               v                     v
    asyncReadChanges(sinceVersion)    asyncOpen(poolId)
            /             \          (Fetch Whole Pool)
     Normal /              \ reloadRequired=true
           v                v
   Import Deltas          Replace Local Copy
  (0 to 1 row download)   (Cloud Wins)
  1. In-Sync Version Check: The opener queries SvLocalRecordStore.inSyncVersionOfPool(poolId). If a local copy exists with no unsent local edits, it calls asyncReadChanges(cloudPoolId, version). For returning users, this transfers 0 to 1 rows instead of re-downloading the entire document.
  2. Whole-Pool Download: If no local version exists, it calls asyncOpen(poolId) and imports the full pool (asyncImportOpenedPool).
  3. reloadRequired Handling: If the server has pruned tombstones since sinceVersion, it returns reloadRequired: true, falling back to a full pool download.
  4. Atomic Snapshot Reading: Readers accept only rows stamped at or below the root's version, ensuring a commit currently being written is never seen half-finished.

Writing & Commit Protocol

Edits are saved locally first at the end of the event loop. Sending them to the cloud is a separate, debounced step: SvObjectPool.asyncCommitToCloud():

  1. Flush Local Writes: Flushes pending local IndexedDB writes and computes changed rows since the last acknowledged cloud snapshot (cloudChanges()).
  2. Post Commit Request: Sends modified rows with the local root's version as baseVersion and a unique requestId.
  3. Idempotent Receipts: The server records the requestId. If a network interruption causes the client to retry the request, the server returns the saved receipt without double-incrementing the pool version.
  4. Optimistic Concurrency Control:
    • committed: The server's new version is stamped on local rows, and the snapshot advances.
    • conflict: Another client committed first. The caller invokes holdsCommitContent(commit) to check if its edits were included before reloading remote changes (cloud wins).
    • refused: The commit violated a rule (e.g. attempting to edit server-owned columns).

Staged Commits (`SvStagedRecordCommit`)

For commits exceeding transaction limits (maxOpsPerCommit() 450 operations or 4 MB of payload), SvStagedRecordCommit stages the write:

  1. Begin: Reserves the next pool version via a compare-and-set operation.
  2. Batch Writes: Pushes row updates in sequential batches.
  3. Finalize: Commits the root row, version, and request receipt in a final transaction.

While a stage is open, the pool answers busy to other readers/writers, and SvCloudRecordStore retries automatically (busyRetryDelays). If a stage is abandoned mid-way, the next access to the pool rolls back incomplete batches from saved pre-images.

Zero-List Folder Membership (`SvCloudFolder`)

Folders (SvCloudFolder) declare setSubnodesArePools(true). A folder never stores a list of child IDs:

  • Placement: A document is in a folder because the document's root row sets its parentId to the folder's ID.
  • Scope Root Listings: A folder queries scope root rows via asyncScopeRootRows(scopeId). Each root row contains enough metadata (title, subtitle, thumbnail hash) to render placeholder tiles without opening child pool bodies.
  • Pruning & Tombstones: If a complete scope listing omits a child, the local child is pruned. Deleting a child queues a tombstone descriptor (pendingCloudDeletes) that survives reloads until confirmed by the server.

Far References (`{ "**": poolId }`)

A pointer across pool boundaries is stored as { "**": poolId }, naming the target pool root rather than an individual object. It keeps nothing alive and grants no permissions: resolving it opens the target pool under the target's own permissions. A dangling far reference is a normal state to display in the UI rather than an error.

Key Persistence Classes Matrix

ClassPurpose
SvObjectPoolManages live objects for a document; handles asyncCommitToCloud & cloudChanges.
SvLocalRecordStoreManages local IndexedDB rows (inSyncVersionOfPool, asyncImportOpenedPool).
SvCloudRecordStoreManages cloud record reads, commits, staged commits, and scope root listings.
SvMemoryRecordStoreReference implementation of the record store protocol for headless tests.
SvStagedRecordCommitDrives multi-batch commits for large payload transactions.
SvRecordRowDefines the serialized row shape and column constraints.
SvCloudFolderManages document collections via root parentId references and scope listings.
SvOrderKeyGenerates fractional sort keys for document order in collections.