Change streams (CDC)¶

fdyno implements DynamoDB Streams on FoundationDB versionstamps. A change record is written in the same transaction as the mutation that produced it, not by a separate capture process polling a log. This page states the ordering and delivery guarantees precisely, then describes the sharding model and the read API.

The guarantee¶

Every committed mutation to a stream-enabled table produces exactly one change record, ordered by the transaction's commit version. A consumer reading a shard in sequence-number order observes the committed mutations routed to that shard in commit order — with no gaps, duplicates, or reordering — and every mutation to a given partition key is routed to one shard, so per-key changes are always observed in commit order.

This is the change-feed equivalent of the store's strict-serializable guarantee: stream order is serialization order, because both come from the same FoundationDB commit version.

How records are written¶

When a table has a stream enabled, every item mutation writes a change record into the table's CDC subspace in the same FoundationDB transaction as the mutation:

flowchart LR
    W["PutItem / UpdateItem / DeleteItem"]
    subgraph txn["single FoundationDB transaction"]
      I["i/ · base item"]
      X["x/ · indexes"]
      V["v/&lt;shard,versionstamp&gt; · change record"]
    end
    W --> txn

The record key is written with SetVersionstampedKey, so FoundationDB stamps it with a globally-ordered 12-byte commit versionstamp (a 10-byte transaction version plus a 2-byte intra-transaction order) assigned atomically at commit. Two properties follow directly, with no coordinator:

Atomic with the data. The record and the mutation commit together or not at all. A change can never be lost or duplicated relative to the mutation that produced it, and a reader never sees a change for a write that did not commit.
Totally ordered. The versionstamp is unique and monotonic in commit order, so it is the record's sequence number.

Sequence numbers¶

A record's SequenceNumber is its commit versionstamp encoded as a fixed-width decimal. The encoding is chosen so the sequence number is:

Stable — AT_SEQUENCE_NUMBER re-reads the exact same record; a sequence number names a specific committed change for the life of the stream.
Strictly increasing within a shard, in commit order.
Decodable back to the record's storage key, so an iterator can seek directly to a position without scanning.

Because sequence numbers are anchored to the versionstamp rather than to a record's position in the log, they are trim-safe: removing old records from the front of a shard (see Retention) shifts no surviving record's sequence number and invalidates no outstanding iterator.

Shards¶

A stream is divided into one or more shards. The number of shards per table is fixed at DYNODB_STREAM_SHARDS (default 1).

Single shard (default)¶

With one shard, a table's entire change feed is a single versionstamp-ordered log — one global order over every mutation to the table. This is the default and needs no configuration.

Multiple shards¶

With DYNODB_STREAM_SHARDS = N > 1, the change feed is hash-partitioned across N shards so consumers can read them in parallel. Each mutation is routed to

shardIndex = hash(item's partition-key value) mod N

This routing has one property that makes it correct rather than merely parallel: a given partition key always hashes to the same shard. All changes to one key therefore land in one shard, where they are versionstamp-ordered — so per-key changes are observed in commit order, which is exactly DynamoDB's per-partition ordering guarantee. Records for different keys may be split across shards and carry no cross-shard order, again matching DynamoDB.

Each shard is a contiguous key sub-range of the CDC subspace, so a consumer reading one shard scans only that shard's records. DescribeStream reports all N shards; with many shards it paginates via Limit and ExclusiveStartShardId and returns LastEvaluatedShardId. Shard identifiers are fixed-width (shardId-…) and independent of the table name, so they satisfy the AWS SDK's 28–65-character ShardId contract for any table name.

The N = 1 layout is a strict subset of the N > 1 layout, so switching a table from one shard to many is the only reconfiguration that does not carry forward already-written records; with N = 1 (the default) the keyspace is unchanged.

The read API¶

fdyno serves the DynamoDB Streams control and read APIs — ListStreams, DescribeStream, GetShardIterator, GetRecords — so a consumer reads exactly as it would against DynamoDB: list streams, describe to find shards, get an iterator per shard, then page through records. All four iterator types are supported, per shard:

Iterator type	Starts at
`TRIM_HORIZON`	The oldest record still retained in the shard
`LATEST`	Only changes committed after the iterator is created
`AT_SEQUENCE_NUMBER`	The record with the given sequence number
`AFTER_SEQUENCE_NUMBER`	The record immediately after the given sequence number

GetRecords returns a bounded page and a NextShardIterator that resumes immediately after the last returned record. A reader that drains a shard and keeps polling the returned iterator sees each record exactly once and then an empty page until new changes arrive.

Stream view types¶

All four DynamoDB StreamViewType values are supported:

View type	Record contains
`KEYS_ONLY`	Only the key attributes of the changed item
`NEW_IMAGE`	The item as it appears after the change
`OLD_IMAGE`	The item as it appeared before the change
`NEW_AND_OLD_IMAGES`	Both images

Retention and trimming¶

Change records accumulate until they are trimmed. An opt-in background trimmer (DYNODB_CDC_TRIM_INTERVAL) removes records older than DYNODB_CDC_RETENTION (default 24 hours) from the front of each shard. Trimming is safe at any time precisely because sequence numbers are versionstamp-anchored, not positional: front-trimming a shard never renumbers a surviving record nor breaks an iterator a consumer is still holding. The trimmer is off by default; with it disabled, records are retained indefinitely.

Differences from DynamoDB¶

DynamoDB splits a stream shard into two child shards as a partition grows, exposing a parent/child shard lineage that consumers walk over time. fdyno uses a fixed set of shards and does not split them dynamically. This is a deliberate boundary, not an omission:

Splitting exists in DynamoDB to follow physical partition splits. fdyno's shards are hash ranges over one FoundationDB keyspace, which FoundationDB already spreads across the cluster; there is no physical partition boundary to mirror, and FoundationDB exposes no split trigger to mirror it from.
A split cannot relieve the one case it would be wanted for — a single hot partition key — because every change to that key must stay in one shard to preserve per-key ordering. Splitting it across shards would break the very guarantee streams exist to provide.

Static hash sharding therefore captures the parallelism a split would buy (independent, ordered shards) without the lineage machinery or the ordering hazard. See Compatibility for the complete list of differences.