Transactions¶

fdyno uses FoundationDB transactions as the atomic unit behind DynamoDB operations. For a single-item write, that unit includes more than the item: the condition read, old and new base representation, secondary indexes, and optional change-stream record are one commit.

The transaction rule is:

Atomicity follows the FoundationDB transaction boundary, not necessarily the HTTP request boundary.

Most data-plane actions use one transaction. Large non-atomic batches, index backfills, backups, restores, and maintenance use multiple transactions. Each workflow defines when its changes become visible.

Atomic write path¶

sequenceDiagram
    participant C as Client
    participant O as Operation
    participant T as FDB transaction attempt
    participant F as FoundationDB

    C->>O: UpdateItem with condition
    O->>T: open transaction
    T->>F: read metadata and current item
    F-->>T: one snapshot
    T->>T: evaluate condition and compute new image
    T->>F: clear old chunks/index entries
    T->>F: set base item/chunks and new indexes
    T->>F: set versionstamped stream record
    T->>F: commit
    F-->>O: success
    O-->>C: success response

PutItem, UpdateItem, and DeleteItem read the current item in the transaction when conditions, indexes, streams, or return values require it. A failed condition returns from the callback before commit. A successful mutation buffers all affected keys and exposes them together at commit.

Write invariants¶

For any successfully committed state-changing item write:

The base item and all of its chunks represent one image.
Entries derived from the old image are removed and entries for the new image are added in the same transaction.
If streams are enabled and the state changed, the change record commits with the item and index entries.
No reader can observe only a subset of those committed writes.

An identical replacement or an absent-item delete may commit without a stream record because there is no state change to report.

Transactional APIs¶

`TransactWriteItems`¶

fdyno supports up to 100 actions containing Put, Update, Delete, and ConditionCheck, including actions across tables. The implementation:

validates the request and rejects two actions targeting the same item;
reads all condition targets in one FoundationDB transaction;
builds per-action cancellation reasons;
applies no mutations if any condition fails;
otherwise writes every item, index entry, stream record, and optional idempotency record before one commit.

A condition failure returns TransactionCanceledException; tests verify that an unrelated put in the same request is absent afterward. Cross-table transaction tests verify that writes to different table directories commit through the same API call.

flowchart TB
    REQ["TransactWriteItems"] --> VALIDATE["validate <= 100 actions<br/>unique targets · <= 4 MiB item payload"]
    VALIDATE --> TX
    subgraph TX["one FoundationDB transaction"]
      TOKEN["read ClientRequestToken record"]
      CONDITIONS["read targets and evaluate all conditions"]
      MUTATIONS["items + chunks + indexes + stream records"]
      SAVE["write token fingerprint"]
      TOKEN --> CONDITIONS --> MUTATIONS --> SAVE
    end
    TX -->|"all pass"| COMMIT["one commit"]
    TX -->|"condition or validation failure"| ABORT["no writes"]

`TransactGetItems`¶

Up to 100 distinct targets are read in one read transaction. Their responses come from one FoundationDB read version, including requests spanning multiple tables.

PartiQL `ExecuteTransaction`¶

ExecuteTransaction parses up to 100 statements, forbids mixing reads and writes, and executes the accepted statements through one FoundationDB transaction. Unlike TransactWriteItems, its request type has no ClientRequestToken; callers should not assume token-based deduplication for PartiQL transactions.

Idempotency with `ClientRequestToken`¶

TransactWriteItems can turn an uncertain resend into a deduplicated replay. fdyno stores this conceptual record in dynodb_txn_tokens:

ClientRequestToken -> (format version, SHA-256 request fingerprint, createdAt)

The fingerprint covers the transaction actions and response options, excluding the token itself. Maps are encoded deterministically; numbers and set members are canonicalized on a copy before hashing. Reusing a live token with a different request returns IdempotentParameterMismatchException.

The token check and token write occur in the same transaction as the protected mutations:

If the commit happened, the token exists and the next attempt performs no second mutation.
If the commit did not happen, neither the data nor token exists, so the next attempt can execute normally.

This survives process restart and works across service instances because the record is in FoundationDB. An integration test submits the same token through two Service instances, including concurrently, and verifies that a numeric increment is applied once.

The deduplication window is 10 minutes. Expired records are ignored semantically and may be removed by the opt-in token garbage collector. A replay after the window is a new request. Malformed, unknown, or weak token records fail closed and cannot prove that two requests are equal.

See Reliable writes for client-side retry patterns.

Automatic retries¶

FoundationDB uses optimistic concurrency. The transaction runner can execute a callback more than once when OnError classifies an error as retryable.

A successful read-only callback returns after its required read futures resolve. It does not commit an unmodified transaction. The reads still share one FoundationDB transaction snapshot. Write callbacks retain commit and unknown-outcome handling.

stateDiagram-v2
    [*] --> Attempt
    Attempt --> ReadSuccess: read-only callback resolves
    Attempt --> Commit: write callback succeeds
    Attempt --> OnError: retryable FDB error
    Attempt --> ReturnError: domain or terminal error
    Commit --> Success: commit acknowledged
    Commit --> OnError: retryable FDB error
    OnError --> Attempt: transaction reset
    OnError --> ReturnError: terminal error or deadline
    Attempt --> ReturnError: request deadline
    Commit --> Unknown: deadline while commit is in flight
    ReadSuccess --> [*]
    Success --> [*]
    ReturnError --> [*]
    Unknown --> [*]

This has two coding consequences:

Attempt-local state: response accumulators must be allocated inside the callback. BatchGetItem does this, and an injected not_committed integration test verifies that retrying does not duplicate results from the failed attempt.
No irreversible callback side effects: only the final committed FoundationDB mutations are authoritative. Time values may be recomputed on retry, but sending an external message from a callback would not be safe.

Conflicting read-modify-write operations are retried against a new snapshot. The concurrency suite's same-key counters, list appends, set unions, and nested updates exercise this behavior and verify that accepted updates are not lost.

Deadlines and unknown commit outcomes¶

The HTTP boundary gives the whole operation one deadline; foreground transactions also set a FoundationDB timeout from the remaining budget. The default budget is 30 seconds.

Cancellation has two meanings:

Before commit is attempted: the transaction is cancelled and no buffered write became durable through that attempt.
While waiting for commit: cancellation can race a successful commit. The runner returns an error whose message says the outcome may be unknown.

A caller must not interpret the second error as proof of rollback. Fixed-value puts and deletes are naturally repeatable, but read-modify-write operations such as numeric ADD can apply twice after a blind resend. Put must-happen-once logic in TransactWriteItems and reuse the same client-generated token.

Transaction limits¶

There are two layers of limits:

Limit	Enforced behavior
400 KiB DynamoDB item	`PutItem`, update paths, batch puts, and transactional puts/updates validate item size.
100 transaction actions	`TransactGetItems`, `TransactWriteItems`, and PartiQL `ExecuteTransaction` reject larger lists.
4 MiB `TransactWriteItems` put payload	The request is rejected before opening the write transaction.
16 MiB HTTP request body	Raw and decompressed bodies are bounded at the protocol edge.
FoundationDB transaction size/value limits	Error codes for an oversized transaction, key, or value are mapped to an actionable validation error.
FoundationDB's bounded transaction/MVCC window	Query/scan pages and maintenance scans are bounded; large workflows use multiple transactions.
30-second default operation budget	Retries share a total request deadline rather than resetting an unlimited timer per attempt.

The physical FoundationDB footprint can be much larger than the DynamoDB item payload: one put can write chunks, several index projections, and a stream image. Thus a request below DynamoDB's logical size limit can still exceed FoundationDB's single-transaction limit.

`BatchWriteItem` is non-atomic¶

BatchWriteItem is not a transactional API. fdyno first tries the validated batch in one transaction. If FoundationDB returns transaction_too_large, fdyno validates the request in a transaction that it rolls back. It then groups writes under a conservative 4 MiB estimate and commits each group separately. A failed group is returned in UnprocessedItems.

Consequences:

invalid input writes nothing because validation finishes before chunk commits;
a valid large batch can be partially applied, which is allowed by the batch API;
clients must resubmit only UnprocessedItems;
this fallback must not be confused with TransactWriteItems, which remains all-or-nothing and is never split.

Multi-transaction workflows¶

GSI creation: metadata registers CREATING; page transactions backfill; a final transaction marks ACTIVE. See Data model.
Backup: pages share an explicitly pinned read version, then backup chunks are written in separate transactions and metadata is written last. If the pinned version ages out, the operation fails instead of mixing snapshots.
Restore: small restores may fit one transaction; larger restores publish the table as CREATING, write bounded batches, then mark it ACTIVE.
TTL: an expired read may hide the item, then a cleanup transaction re-reads and re-checks expiry before deleting, so a concurrent TTL refresh is not removed.

These workflows are consistent by protocol and state transitions, not because the whole HTTP request is one transaction.

Failure matrix¶

Failure	Observable result
Validation or failed condition	No transaction commit; DynamoDB validation, conditional, or cancellation error.
Retryable conflict	Callback is retried until success or operation deadline.
Oversized transactional request	Validation error; the client must reduce item count/size.
Oversized `BatchWriteItem`	May be split; incomplete groups are returned as `UnprocessedItems`.
Process exit before commit	No partial transaction is visible.
Process exit after commit but before response	Data is durable; caller may see an ambiguous failure and must retry safely.
FoundationDB unavailable	Request fails; no local fallback accepts the write.
Same live token, same request	Successful replay response with no second mutation.
Same live token, different request	`IdempotentParameterMismatchException`.

Not implemented¶

fdyno does not expose a transaction handle across API calls, support arbitrary interactive transactions, or promise one snapshot across paginated HTTP requests. It also does not add an idempotency key to APIs that do not have one in their DynamoDB request format.

The formal isolation guarantee and pagination boundaries are described in Consistency model.

Code and tests¶

Transaction runner, deadlines, and ambiguous commit result: internal/dynodb/transaction_runner.go
Single-item atomic paths: internal/dynodb/item_ops.go
Transaction validation and execution: internal/dynodb/txn_ops.go
Durable token format, expiry, and fail-closed parsing: internal/dynodb/idempotency.go, internal/dynodb/retry_safety_test.go
Cross-instance and concurrent token replay: TestTransactWriteTokenIsStrongAndCrossInstance in internal/dynodb/retry_safety_integration_test.go
Condition rollback and mixed/cross-table transactions: test/extenddb/test_transaction_operations.py and test/nubo-db/tests/tier2/transactions/transactWrite.test.ts
Action and 4 MiB limits: test/nubo-db/tests/tier3/limits/transactionLimits.test.ts
Same-key conflict workloads: test/extenddb/test_concurrency.py and test/extenddb/test_conditional_writes.py
Batch split behavior: internal/dynodb/batch_ops.go

No repository test injects a true commit-acknowledgement network loss. Unknown-outcome handling is implemented and surfaced explicitly, while end-to-end fault verification remains an evidence gap.