@proofoftech/flowsafe

Durable approvals and physically isolated long-running execution for Mastra on Cloudflare.

Flowsafe runs Mastra workflows and agents through Cloudflare Durable Objects, stores their snapshots in D1, turns suspensions into human approval requests, and derives connector grants from stored decisions when a run resumes.

Documentation · Getting started · API reference · Live demo

Install

npm install @mastra/core@1.73.0 @proofoftech/flowsafe
Copy

Install @proofoftech/breakwater when resumed steps call approval-protected connectors:

npm install @proofoftech/breakwater
Copy

Guarded durable calls follow the durable call-option restrictions.

Install React and React DOM only when you import @proofoftech/flowsafe/approval-ui.

Compatibility:

• Node.js 22.13.0 or later (engine range >=22.13.0)
• ESM only
• TypeScript moduleResolution: "NodeNext", "Node16", or "Bundler"
• @mastra/core 1.73.0
• react and react-dom >=18 <20 (React 18 or 19) for the optional approval UI
• @proofoftech/breakwater >=0.17.0 <1.0.0 when used
• host-provided Wrangler >=4.118 <5 for the optional flowsafe-provision CLI

Choose an export

New host-side and React features are subpath-only. This keeps the root import free of Breakwater, durable-agent-host, and UI dependency graphs.

Start with the baseline Worker

Copy the reference deployment. It supplies:

• one Durable Object per run;
• D1-backed Mastra snapshots and approvals;
• bearer-authentication seam and actor resolver;
• strict D1 deployment sentinel plus authenticated Worker-to-Durable-Object calls;
• workflow catalog, start, status, resume, and idempotent termination routes;
• suspension-to-approval bridge;
• exact suspension-bound breakwater grant derivation;
• role checks and separation of duties;
• optional live approval/run WebSockets;
• alarm-owned deadline, SLA, and retention duties;
• structured audit and optional Queue-to-SIEM export;
• a sample gated workflow.

Replace the example workflow and identity verifier. Keep the trusted host-kit composition and grant provider.

import {
  approvalGrantProvider,
} from '@proofoftech/flowsafe/approval-api';
import {
  DurableObjectRunner,
  init,
  type RunnerRuntime,
} from '@proofoftech/flowsafe/do-runner';
import { approvalStoreFactoryFor } from '@proofoftech/flowsafe/host-kit';

export class AppRunner extends DurableObjectRunner<Env> {
  protected runOwnership(env: Env) {
    return approvalStoreFactoryFor(env.DB).resources();
  }

  protected build(env: Env): RunnerRuntime {
    const approvals = approvalStoreFactoryFor(env.DB).store();
    const { createWorkflow, createStep, runtime } = init(env, {
      requestContextForRun: approvalGrantProvider(approvals),
    });

    // Define and commit workflows through the import-swapped factories.

    return runtime;
  }
}
Copy

init() creates D1-backed Mastra storage from the conventional DB binding unless you inject storage. Workflow definitions use the same createWorkflow() and createStep() shape as Mastra. Flowsafe pins @mastra/cloudflare-d1 1.4.0 because the shipped D1 storage is written against that release's domain surface: it subclasses the adapter's background-tasks domain to apply the TaskFilter.resourceId predicate the adapter accepts but omits from its SQL builder, and hand-writes the schedules, notifications, and thread-state domains the adapter does not ship at all. D1StorageOptions.domains accepts host-supplied workflowDefinitions and knowledge domains, which have no D1-backed defaults. The pin also holds the adapter on its @cloudflare/workers-types v4 peer, which is the major Flowsafe still builds against.

Storage initialization (init()) adds ownerId and leaseExpiresAt to mastra_background_tasks with additive ALTER TABLE … ADD COLUMN statements. No manual migration is needed. Mastra's background-task manager claims tasks with a lease, renews it while running, skips live leases during recovery, and clears the lease on suspension. Recovery reclaims running tasks with expired or absent leases. Flowsafe continues applying the resourceId filter that the D1 adapter's listTasks omits.

If the deployment uses a table prefix, pass one shared constant to storage and host maintenance:

const storageTablePrefix = 'agent_';

const { runtime } = init(env, { tablePrefix: storageTablePrefix });
const worker = createFlowsafeWorker({
  ...workerConfig,
  storageTablePrefix,
});
Copy

The prefix may be empty or must start with an ASCII letter or underscore and continue with ASCII letters, numbers, or underscores. It can contain at most 39 characters because Mastra limits final table identifiers to 63 characters and mastra_workflow_snapshot uses the remaining 24. The exported low-level purge functions validate the same contract before preparing D1 statements. storageTablePrefix is not auto-discovered and must match the tablePrefix used by init() or createD1Storage().

Approval lifecycle

When a workflow or durable agent suspends:

1. The authoritative run summary captures stepPath, suspendedAt, and resumeCount.
2. The trusted host bridge creates one open approval in the deployment store.
3. The record's connector IDs and optional Mastra toolCallId come from a server-authored suspend payload.
4. A reviewer claims, delegates, approves, or rejects through the service or REST router.
5. A compare-and-swap commits the mutation.
6. An approval re-enters the owning runtime.
7. approvalGrantProvider() reads D1 and writes only the matching structured grants into the resumed leg's request context.
8. Another suspension creates a new approval. A completed or failed run ends the loop.

Durable-agent records produce tool-call scope. Workflow gates produce exact suspension scope. Trusted runScoped records produce explicit run scope. Grants never travel in public request bodies. A raw or forged resume finds no stored capability and fails at the Breakwater connector gate.

The same durable tool call may retry with its persisted toolCallId; a new model tool call requires approval. Legacy records without explicit scope and legacy connector ID arrays fail closed.

Terminate runs and enforce deadlines

The composed host mounts POST /runs/:workflowId/:runId/terminate. It transitions running, waiting, retry, and suspended runs to cancelled. The summary contains this exact structured reason:

{
  "status": "cancelled",
  "errorEnvelope": {
    "code": "CANCELLED",
    "message": "run was cancelled"
  }
}
Copy

Termination is idempotent. A retry by the same trusted principal returns the persisted terminal summary after ownership cleanup. An unrelated principal receives the same opaque 404 as any inaccessible run. A disputed economic settlement returns 409 with reason.code set to DISPUTED_SETTLEMENT before the runtime cancels active work. On FencedWorkflowsStorageD1 storage, a run that completes before a terminate records its intent keeps its result and the terminate answers 409. After the intent is recorded, a leg that completes, in the same object or on another instance, is recorded as cancelled, unless an earlier flowsafe version runs that leg during a deploy or a rollback. A leg whose resume recorded a disputed economic settlement removes the intent, and the terminate then answers 409 with DISPUTED_SETTLEMENT; an earlier flowsafe version running that leg can instead drop the dispute, and the run is recorded as cancelled with its result cleared. The intent does not stop a leg on another instance, so a cancelled run may have run every step, with its effects, and its summary carries no result: check a cancelled run's effects before running its work again. A terminate that keeps losing to a leg writing on another instance answers 503 and leaves the intent recorded: the leg keeps running, an approval resume of the run answers 409, and a retry of the terminate completes the cancellation. Retry it. Over a run snapshot that a flowsafe version earlier than 0.26.0 stored and SQLite cannot parse, a leg writing continuously, in the object or on another instance, keeps the terminate from recording its intent: it answers 503 with nothing recorded.

A run whose execution leg stops mid-step, because the platform ended the Durable Object invocation, does not stay running. A running leg marks its run row in D1 every 30 seconds. Once the row has gone six minutes without a write, the run's own object settles it on its next wake, and it never re-executes the interrupted step:

{
  "status": "failed",
  "error": "Run execution stopped mid-step before a durable outcome was recorded; external effects may have occurred. This run will not be automatically re-executed.",
  "errorEnvelope": {
    "code": "INTERRUPTED",
    "message": "Run execution stopped mid-step before a durable outcome was recorded; external effects may have occurred. This run will not be automatically re-executed."
  }
}
Copy

A run with a recorded cancellation or timeout completes that transition instead. A deploy does not interrupt a leg: the outgoing instance keeps running it, and its touches keep the run from being settled. When another instance terminates, times out or otherwise settles a run whose leg is still running, that leg's next touch aborts it: no further step starts, and the step in flight sees its abortSignal aborted. Pass a step's abortSignal to the external calls it makes so they stop too; a call the abort cuts mid-flight may already have taken effect, and its abort error does not show whether it did. Mastra re-runs a step with retries that throws after the abort, so such a step should check abortSignal.aborted before its side effects and throw MastraNonRetryableError (from @mastra/core/error) once it is aborted. A leg run by a flowsafe version earlier than 0.25.0 is not aborted at all, and one run by a version earlier than 0.26.0 is not aborted over a stored snapshot SQLite cannot parse, such as one nested more than 1,000 levels deep, so its next write lands over the settlement. A Workers runtime update gives in-flight requests at most 30 seconds, so it interrupts a longer leg. Without a connected client, a Durable Object invocation lasts about 15 minutes, so each step must finish within one. Split long work into steps, and wait without an approval rather than in an in-memory polling loop. Settlement needs the FencedWorkflowsStorageD1 workflow domain that createD1Storage() composes; terminate a stranded run on other storage. See interrupted legs.

A run whose leg returns state nested more than 1,000 levels deep does not stay running either. SQLite's JSON functions cannot parse such state, so FencedWorkflowsStorageD1 refuses to store it and the runtime fails the run at once, over the last state it stored, with errorEnvelope.code set to RUN_STATE_NOT_STORABLE. The step is not re-executed, so check its external effects. Start and resume refuse inputData, initialState, a request-context value or resumeData nested more than 256 levels deep with HTTP 400.

Economic settlement projections are trusted internal StartRunOptions.economicOperations and ResumeRunOptions.economicOperations inputs. The host fixes the projection at an execution-leg boundary before that leg becomes cancellable. Public start and resume bodies cannot set it, and Flowsafe does not expose a dynamic mid-leg update API.

The terminal snapshot commits before lifecycle cleanup. Cleanup abandons open approvals, settles an executing agent-schedule receipt as discarded, releases run ownership, and then records completion. It settles the receipt as discarded even when a leg on another instance did the run's work first, so a schedule fire recorded discarded does not show that the scheduled run did nothing. A crash between those steps reuses the terminal snapshot as its retry anchor. Hosts that compose the run route must override DurableObjectRunner.runLifecycle(); use createFlowsafeRunnerLifecycle() so approval cleanup, and the approvals the run object files on its own, use the same approval service and separation-of-duties configuration as the Worker.

POST /runs and the resume body accept an optional nonnegative deadlineMs. Resume replaces the prior deadline relative to the accepted resume leg. RunSummary.deadlineAt exposes the persisted epoch-millisecond deadline. The independent deadline maintenance duty sends a compare-and-swap request to the owner Durable Object and transitions each expired run once:

{
  "status": "timed_out",
  "errorEnvelope": {
    "code": "TIMED_OUT",
    "message": "run deadline expired"
  }
}
Copy

A timeout records its intent as a terminate does, so the cancelled caveat in the termination paragraph above holds for a timed_out run, including the check of its effects before its work runs again; unless an earlier flowsafe version runs that leg, a resume that moved the deadline, or a recorded disputed economic settlement, keeps the leg's outcome instead.

Monitor nextDeadlineAt, lastDeadlineAt, lastDeadlineAttemptAt, and lastDeadlineError on the maintenance health projection. The sweep is bounded by maintenance.deadlineLimit, which defaults to 100.

Time out one suspension

A suspended step can carry its own deadline. It arms one by adding the reserved SUSPENSION_DEADLINE_PAYLOAD_KEY to the payload it passes to Mastra's suspend(). When the deadline elapses before the awaited signal arrives, the run's own Durable Object resumes that step with a payload the step can tell apart from a real signal. createStep below is the import-swapped factory init() returns:

import {
  isSuspensionTimeoutResumeData,
  SUSPENSION_DEADLINE_PAYLOAD_KEY,
} from '@proofoftech/flowsafe/do-runner/constants';
import { z } from 'zod';

const gate = createStep({
  id: 'gate',
  inputSchema: z.object({ topic: z.string() }),
  outputSchema: z.object({ topic: z.string(), settledBy: z.string() }),
  execute: async ({ inputData, resumeData, suspend }) => {
    if (!resumeData) {
      return suspend({
        reason: 'awaiting approval',
        [SUSPENSION_DEADLINE_PAYLOAD_KEY]: 900_000,
      });
    }
    return {
      topic: inputData.topic,
      settledBy: isSuspensionTimeoutResumeData(resumeData)
        ? 'timeout'
        : 'signal',
    };
  },
});
Copy

Use isArmableSuspensionDeadlineMs(value) to validate relative milliseconds against the runner's safe-integer and inclusive duration bounds. Import it, those duration bounds and the reserved payload keys from @proofoftech/flowsafe/do-runner/constants to avoid loading the runner graph; the per-run cap MAX_SUSPENSION_DEADLINES_PER_RUN comes from @proofoftech/flowsafe/do-runner. A step declaring a Zod suspendSchema must declare the reserved fields it sets, the timer marker included, or use a loose object, because Mastra replaces the suspend payload with parsed output. Its resumeSchema must accept the timeout envelope as well as the signal shape.

For workflow tests, import suspensionTimeoutResumeData from @proofoftech/flowsafe/do-runner/testing and call it with { step, deadlineAt } and an expiry time. It returns the alarm's envelope shape; isSuspensionTimeoutResumeData checks that shape without authenticating its origin. Public resume requests containing the reserved key are rejected. An alarm resume keeps the run's recorded requester, so separation of duties applies to the next gate as it would without the timeout, and it grants no approval.

Import suspensionDeadlinesOf from @proofoftech/flowsafe/do-runner to inspect a RunSummary. It returns derived entries and rejected requests without scheduling a wake or mutating the summary. See the acceptance rules.

Only a top-level suspended step can arm a deadline. A step suspended inside a nested workflow is reported under the nested path while its suspension time is recorded against the enclosing step, so there is nothing to fence the resume against; the deadline is refused and logged instead of armed.

Wake precision is the Durable Object alarm's, and there is no maintenance-sweep backstop for it, so treat a suspension deadline as best-effort near its due time. Run-level deadlineMs remains the swept mechanism. MAX_SUSPENSION_DEADLINES_PER_RUN caps how many deadlines one run arms; the stored record, its parser, and the wake arithmetic stay inside the run's Durable Object, because nothing outside it can act on that state safely. Suspension deadlines apply to DurableObjectRunner-hosted workflow runs; durable agents keep their own resume path.

Wait without an approval

A step that only waits, such as a poll between attempts, suspends as a timer: it sets SUSPENSION_TIMER_PAYLOAD_KEY to true beside its deadline. The run's Durable Object resumes the step when the deadline expires, lists the step in RunSummary.suspensionTimers, and the approval bridges file no approval for it. A run that must outlast one Durable Object invocation, about 15 minutes without a connected client, waits this way between steps.

import {
  SUSPENSION_DEADLINE_PAYLOAD_KEY,
  SUSPENSION_TIMER_PAYLOAD_KEY,
} from '@proofoftech/flowsafe/do-runner/constants';

const poll = createStep({
  id: 'poll',
  inputSchema: z.object({ jobId: z.string() }),
  outputSchema: z.object({ jobId: z.string() }),
  execute: async ({ inputData, suspend }) => {
    // Checked on every resume, whatever resumed the step.
    if (!(await jobFinished(inputData.jobId))) {
      return suspend({
        [SUSPENSION_TIMER_PAYLOAD_KEY]: true,
        [SUSPENSION_DEADLINE_PAYLOAD_KEY]: 60_000,
      });
    }
    return { jobId: inputData.jobId };
  },
});
Copy

A timer can still receive an approval: the run object lists a step only while it can vouch for the wake, and every other case files an ordinary approval (see timer suspensions). So a timer step accepts an approval decision's { approved, comment?, decidedBy? } as well as the timeout envelope, in its resumeSchema and its branch, and re-checks its wait condition on every resume rather than treating any resume as permission to proceed. The run object files the approvals a timer leaves to a person through the reconcileApprovals hook that createFlowsafeRunnerLifecycle() fills; see approvals the run object files for when it files and what it leaves to a host status read. After every timeout resume it also attempts to close an approval still open for the suspension that resume ended.

The marker lives in the step's suspend payload, so a step that spreads untrusted data into that payload can make a gate wait without an approval for up to the deadline. A foreach with concurrency above 1 reports one payload for its suspended iterations, so one iteration's marker applies to the batch.

Read the Durable Object runner design for the suspension fence, the stored record, and the failure modes.

REST surface

The default approval base is /api/approvals.

GET  /
GET  /metrics
GET  /:id
POST /:id/claim
POST /:id/delegate
POST /:id/decide
POST /batch/decide
Copy

The create route is disabled by default. If enabled, it rejects capability, attribution, fingerprint, and resume-target fields.

Queue filters cover status, workflow, run, claimant, requester, strict creation-time bounds, bounded pagination, and reviewer-priority ordering. Batch decisions contain per-record outcomes and preserve all single-record checks.

Separation of duties

Self-decision is denied by default. A non-exempt actor also cannot approve a later gate after approving an earlier gate that advanced the run. Set a narrow role exemption only for an installation that cannot supply an independent reviewer.

SLA, notifications, and retention

sweepSLA() accepts the deployment store and escalates overdue open records. It is intended for a scheduled invocation and is not an HTTP service method.

ApprovalNotificationSink receives creation and escalation events. Failures are contained and audited. ApprovalStreamSink publishes mutations to the optional deployment hub.

purgeExpiredApprovals() deletes only approved and rejected records past the configured age. Pending, claimed, and escalated records remain live at any age.

Read the approval-system guide for exact grant binding and recovery behavior.

Approval dashboard

The optional UI exports a plain-HTML default and an injected component contract:

import {
  ApprovalApiClient,
  createApprovalDashboard,
} from '@proofoftech/flowsafe/approval-ui';

const client = new ApprovalApiClient({
  headers: {
    authorization: `Bearer ${token}`,
  },
});

createApprovalDashboard(document.getElementById('root')!, {
  client,
});
Copy

The package includes:

• queue, detail, metrics, filter, and batch-decision views;
• useApprovalDashboard() for headless composition;
• ApprovalUIProvider and ApprovalUIComponents slots;
• optional checkbox, select, toast, and presence slots;
• optimistic decisions with conflict reporting;
• polling reconciliation;
• WebSocket live merge and liveness heartbeat;
• ApprovalApiClient without a DOM dependency.

Flowsafe has no Astryx or CSS dependency. The repository showcase injects Astryx in the application only.

Physical deployment isolation

Flowsafe serves one organization per Worker, D1 database, and set of Durable Object namespaces. It does not provide a pooled multi-tenant mode.

Deployment identity fails closed

Provisioning writes the same stable tag to the DEPLOYMENT_TENANT binding and the singleton flowsafe_deployment D1 row. The Worker checks the pair before protected routes and maintenance work. Every Worker topology stamps DEPLOYMENT_IDENTITY_SECRET on Durable Object requests; the target compares the credential and validates its own tag and sentinel before reading storage.

Seed the sentinel before application migrations or traffic. First-time provisioning refuses any database that already contains application tables. A missing binding, malformed sentinel schema, non-singleton row set, malformed tag, credential mismatch, or tag mismatch returns 503 at the Worker or prevents Durable Object initialization.

The package ships the flowsafe-provision CLI. Wrangler is not a package peer. The CLI resolves a host-provided Wrangler >=4.118 <5 from the application instead of downloading it at runtime:

npm install --save-dev "wrangler@>=4.118 <5"
npx flowsafe-provision \
  --database <database> \
  --tag <tag> \
  --initial-fence-state open \
  --remote \
  --config wrangler.jsonc
Copy

Fleet control planes can import the same fail-closed sentinel implementation from @proofoftech/flowsafe/deployment-identity-protocol instead of duplicating its schema or race handling.

Fence and drain a deployment

One Flowsafe deployment is one tenant, so the execution fence controls the complete deployment. open admits all work. draining refuses new run mints and future-work authoring while existing runs and deliveries finish. It still accepts new background-task enqueues and dispatches queued tasks because both are drainable work. Background-task enqueue, dispatch, and stale-task re-drive are refused in migration-locked and proof-only. Signal wakes that would mint a run persist instead. migration-locked refuses execution, and proof-only admits only the nominated start and its bound run. The fence never preempts compute already in flight.

Provisioning requires --initial-fence-state open or --initial-fence-state migration-locked; it never chooses a default. An absent pre-0.20 table or empty five-column legacy table reads as optional open. Initialization adds epoch/revision metadata without reopening existing state. A missing row once any metadata column exists is unreadable and is never silently refilled. For locked-at-birth provisioning, read GET /admin/execution-fence afterward and fail unless it reports migration-locked.

Administrative readings include mutationEpoch, requireMutationEpoch, and transitionRevision. Upgraded commands compare expected state, epoch, and revision; exact retries preserve proof bindings and timestamps while that command remains the last applied command. Advancing the epoch also sets its sticky requirement; ordinary lock, proof, and reopen transitions preserve both. Upgrade the deployment's writers and configure their trusted artifact epoch before activation. See the administration contract for request fields, compatibility, and conflicts.

The do-runner and host-kit entry points export normalizeRunExecutionIdentity, normalizeD1RunExecutionIdentity, normalizeStartIdentity, normalizeStartExecutionIdentity, and mutation-epoch validation/header helpers. They copy validated identity data without authenticating it. String D1 prefixes normalize to lowercase; null explicitly means no D1 namespace and differs from the empty default prefix.

New reservations are unbound until an exact winning claim or an observed result binds their physical execution. Runtime generates a separate execution token for each new run generation, and proof-only re-entry compares its namespace, workflow, run and generation. Stored proofExecution stays server-side and is omitted from admin JSON.

D1 schedule mutations enforce the captured caller epoch at their final SQL boundary. A fenced custom facade must provide FENCED_SCHEDULE_STORAGE on the same database as its configured fence. Direct D1 methods require batch() and accept a trailing MutationEpochContext; omission refuses after activation. Pause/delete and HTTP no-ops retain epoch checks. Use the schedule guide for fixed pause/resume methods, custom facade compatibility and uncertain-write outcomes.

With a configured fence, Runtime requires the actual D1 domain's positive initial-write witness before engine entry. Without a fence, capable D1 keeps its real namespace and ordinary persistence behavior; custom storage explicitly asserts no D1 namespace. Managed hosts persist preparation journals and require a matching nonpending durable outcome before acknowledging execution. A valid pending generation returns RUN_START_PENDING with status 503; keyed retries use the idempotent-start refusal contract. See the runner design for exact claims, replay and recovery.

Use GET /admin/inventory while the fence remains draining. A drain is proven only after every work category is empty across two complete sweeps at least 60 seconds apart. A pending agent-inbox notification scheduled for later, or carrying no due timestamp, is work and keeps the proof open. Readings are point-in-time observations rather than snapshots and can move in either direction while draining admits work. Empty results cannot over-count, and keyset pagination never skips a row that existed before the sweep began. If you need a hard guarantee, re-sweep once after transitioning to migration-locked: an empty post-lock sweep is conclusive; a non-empty one means work is still outstanding, either because it entered after the proof or because the lock parked it before it finished. Return to draining and repeat the proof. An inventory read taken under migration-locked measures what the fence parked rather than what the deployment would otherwise be doing. Schedules and signal subscriptions are standing configuration and need not empty. Persisted idle signals are deliberately unenumerable and carry into the replacement deployment.

Continue verified remote results

Trusted hosts can continue a retained workflow result through the owning RunDO. The opt-in private route admits the verified result once and preserves uncertain claims across restarts. It does not enable automatic engine replay.

Runtime ids are opaque

The host mints opaque, path-safe run and thread ids. RunnerRuntime.start() requires a host-owned run id and has no generation fallback. A run id is single-use: never start one again, even after retention removed its run. The id scopes the snapshot, Durable Object, approval lookup, stream address, and artifact path, but it carries no customer identity.

Callers that need exactly-once start behavior supply an idempotencyKey, never a run ID. The key is available on POST /runs, trusted agent-host starts, and streamUntilPersisted(). A retry returns the same persisted run. IDEMPOTENT_START_PENDING includes pendingSince; re-probe the point-in-time IDEMPOTENT_START_UNRESOLVABLE result before acting. For an agent start, keep re-probing for at least seven minutes after its last activity before choosing a fresh key: until the thread object repairs a start whose leg stopped or runs on another instance, its replay answers IDEMPOTENT_START_UNRESOLVABLE. A key remains valid until its reservation-retention horizon expires.

Pass application context at start

An authenticated POST /runs can supply a non-reserved application context:

{
  "workflowId": "publish",
  "inputData": { "topic": "release notes" },
  "idempotencyKey": "publish-request-123",
  "requestContext": { "app.agentId": "agent_123" }
}
Copy

requestContext must be an object when present. Invalid shapes and reserved keys return HTTP 400 with reason: "reserved-context-key". The router validates it after authentication and workflow authorization, then passes it to the host's beforeStart policy. Use that hook for application-specific attribution rules; possession of a run token does not establish an arbitrary context value's business meaning.

The protected topology carries this value into storedRequestContext. Provider application values override stored values, while trusted execution identity and capabilities retain their authority. Verified schedule targets supply their own context, including an omitted value. Persisted application values survive resume; a keyed replay retains the winning run's context and still performs validation and host policy checks. See the request-context guide and host policy signatures.

Stores are deployment-wide

D1ApprovalStoreFactory.store() and D1SubscriptionStoreFactory.store() return the store for the bound database. Tables and indexes contain no tenant column. Legacy pooled schemas require a fresh database.

D1ApprovalStoreFactory.resources() records ownership of server-minted runs, threads, and schedules plus validated host-owned resource keys. Operators and builders can access only their principal's resources; reviewers and viewers can read existing resources; admins can administer them. Inaccessible resources return 404 before role errors.

deploymentTag is audit attribution only. It comes from verified infrastructure and never authorizes a request or scopes a query.

Memory and retention

Mastra memory ids are caller-chosen by default. A Flowsafe host must:

• mint threads with ActorContext.newThreadId();
• validate host-owned resource keys with ActorContext.resourceIdFromKey(key);
• reject client bodies that name memory ids;
• reach a thread Durable Object only through createThreadTopology().

TTL retention helpers cover:

• terminal workflow snapshots;
• decided approvals;
• idle threads and their messages;
• terminal notifications;
• thread state and goals;
• schedule trigger history;
• terminal background tasks.

Schedules, resources, working memory, and provider subscriptions are standing state. The schedule and subscription routes support authorized deletion. Flowsafe does not ship a public resource or permanent thread-deletion route: a host that adds one must remove every authoritative binding and wake source before it calls ActorContext.releaseResource() for the thread or resource claim. Run retention and schedule deletion release their ownership claims; idle-thread retention deliberately does not. Deployment decommissioning removes any remainder. TTL retention, explicit teardown, and decommissioning are separate lifecycle mechanisms.

Artifacts

R2ArtifactStore uses keys shaped as:

[prefix/]workflowId/runId/name
Copy

Workflow and run ids use the runner's path-safe pattern. Artifact names are validated segment by segment. deleteRun() pairs with terminal-run retention. Deployment decommissioning removes the remaining bucket data.

Construct the retention store from the current maintenance invocation's environment. A module-scoped Worker configuration can safely retain the factory, but must not retain an invocation-specific R2 binding:

const worker = createFlowsafeWorker({
  ...workerConfig,
  artifactStore: (env) => new R2ArtifactStore(env.ARTIFACTS),
});
Copy

The factory and artifact deletion run inside the snapshot-retention failure boundary. Either failure preserves the snapshot row for retry and does not stop sibling maintenance duties. Omitting the factory or returning undefined keeps row-only retention.

InMemoryArtifactBucket is available for tests and offline demos.

Audit export

queueAuditSink(queue) turns the shared audit contract into Cloudflare Queue messages. createAuditQueueConsumer() batches records as NDJSON, posts them to the configured collector, acknowledges the batch on 2xx, and retries otherwise.

For an externally authored fleet release, do not expose that shared Queue binding to the candidate. Bind the candidate to the trusted state Worker as AUDIT_PROXY and enable the matching candidate and trusted-ingress fleet markers. The adapter authenticates with the deployment identity; trusted state supplies canonical tenant, environment, and script attribution before enqueueing. The envelope marks the candidate event itself as untrusted, so its action, decision, resource, and detail remain claims rather than infrastructure attestations.

The types are structural and do not require @cloudflare/workers-types. A transform seam can map internal events to your SIEM schema.

Live streaming

Live updates are opt-in behind a HUB Durable Object binding and STREAM_TICKET_SECRET.

An authenticated REST request mints a short-lived HMAC ticket. The browser presents it when opening a queue or run WebSocket. The Worker is the only verifier. The singleton hub and each run Durable Object rebind the addressed channel through their own identities.

Tickets carry addressing information only. They contain no connector grant or approval decision. Polling remains available as fallback and reconciliation.

Long-running agents

The following surfaces are supported and opt-in: they are tested and covered by package compatibility guarantees, but the host must explicitly wire the required routes, bindings, storage domains, or scheduled duties.

Durable agents

Use @proofoftech/flowsafe/agent-host for a public protected surface. createAgentRouter() lists server-owned metadata and exposes authenticated start, status, and newline-delimited JSON observation routes. The router mints every ID and rejects trusted context and execution overrides. Authenticated human starts must satisfy both the global start roles and the selected agent's allowedRoles. Automated entry uses trusted host paths instead: it never consults human roles and requires a matching principal kind and entry path in the selected agent's allowedAutomation.

Agents can add AgentMeta.requiredPermissions as an all-of list of Permission identifiers. Each identifier uses canonical lowercase dotted form. Catalog construction rejects a non-array, an empty list, duplicates, and malformed identifiers.

Configure ThreadAgentHostOptions.resolvePrincipalPermissions with a server-owned PrincipalPermissionResolver. The resolver receives only the trusted ExecutionPrincipal and returns a PrincipalPermissionResolution containing effective permissions and policyVersion. The host evaluates permissions after the existing human-role or automated-entry gate. A missing resolver, a resolver failure, or malformed output denies an agent that requires permissions.

A configured resolver runs on every authorized entry, and its resolution is projected into the run's derived request context as breakwater.principalPermissions on every start and resume leg (an explicit null when no resolution exists). Breakwater connectors that declare PermissionManifest.requiredPermissions enforce their own all-of list against that projection before their dry-run branch and approval grant. Agents that use only allowedRoles and allowedAutomation invoke a configured resolver but do not require it: a failed resolution still starts the run, is audited as agent.permissions.resolve, and leaves the projection null, so permission-declaring connectors inside that run fail closed.

Permission authorization audit detail records requiredPermissions and permissionPolicyVersion; it does not record effective permissions or identity-provider groups.

createThreadAgentHost() validates Breakwater's guarded-handle brand and versioned host protocol before it registers the agent with Mastra. It persists the thread/agent binding and original run principal, so eviction recovery and approval resume cannot switch agents or actors.

Agent resume is approval-only. createAgentApprovalResumer() rejects legacy agent targets without the original principal, then rechecks that principal against the current catalog. A human must still satisfy the selected agent's allowedRoles. An automated principal's kind must still appear in allowedAutomation; approval.resume is implied for a declared kind. The thread host then enforces any requiredPermissions through the current resolver policy. The resumer delegates non-agent workflow records to the existing resume function.

createFlowsafeDurableAgent() remains the lower-level compatibility API. It routes Mastra's durable-agent workflow through RunnerRuntime, but it does not guard an arbitrary raw agent. Each durable entry snapshots data-property call options before validating or delegating and rejects accessor-backed options. When passed a Breakwater guarded agent, it rejects structured output on stream, generate, and prepare because Mastra's durable runner bypasses the narrow handle. Use agent-host when an HTTP surface must enforce catalog and Breakwater invariants.

Agent event replay lasts only as long as the configured Mastra cache. The default in-memory cache does not survive process restart. A 409 stream response means the client must read the authoritative status route.

Signals and notifications

createThreadSignalRoutes() hosts message, queue, signal, state, and notification delivery in the thread Durable Object. createSignalRouter() authenticates Worker requests before reaching those routes. Its optional validateThreadTarget uses the existing BoundThreadTargetValidator type for host-specific binding or ownership restrictions. See the signal-ingress guide for strict-owner composition.

Signal acceptance is audited after the downstream response succeeds. Thread refusals with status 404 use a consistent public response across registry, validator and downstream checks. An audit-sink failure retains the selected response.

The thread routes reject a signal whose tagName is not an XML name, and drop an attributes object carrying a value Mastra cannot render or a key that is not an XML name. Both are what core would otherwise throw on while rendering the signal inside the agent turn.

Configure ThreadSignalRoutesOptions.contentPolicy when signal content needs a domain policy before it becomes model input. The Thread Durable Object invokes this structural callback for direct ingestion, provider delivery, schedule fires, and notification dispatch. Its text is Mastra's canonical escaped XML representation. A denial stops direct delivery with 422, settles a scheduled fire as discarded, or terminally discards the affected notification.

A policy failure is opaque: direct delivery returns 503, schedule state is left unsettled so the lease expires and a later tick retries, and notification dispatch records a bounded retry. A webhook whose matched deliveries the deployment could not decide is answered with 503 so the provider's own at-least-once redelivery recovers it; each delivery carries a dedupe key derived from the signed bytes and the subscription, so a redelivery coalesces into a still-pending row rather than duplicating it. A content denial is terminal and answers 2xx, because redelivering the identical bytes would only be denied again. Poll deliveries report the same three outcomes and depend on the adapter re-reporting state it has not seen accepted. Give a network-backed policy its own timeout and failure budget inside the callback; FlowSafe imposes neither.

Configure the same positive safe-integer maxDeliveryAttempts on createNotificationDispatchTick() and createThreadSignalRoutes(). Both default to DEFAULT_MAX_NOTIFICATION_DELIVERY_ATTEMPTS. A recorded failure at the bound discards the notification with deliveryReason: "delivery-attempts-exhausted", retaining its count and last error. An already-exhausted row is never sent; its discard is conditional on the observed record remaining current. Below the bound, existing retry delays apply.

Dispatch requires NotificationDeliveryStorage; D1NotificationsStorage implements it. A custom store's updateNotificationDeliveryIfUnchanged() must compare the captured observation and apply the failure patch atomically against its other writers. Ordinary Core storage still serves notification ingestion. See the delivery and receipt guide for custom-store migration, lost responses and receipt retention.

Summary items arise only from rows written by Mastra's delivery policy, which means an unbranded agent's owner notifications. Summary source counts and a configured source delivery policy read own properties: the declared @mastra/core peer guards both lookups, so a source named after an Object.prototype member is counted under its own key and resolves the configured priority or default action. A core outside the declared peer range that predates mastra-ai/mastra#23693 and mastra-ai/mastra#23694 miscounts such a source and selects the inherited policy entry instead.

Adapt Breakwater without adding a FlowSafe runtime dependency on it:

import { RequestContext } from '@mastra/core/request-context';
import {
  ACTOR_CONTEXT_KEY,
  AGENT_AUDIT_CONTEXT_KEY,
  assertAcceptedCallProviderOptions,
  createContentPolicyGate,
  providerOptionsCarryContent,
} from '@proofoftech/breakwater';
import {
  breakwaterActorFor,
  principalAuditFields,
} from '@proofoftech/flowsafe/approval-api';
import {
  createThreadSignalRoutes,
  type SignalContentPolicyInput,
} from '@proofoftech/flowsafe/signals';

const inspectContent = createContentPolicyGate({ policies });

function signalPolicyContext(input: SignalContentPolicyInput): RequestContext {
  const requestContext = new RequestContext();
  requestContext.set(ACTOR_CONTEXT_KEY, breakwaterActorFor(input.principal));
  requestContext.set(AGENT_AUDIT_CONTEXT_KEY, {
    agentId: input.agentId,
    ...(input.deploymentTag === undefined
      ? {}
      : { tenantId: input.deploymentTag }),
    ...(input.runId === undefined ? {} : { runId: input.runId }),
    threadId: input.threadId,
    ...(input.resourceId === undefined
      ? {}
      : { resourceId: input.resourceId }),
    entryPath: input.entryPath,
    ...principalAuditFields(input.principal),
  });
  return requestContext;
}

const signalRoutes = createThreadSignalRoutes({
  // Existing server-owned resolvers and dispatch seams.
  notificationDispatchAllowed: (scope, agentId) =>
    host.notificationDispatchAllowed(scope, agentId),
  scheduleProviderOptionsPolicy: ({ providerOptions }) => {
    try {
      if (providerOptionsCarryContent(providerOptions)) {
        return { allowed: false, outcome: 'denied' };
      }
      assertAcceptedCallProviderOptions(providerOptions);
      return { allowed: true };
    } catch {
      return { allowed: false, outcome: 'denied' };
    }
  },
  contentPolicy: (input) =>
    inspectContent({
      text: input.text,
      requestContext: signalPolicyContext(input),
    }),
});
Copy

Build the request context only from the callback input. FlowSafe derives those fields from the asserted Thread Durable Object scope and server-owned metadata; never merge HTTP body context or caller-supplied identity into it.

A non-owner's notification, and an unbranded agent's owner notification, are checked before the inbox write, so a denied one is never recorded. Storage owns the id, the timestamps and any coalescing, so ingestion inspects a canonical prospective notification carrying every untrusted model-visible field, as the individual signal and as the single-record summary. An owner notification to a runtime-driven agent is delivered at ingestion instead: the route records it, checks the exact individual signal built from the stored row, and sends only that signal; a denial discards the row. Dispatch checks the exact persisted individual or summary signal for each record it delivers.

D1NotificationsStorage and D1ThreadStateStorage mirror Mastra's in-memory domains on D1. SignalClient is available from the browser-safe signals/client export.

Goals

createObjectiveRouter() manages one durable objective per thread through Mastra's own goal read/write shape. Writes are role-gated, size- and run-budget-capped, and audited.

Schedules

D1SchedulesStorage implements the schedule domain. createScheduleRouter() provides deployment-wide role-gated CRUD and read-only trigger history. createScheduleTick() claims due fires with compare-and-swap (CAS), protects reserved context keys, consults the configured unattended-run cap, mints an opaque run id, and starts a workflow or agent through the correct topology. Without the optional cap seam, unattended starts are uncapped.

Background tasks

createBackgroundTaskD1Domains() supplies serialized workflow and deployment task storage. BackgroundTaskHost owns one singleton manager Durable Object, while createBackgroundTaskRoutes() exposes its host surface. Terminal task cleanup is opt-in through the composed Worker.

Signal providers

Flowsafe providers route polling and webhook deliveries through the thread topology instead of Mastra's in-process registry.

SignalProviderHost runs one alarm-driven host per deployment. D1SubscriptionStoreFactory persists subscriptions. External resource ids are opaque, but must be non-empty, contain no ASCII control characters, and fit within 1,024 UTF-8 bytes. A subscription metadata value nested more than 256 levels deep is refused with 400. Human-only subscription routes reconcile provider alarms after each committed mutation. Webhook routes verify raw bytes before parsing, look up the stored subscription, apply a provider rate cap, and bound forgery audit.

githubSignalProvider() is the reference provider.

Complete wiring is in the durable-agents guide and advanced starter.

Deployment and operations

The composed createFlowsafeWorker() owns the shared route and maintenance-duty pipeline. Hosts inject workflows, identity verification, topology-backed optional routers, budget wrappers, notification transport, an invocation-scoped artifact-store factory, the storage table prefix, schedule and notification ticks, and extra purge duties.

Configure mutationEpoch with a nonnegative safe integer or a synchronous environment callback. The Worker captures it before deployment verification or authentication, then forwards it through trusted contexts and protected internal headers. Start paths preserve the original actor, principal, epoch and selectors across waits, including class-backed context method receivers. Do not accept the epoch from public headers or start JSON.

The internal durable-agent host start requires an eighth AgentStartAuthority argument, with an explicit onPreparedStartIdentity property. This transport does not activate final-write epoch checks or managed recovery; the deployment reference and durable-agents guide describe the boundary.

Protect GET and POST /admin/execution-fence plus GET /admin/inventory with a distinct MAINTENANCE_ADMIN_SECRET. Fence transitions use CAS and return 409 with FENCE_CAS_CONFLICT when the expected state is stale. Fenced execution returns 503 with an EXECUTION_FENCED reason. The agent-host and stream routers answer every refusal with its own status, message and reason. The readers of a run or thread object's answer replace the message of a 5xx that carries no reason code with internal error and log the original; a RunRouteError a host authors is forwarded as written.

Every leaf option that accepts ExecutionFenceWiring requires an explicit store or 'none'. Run-router, agent-thread-topology, and storage initialization also require explicit start-idempotency wiring. Use 'none' only when no database exists. BackgroundTaskHost no longer exposes its manager; call enqueue(), getTask(), listTasks(), or stream() on the host and use BackgroundTaskReads for read-only route composition.

createFlowsafeMaintenanceDurableObject() runs deadline expiry, approval service-level agreement (SLA) sweep, retention purge, and optional schedule fire and notification dispatch as separate alarm invocations. Provider polling and background recovery use their own Durable Object alarms.

Read:

• Deployment reference
• Operations runbook
• Security threat model

Security boundaries

Flowsafe does not authenticate users, execute an identity-provider protocol, provide a network sandbox, or decide which business action is safe. It supplies enforcement seams after your host has verified identity and selected policy.

Critical host obligations:

• deploy one Worker, D1 database, and set of Durable Object namespaces per organization;
• seed flowsafe_deployment before application migrations, configure DEPLOYMENT_TENANT, and keep the values identical;
• set a distinct DEPLOYMENT_IDENTITY_SECRET for internal Durable Object calls;
• set a separate MAINTENANCE_ADMIN_SECRET, bootstrap the fixed maintenance singleton after deploy, and monitor its alarm status; for externally authored releases, keep that secret only in trusted state and relay short-lived signed capabilities through the candidate;
• derive actors and roles from verified credentials;
• mint run, thread, schedule, and subscription identities server-side, validate host-owned resource keys, and preserve their principal ownership records;
• keep connector lists and agent resume targets server-authored;
• persist the original authorized agent requester and resume execution as that principal;
• expose decisions only through the approval path;
• configure shared stores for cross-isolate connector budgets;
• keep raw Durable Object namespaces behind the exported topologies;
• schedule every enabled domain's retention path and define deployment decommissioning;
• protect model, provider, stream-ticket, webhook, and SIEM secrets separately.

Verification

The repository runs a deterministic workerd spike against the package. Durable Object conformance suites separately cover wrong and missing internal caller credentials.

pnpm --filter @proofoftech/flowsafe spike:verify
Copy

The optional live-model proof requires SPIKE_LLM_MODEL_ID and SPIKE_LLM_API_KEY:

pnpm --filter @proofoftech/flowsafe spike:verify:llm
Copy

License

Apache-2.0.