Hosting and identity
What the server stores, who owns it, and how a host adds accounts, storage and product hooks.
This page collects what an operator or an embedding host needs beyond a first start: what the server stores and who may read it, how assets are reclaimed, which settings stop the server, how requests are attributed to owners, and the extension points a host registers from instrumentation.ts. For installing and upgrading, start with Deployment; for models, Configuration.
Docker Compose as a personal installation
The repository's Compose file is set up as a personal installation:
- One owner.
docker-compose.defaults.envturns on single-user mode: every request resolves to one owner, so every browser sees the same course library and publishing works. No anonymous cookie is minted. - Two containers. The app and PostgreSQL; the app starts once PostgreSQL reports healthy.
- Loopback only. The app is published on
127.0.0.1:3000, so only this machine can reach it. PostgreSQL is not published at all.
To reach it from other machines, protect it first:
- Set a long random
ACCESS_CODEin.env.local(see ACCESS_CODE). This is strongly recommended: without it, anyone who can reach the port is the single owner and shares, edits and can delete the whole library. - Set
PERSISTENCE_POSTGRES_PASSWORDto a random value of letters and digits before the first start (for an existing volume, see PostgreSQL password). - Publish on the network address:
OPENMAIC_PUBLISH_ADDRESS=0.0.0.0 docker compose up -d --build.
These Compose-level variables (OPENMAIC_PUBLISH_ADDRESS, OPENMAIC_PORT for
the host port, PERSISTENCE_POSTGRES_PASSWORD) come from your shell or a .env
file next to docker-compose.yml, not from .env.local. Single-user mode
without ACCESS_CODE logs a prominent warning at startup, and the app also
warns when it is published beyond loopback with the default PostgreSQL
password; neither stops the server. A later first-run setup flow may prompt for
an access code; until then, setting ACCESS_CODE is up to you.
Each default in docker-compose.defaults.env can be overridden in .env.local,
which Compose reads after it: for example OWNER_SINGLE_USER=false for one
anonymous owner per browser (what pnpm dev does), or to use
PERSISTENCE_SHARED_OWNER_ID instead, or your own DATABASE_URL for an
external database. The bundled postgres service still starts in that case
(the app waits for its health check) but is not used; remove it from a copy of
the Compose file if you do not want it.
Upgrading a Compose deployment
docker compose up now starts
PostgreSQL, the app always stores courses there, and the app is published on
127.0.0.1 only.
- If you served the app to other machines, start with
OPENMAIC_PUBLISH_ADDRESS=0.0.0.0, and setACCESS_CODE: every visitor is now the same single owner. --profile server-persistenceis still accepted and changes nothing; PostgreSQL always starts.- Courses an earlier browser-only deployment stored in the browser are not deleted: the first time each browser opens the upgraded app, a one-way importer moves them to the server automatically (see Upgrading from a browser-only build) and leaves the browser copy untouched.
- Courses an earlier server-backed deployment stored under each browser's
anonymous cookie stay with those anonymous owners: nothing is merged into
the single owner automatically. To bring them in, claim them explicitly
(see Single-user mode). If several people used that
deployment, consider
OWNER_SINGLE_USER=falseinstead, so each keeps their own library. - A
DATABASE_URLin.env.localstill wins (an external database, or a password you rotated); without one, the app uses the bundled PostgreSQL withPERSISTENCE_POSTGRES_PASSWORD. - If
.env.localsetsPERSISTENCE_SHARED_OWNER_ID, also setOWNER_SINGLE_USER=falsethere: the two exclude each other and the app refuses to start with both. - There is no browser-storage-only image any more: the
NEXT_PUBLIC_PERSISTENCEbuild argument is gone and ignored.
What the server stores
OpenMAIC always stores courses on the server. The
Docker deployment runs exactly two containers, the
OpenMAIC app and PostgreSQL. The persistence HTTP server is embedded in the app
at /api/persistence; there is no standalone persistence service.
DATABASE_URL is required. Without it the server does not start: it prints
[boot] Invalid server configuration; the server will not start: DATABASE_URL is not set. ... with the fix and exits with code 1. Outside Compose, build
normally and run with a DATABASE_URL:
pnpm build
DATABASE_URL=postgres://openmaic:password@localhost:5432/openmaic pnpm startFor local development, pnpm db:up starts a separate development database
(the Compose postgres service definition under its own project and volume,
openmaic-dev-db, shared by every checkout on this machine) and publishes it on 127.0.0.1 (port OPENMAIC_DB_PORT, default 5432); the
matching DATABASE_URL is commented in .env.example, and pnpm db:down stops
it again.
Configure models as usual (openmaic.yml with keys in .env.local, or Settings → Model Services). Course documents, folders,
chat history and learner runtime sessions, and generated media are stored on
the server. What stays in the browser is what belongs to the device and can be
lost without losing a course: app settings and UI preferences, the playback
position and the editor's current scene, the editor's undo history, a local
cache of narration and media the server already stores (and bytes a full store
refused, kept for a retry), PDF images staged during generation, and TTS voice
profiles registered from that browser. Settings → Clear Local Cache clears
exactly that and nothing on the server.
Upgrading from a browser-only build
Courses an earlier browser-only build
stored in the browser move to the server automatically, with no action and no
UI: the first time that browser opens the upgraded app, once the page is idle,
a one-way importer copies each course, with its chat, learner runtime, playback
position, agent roster, folders and membership, quiz progress and media, to the
owner the server resolves for that browser (the anonymous cookie owner by
default), and the course appears in the library. This happens once per browser:
the server binds the browser to the first owner that asks
(POST /api/identity/legacy-import-binding), and a claim carries the binding to
the account (an anonymous owner that signs in and is claimed). Every importer
request carries the browser's id and is refused
(409 LEGACY_IMPORT_NOT_BOUND) for any owner that does not hold the binding, so
another owner that later uses the same browser gets nothing imported. The browser copy is left
untouched, and Settings → Clear Local Cache does not delete it. A course the
server already has for that owner stays as the server has it; one whose id
another owner holds is imported under a new id; one deleted on the server is not
brought back. The importer records its progress in the browser (with a random browser id and
no owner information), so an interrupted import resumes on a later load and nothing is imported twice; problems are
logged in the browser console under [legacy-browser-import]. The importer is
temporary and will be removed a few releases later.
Course library and agent runtime
The server course library and its folders (/api/stages/**, /api/folders/**)
serve whether or not the agent runtime
(OPENMAIC_AGENT_RUNTIME_ENABLED) is on. GET /api/agent/runtime reports
persistence, next to the runtime's own enabled and runtimeEnabled.
Who can read and write stored data
Every /api/persistence request is attributed to the owner the
owner identity seam resolves — by default the
anonymous cookie (400 days, renewed while in use), one owner per browser; in the Compose deployment, the one
single-user owner. There is no separate persistence
credential:
- Documents. A read is capability-by-id: if the stage meta exists and is
not tombstoned,
decideDocumentAccessallows it with no owner check (lib/persistence/document-access.ts), so anyone who can reach the endpoint and knows a stage id can read that course. Writes and deletes are owner-checked. - Runtime sessions (
/runtime/*) are partitioned by learner key, and the learner key is the owner id. The browser learns it fromGET /api/persistence/learner-key; a request naming any other learner key is refused (403 FORBIDDEN_LEARNER), and another learner's session answers404. Runtime of a deleted (tombstoned) course reads as absent and takes no new writes. Learner merge and the admin wipes stay refused. - Assets are allocated in a per-owner partition, so
ASSET_QUOTA_BYTESis a per-owner ceiling and only the owner can replace or delete an entry. Reads stay capability-by-id for media a course viewer needs: an owner reads its own entries, and anyone reads another owner's committed entry while a live course of that same owner references it. A course only references (and commits) its owner's own media: naming another owner's asset id in your course records nothing, so it can neither expose their unsaved uploads nor keep their media alive. Entries written before per-owner partitions (the old shared partition) stay readable by id to everyone, and can be replaced or deleted only by an owner who owns every course referencing them; the collector reclaims them as courses stop naming them, as before.
Without a host auth method the owner is only as strong as a cookie (or, in
single-user mode, as ACCESS_CODE or the loopback binding): this is suitable
for localhost, trusted-network, or single-team deployments. A
deployment with its own accounts registers owner auth methods (see
Owner identity) and every surface above follows it.
Upgrading server persistence
PERSISTENCE_DEV_TOKEN,
NEXT_PUBLIC_PERSISTENCE_TOKEN and PERSISTENCE_ALLOW_INSECURE_DEV_AUTH are
removed and ignored; drop them from your environment and build arguments.
Runtime sessions written before this change are keyed by a learner key the
browser minted, not by an owner id, so they are no longer reachable (course
documents and media are unaffected). They are not migrated automatically,
because trusting a client-supplied old key would bring client-chosen identity
back.
If PERSISTENCE_DEV_TOKEN was your only access gate, act before upgrading.
Without it the endpoint answers every visitor who reaches it, each as their
own anonymous owner. Put the deployment behind ACCESS_CODE or your own
gateway, or register owner auth methods backed by your accounts (see
Owner identity).
PostgreSQL password
PERSISTENCE_POSTGRES_PASSWORD (default openmaic-dev, for local use only)
initializes the PostgreSQL role only when the data directory is empty, and
the default DATABASE_URL in docker-compose.defaults.env is built from the
same variable without encoding, so use letters and digits only (characters
such as @, /, # or ? break the URL; for such a password, set an
encoded DATABASE_URL in .env.local instead). Changing it later does not rotate an existing
openmaic-postgres volume. For a disposable local database, run
docker compose down -v, set the new password, then start again. To preserve
data, run
docker compose exec postgres psql -U openmaic -d openmaic -c "ALTER ROLE openmaic WITH PASSWORD 'new-password';",
then start with PERSISTENCE_POSTGRES_PASSWORD=new-password (or set the
matching DATABASE_URL in .env.local).
Asset storage
Collection
Assets are reclaimed by an offline collector rather than on a request path.
This deployment runs that collector by default, so nothing has to be
configured for asset storage to stop growing. A pass runs every
ASSET_COLLECTION_INTERVAL_MS (default 15 minutes) and has two levels. It first
releases registry entries — an allocation no document claimed before its pending
window ran out, and an entry whose last document reference left longer ago than
ASSET_COLLECTION_GRACE_MS (default 1 hour) — and then deletes the bytes whose
last entry left, after the same grace. The two levels wait in sequence:
releasing an entry is what leaves its bytes unreferenced, so the bytes start
their own grace only once the entry has served its. The worst case from "the
last document stopped naming this" to "the bytes are gone" is therefore two
grace periods, not one. That window is the retention a user's deleted media
actually gets, so raise it deliberately. Set
ASSET_COLLECTION_ENABLED=0 to switch collection off in a process. A
horizontally scaled deployment may leave it on in every instance — each row is
locked and re-checked before anything goes, so concurrent collectors serialize
rather than race — or disable it everywhere and run its own.
The server owns that bookkeeping end to end, and it needs no configuration because it is not optional here: every document write records which assets the document names and commits the allocations it names, which is exactly what the collector reads. A browser never deletes an asset and is never asked to.
Deleting a course
Deleting a course releases the assets it was holding. The course id itself is retired permanently rather than removed — that is what keeps a deleted id from being claimed again — but the references it held are withdrawn in the same transaction, so its media stops counting against the quota immediately. The entry is released after one grace period and its bytes after a second, as above. The grace period is the undo: within it the assets are still there.
Pending allocations
ASSET_PENDING_TTL_MS (default 24 hours) is how long an allocation stays
pending — its bytes are stored, but no document names its id yet. A client
stores bytes first and writes the id into the document afterwards, and nothing
leases that gap, so the window has to outlive a whole generation pass plus a
write-back waiting for the slide it belongs to: media routinely finishes before
that slide exists. A day is deliberately generous, because unclaimed bytes cost
storage while an expiry that fires early costs a course its media. A value that
is not a positive integer stops the server from starting, for the same reason
ASSET_QUOTA_BYTES does.
Quota
Each owner may hold ASSET_QUOTA_BYTES (default 10 GiB) of live assets —
pending-unexpired or still referenced by a document — before further
allocations are refused; the store enforces it inside the write transaction, so
concurrent uploads cannot race past it. The ceiling is per owner, not per
deployment: with the default anonymous-cookie owners a visitor who clears their
cookie becomes a new owner with a fresh quota, so bound total storage elsewhere
if that matters. Entries from before per-owner partitions keep counting against
the old shared partition. Set ASSET_QUOTA_BYTES=0 to opt out and bound
storage elsewhere; any spelling of zero does it. A value that is not a
non-negative integer is refused when the server starts, rather than replaced by
the default, so a mistyped ceiling stops the process instead of quietly running
on a limit nobody chose.
The browser never deletes an asset: one nothing references is left to the collector, and nothing on the wire changes when one is committed — a document write does that as a side effect. Replacing or deleting through the endpoint is limited to the owner, as described above.
Byte egress and HTTP contracts
Asset byte egress is direct by default: the embedded route materializes the
bytes in the response body. Setting ASSET_BYTE_EGRESS=redirect opts into
indirect egress, under which a byte GET answers with a short-lived signed
S3 URL when the byte layer can sign (S3 can; the PostgreSQL byte column cannot
and falls back to direct bytes). Two object-store prerequisites make that safe:
the bucket must allow this app's origin via CORS and expose Content-Type on
the signed response, and the signing identity must hold s3:ListBucket on the
bucket so a missing key answers 404 NoSuchKey rather than 403 — a client can
only read a reclaimed asset as a miss when the store confirms it by code. The
tradeoffs this opts into are specified in the
asset HTTP contract.
The embedded endpoint implements the package's RuntimeStore HTTP contract and DocumentStore HTTP contract.
Startup checks
Invalid configuration stops the server. The register() hook of
instrumentation.ts refuses to start on:
- a missing
DATABASE_URL; - a malformed
ASSET_QUOTA_BYTES,ASSET_PENDING_TTL_MS,OWNER_WRITE_LOCK_WAIT_MSorOWNER_CLAIM_LOCK_WAIT_MS; OWNER_CLAIM_TRIGGERset to anything butexplicitorauto;OWNER_ANONYMOUS_PREMINTthat is not a boolean;- the removed
OWNER_AUTHENTICATOR/TRUSTED_PROXY_*variables, when set; PERSISTENCE_SHARED_OWNER_IDthat is malformed, set withoutACCESS_CODE, or set beside an owner auth registration that leaves outsharedTeamAuthMethod();sharedTeamAuthMethod()registered without the variable, or not as the last method;OWNER_SINGLE_USERthat is not a boolean, a malformedOWNER_SINGLE_USER_IDor one set while the mode is off, single-user mode besidePERSISTENCE_SHARED_OWNER_ID, or beside a registration that leaves outsingleUserAuthMethod();singleUserAuthMethod()registered without the switch, or not as the last method;ASSET_S3_BUCKETset beside a registered asset byte store, orASSET_BYTE_EGRESS=redirectwith a registered byte store that does not declaresignsReadUrls: true.
For any of these, the Node.js server prints a single line,
[boot] Invalid server configuration; the server will not start: followed by
the reason, and exits with code 1 (under next start and the standalone
server.js alike), so a supervisor or container runtime sees the failure
instead of a process that listens and answers every request with 500. Any
other failure during boot, such as a module missing from the build or a host
registration call that throws, also exits with code 1, printed as
[boot] Server startup failed; the server will not start: with its stack.
Warnings, such as the unset ACCESS_CODE notice and the model-routing checks,
never stop the server.
Agent workbench and runtime
The Pro workbench is a usable course-building surface entered from the home
page. Its collapsible navigation rail, conversation pane, and tabbed classroom
pane share /api/agent/* control-plane routes and an in-process session runner.
It is off by default. Enable its build-time entry point and the server runtime;
it uses the same PostgreSQL connection as the rest of the app:
NEXT_PUBLIC_PRO_WORKBENCH_ENABLED=true
OPENMAIC_AGENT_RUNTIME_ENABLED=true
DATABASE_URL=postgres://openmaic:openmaic-dev@postgres:5432/openmaicThe agent runs on the agent slot of openmaic.yml (or the model settings), which follows the default chat model unless it has its own assignment and needs a model with tool calling:
providers:
openai:
preset: openai
apiKey: ${OPENAI_API_KEY}
slots:
llm: openai:gpt-5.5
agent:
model: openai:gpt-5.5
api: openai-completions # or openai-responsesWhile the flag is off, the /api/agent/sessions* and /api/agent/owner-events
routes answer 404; the course library and folder routes do not depend on the
flag (see Course library and agent runtime). The agent slot's
api is openai-completions (the default) or openai-responses, and it must not set
thinking.effort (the configuration is refused at startup or when saved); it may set
thinking.mode. An effort the agent inherits from llm is dropped for the agent. A deployment still configured only through DEFAULT_MODEL has the
agent turned off until the agent slot is assigned.
Runner cadence (scan interval, heartbeat, lease TTL, concurrency, attempts) and
the reserved compaction knobs are listed in .env.example.
Owner identity
Courses, folders, materials, agent sessions and skills are partitioned by an
owner id, which the server resolves for every request in one place
(lib/server/identity/). Every owner-scoped route and Server Action asks it,
once per request; nothing else reads identity cookies or headers.
Resolution asks an ordered list of owner auth methods. Each method looks for one kind of credential and answers exactly one of:
| Answer | Meaning | Resolution |
|---|---|---|
authenticated | Its credential is present and valid | That principal is the owner; later methods are not asked |
not-applicable | No credential of its kind is present | The next method is asked |
invalid | Its credential is present but invalid | 401 INVALID_CREDENTIAL at once; no later method and no fallback is asked |
When every method answers not-applicable, the built-in anonymous
fallback resolves the request: one owner per browser, anon:<uuid> from an
HttpOnly anonymous_id cookie that lasts 400 days (the longest browsers
keep one) and is renewed, same value, on every route handler and Server Action
response that resolves to it, so it expires only after 400 days without use.
Page responses do not renew it, so pages stay cacheable, and a response that
clears it (a claim, a retired owner) never renews it. Losing it (a manual
clear, or 400 days idle) loses access to that owner's library from the
browser: anonymous identity has no other key, which is why the Compose
deployment defaults to single-user mode and a multi-user host should use
accounts. The middleware mints it on the page
response of a browser's first load, so every request the page sends presents
one owner; a route handler reached without a valid cookie mints one the same
way, and a valid cookie is never replaced. It cannot publish. A host can turn the fallback off, and then such a request is a 401
too. A refused request is never served as an anonymous owner.
Out of the box nothing is registered, so every request is an anonymous owner, unless one of two built-ins is selected by the environment (they exclude each other):
PERSISTENCE_SHARED_OWNER_ID(requiresACCESS_CODE): the built-insharedTeammethod resolves every request to that fixed id, so the team behind the access code shares one library and may publish.OWNER_SINGLE_USER=true(the Compose default): the built-insingleUsermethod resolves every request to one owner for a personal installation; see Single-user mode.
Authorization reads the principal's kind and roles, never the shape of the
id. The core roles are course:publish (publish and unpublish a course) and
admin (reserved for administrative surfaces; no built-in grants it).
Single-user mode
OWNER_SINGLE_USER=true resolves every request to one fixed owner,
OWNER_SINGLE_USER_ID (default local; 1-128 characters of [A-Za-z0-9._-],
so the reserved anon: prefix is impossible). The principal is
kind: 'user' with the course:publish role: it is one person's own
installation, so publishing works, and unlike sharedTeam (a team behind one
code, no one person) it gets a claim candidate, see below. No anonymous cookie
is minted.
Exposure. Every request becomes the owner of the whole library, and a route
handler cannot tell a local client from a remote one (it does not see the TCP
peer, and forwarding headers are set by the client), so nothing inspects
requests. Single-user mode runs with or without ACCESS_CODE:
- With
ACCESS_CODE, the access-code gate admits requests, as forsharedTeam. - Without it, the deployment relies on nobody else reaching the server:
bind it to loopback or a private network (the Compose file publishes on
127.0.0.1by default; outside Compose, for examplepnpm start -H 127.0.0.1). The server logs one prominent warning at startup explaining that anyone who can reach it shares, edits and can delete the single library, and how to setACCESS_CODE. It does not refuse to start.
A later first-run setup flow may prompt for an access code; for now, set
ACCESS_CODE yourself before the server is reachable by others.
Earlier anonymous work. A browser that used the deployment anonymously
before still sends its anonymous_id cookie. The single-user principal gets a
pendingClaim for it (see Claiming anonymous work),
but nothing moves on its own: the default trigger is explicit. To bring that
work into the single owner, send POST /api/identity/claim (same-origin JSON,
body {}) from that browser, or set OWNER_CLAIM_TRIGGER=auto knowingly.
A claim is irreversible. With OWNER_CLAIM_TRIGGER=auto, every browser
that visits merges its anonymous library into the single owner on its first
request. If several people used the deployment anonymously before, that
merges all their libraries into one shared, deletable library.
The owner id is permanent. Changing OWNER_SINGLE_USER_ID later, or
switching from PERSISTENCE_SHARED_OWNER_ID, leaves the previous owner's
library stranded: it is not anonymous, so it cannot be claimed. To keep a
shared-team library, set OWNER_SINGLE_USER_ID to the same id.
A host that registers its own methods and also wants the single owner as the
last resort includes singleUserAuthMethod() (exported from
@/lib/server/identity) last, under the same rules as sharedTeamAuthMethod().
Registering owner auth methods
A host with its own accounts writes a method per credential it accepts and
registers them once, from instrumentation.ts register(), before the server
serves a request:
const { configureOwnerAuthentication } = await import('@/lib/server/identity');
configureOwnerAuthentication({
methods: [
{
name: 'session',
async authenticate(req) {
const session = await readSession(req.headers); // host code
if (session === undefined) return { status: 'not-applicable' };
if (!session.valid) return { status: 'invalid', reason: 'expired session' };
return {
status: 'authenticated',
principal: {
ownerId: `user:${session.userId}`,
kind: 'user',
roles: new Set(session.canPublish ? ['course:publish'] : []),
assurance: 'verified',
},
};
},
describeStoredOwner: (ownerId) =>
ownerId.startsWith('user:') ? { kind: 'user' } : undefined,
},
],
// anonymousFallback: false, // refuse requests no method applies to
});- Present but unusable is
invalid, nevernot-applicable. Answernot-applicableonly when the method's header or cookie is absent. A method that answersnot-applicablefor a malformed or expired credential of its own kind silently hands the request to the next method or to an anonymous owner; core cannot tell. A method that cannot decide (its key endpoint or session store is down) throws, and the request fails as a server error. setCookieson anauthenticatedanswer ride every response, errors included.reasonon aninvalidanswer is for server logs only.- Server Actions ask the same methods in the same order, through
authenticateFromContext()when a method has one and otherwiseauthenticate()with the request headers. In a Server Action cookies must be written throughnext/headers; an answer carryingsetCookiesis refused. describeStoredOwner(ownerId)says what kind of owner a stored id is, for work that holds only the id (an agent run, a claim). The anonymous fallback is asked first, then the methods in order. An id nobody recognizes is auserwith no roles.- The middleware mints the anonymous cookie on page navigations whenever
neither
OWNER_SINGLE_USERnorPERSISTENCE_SHARED_OWNER_IDis set: it can run in the Edge runtime and cannot see this registration. A page cookie next to a host credential is a claim candidate like any other anonymous cookie.- With
anonymousFallback: false, setOWNER_ANONYMOUS_PREMINT=false: the cookie would serve no request, and withOWNER_CLAIM_TRIGGER=autoit would be claimed and cleared after every cookieless page load. The server warns at startup when this registration leaves pre-minting on. - With the anonymous fallback on (the default), keep pre-minting: without
it, an anonymous visitor's first API requests each mint their own owner
again. To avoid a claim candidate beside your credential, skip
anonymousOwnerForNavigationinmiddleware.tsonly for the requests your methods authenticate themselves (a page request carrying your session cookie, say).
- With
issuesAnonymousOwners: truewithclearCredential()is only for a method that authenticates anonymous principals itself with a cookie: thoseSet-Cookievalues ride every403 OWNER_RETIRED. Core never callsclearCredential()there on any other method, so an account's session cookie is never cleared because an anonymous identity was retired.- Principals are checked per request: an owner id that is not 1-256 printable
non-space ASCII characters, an unknown
kindorassurance, or apendingClaimset by the method is a500for that request rather than stored. The same resolved owner is the runtime learner key and the asset partition of/api/persistence, so the methods govern those too.
Registration is checked at boot, and each of these throws from register()
so the server does not start: a second call, a call after owner resolution has
started, an empty method list, a malformed or duplicate-named method, and the
sharedTeam rules below. To keep a shared team owner beside host methods,
include sharedTeamAuthMethod() (also exported from @/lib/server/identity)
as the last method: it always authenticates, so nothing after it would be
asked. PERSISTENCE_SHARED_OWNER_ID set beside a registration that does not
include it, or sharedTeamAuthMethod() registered without the variable, fails
the boot rather than being ignored.
OWNER_AUTHENTICATOR and the TRUSTED_PROXY_* variables of an earlier
built-in gateway-header authenticator no longer exist; if any of them is set,
the boot fails with a message pointing here, instead of silently serving every
request as an anonymous owner.
Recipe: accounts through an identity gateway (signed JWT)
OpenMAIC has no built-in gateway authenticator. A deployment behind an identity gateway writes a small method that verifies the signed assertion the gateway forwards, against the identity provider's published keys. Unlike a plain user header, a signed token cannot be forged by a client that reaches the app around the gateway. Common sources:
| Gateway | Header | Issuer / keys |
|---|---|---|
oauth2-proxy with --pass-authorization-header (or injectRequestHeaders in --alpha-config setting Authorization: Bearer <id_token>) | Authorization: Bearer … | The IdP's issuer and jwks_uri; the audience is the OAuth client id |
| Cloudflare Access | Cf-Access-Jwt-Assertion | https://<team>.cloudflareaccess.com, keys at /cdn-cgi/access/certs; the audience is the application's AUD tag |
| Google Cloud IAP | x-goog-iap-jwt-assertion | https://cloud.google.com/iap, keys at https://www.gstatic.com/iap/verify/public_key-jwk |
The sketch below uses the jose library, which
the host adds as its own dependency. It is a recipe the host owns and must
test, not code OpenMAIC ships or maintains. Put the file in
lib/server/identity/host/: the boundary test
(tests/server/identity/cookie-guard.test.ts) fails on reads of gateway
identity headers or of an incoming Authorization header anywhere else, core
identity files included. Register the method from instrumentation.ts.
// lib/server/identity/host/gateway-jwt.ts (host code)
import { createRemoteJWKSet, errors, jwtVerify } from 'jose';
import type { OwnerAuthMethod } from '@/lib/server/identity';
const ISSUER = 'https://idp.example.org/';
const AUDIENCE = 'openmaic';
const JWKS = createRemoteJWKSet(new URL('https://idp.example.org/.well-known/jwks.json'));
const ADMIN_GROUPS = new Set(['openmaic-admins']);
/** jose errors that mean "this token is bad", as opposed to "the keys could not be fetched". */
const TOKEN_ERRORS = [
errors.JWTExpired, // exp / nbf
errors.JWTClaimValidationFailed, // iss, aud, other claim checks
errors.JWTInvalid, // not a usable JWT payload
errors.JWSInvalid, // malformed compact serialization
errors.JWSSignatureVerificationFailed,
errors.JWKSNoMatchingKey, // unknown kid
errors.JWKSMultipleMatchingKeys, // no kid and several candidate keys
errors.JOSEAlgNotAllowed, // alg outside `algorithms`
errors.JOSENotSupported, // alg or header this library cannot verify
];
/** The token, `undefined` when this method does not apply. */
function bearerToken(headers: Headers): string | undefined {
// Cloudflare Access / IAP: return headers.get('cf-access-jwt-assertion') ?? undefined;
const match = /^Bearer\s+(\S+)$/i.exec(headers.get('authorization')?.trim() ?? '');
return match?.[1];
}
export const gatewayJwtMethod: OwnerAuthMethod = {
name: 'gatewayJwt',
async authenticate(req) {
const token = bearerToken(req.headers);
if (token === undefined) return { status: 'not-applicable' };
let payload;
try {
// Checks the signature against the IdP's keys, `iss`, `aud`, `exp` and `nbf`.
({ payload } = await jwtVerify(token, JWKS, {
issuer: ISSUER,
audience: AUDIENCE,
algorithms: ['RS256', 'ES256'],
clockTolerance: 30,
}));
} catch (error) {
// Only a failure of the token itself is `invalid`. Anything else (the key
// endpoint unreachable, answering non-200 or unparsable data, a timeout)
// is a server fault: rethrown, it fails the request as a 500 instead of
// refusing every user as if their token were forged.
if (TOKEN_ERRORS.some((type) => error instanceof type)) {
return { status: 'invalid', reason: (error as errors.JOSEError).code };
}
throw error;
}
const ownerId = `user:${payload.sub ?? ''}`;
if (!payload.sub || !/^[\x21-\x7e]{1,256}$/.test(ownerId)) {
return { status: 'invalid', reason: 'unusable sub' };
}
const groups: unknown[] = Array.isArray(payload.groups) ? payload.groups : [];
const roles = new Set(['course:publish']);
if (groups.some((group) => typeof group === 'string' && ADMIN_GROUPS.has(group))) {
roles.add('admin');
}
return {
status: 'authenticated',
principal: { ownerId, kind: 'user', roles, assurance: 'verified', channel: 'gateway' },
};
},
describeStoredOwner: (ownerId) =>
ownerId.startsWith('user:') ? { kind: 'user', roles: new Set(['course:publish']) } : undefined,
};// instrumentation.ts, inside register()
const { configureOwnerAuthentication } = await import('@/lib/server/identity');
const { gatewayJwtMethod } = await import('@/lib/server/identity/host/gateway-jwt');
configureOwnerAuthentication({ methods: [gatewayJwtMethod], anonymousFallback: false });Notes for the host:
- Absent means not-applicable, present but bad means invalid. A request
without the header falls through (to the anonymous fallback, or a
401withanonymousFallback: false, the usual choice when the gateway covers every route). A wrong signature, issuer or audience, or an expired token, is a401, never an anonymous owner. - A key endpoint outage is not a bad token. Only the listed token errors
answer
invalid.josereports a JWKS endpoint that is unreachable, answers non-200 or returns unparsable data as a genericJOSEError(ERR_JOSE_GENERIC), which the recipe rethrows: the request fails as a server error rather than refusing every user as if their token were forged. - Pin
issuer,audienceandalgorithms. Without an audience check any token the IdP issued for another application would be accepted. subis the stable id; an email can change or be reassigned. Choose the owner id scheme once: changing it later changes every owner.- Groups are IdP-specific (a
groupsclaim needs the IdP to release it; IAP sends none). Drop the admin mapping if yours has no such claim. - The gateway still should not be bypassable, and must not exempt app routes from authentication, but a client that reaches the app directly cannot mint a valid token, so no shared secret is needed.
Claiming anonymous work
A visitor who works anonymously and then signs in has two owners: the anonymous one their courses were written under, and the account. A claim moves everything the anonymous owner holds to the account in one database transaction, and retires the anonymous id.
When a claim can arise. Core attaches the claim candidate itself: when a
host method authenticates a non-anonymous principal and the same request also
carries a valid anonymous_id cookie, the principal gets a pendingClaim
naming that anonymous owner. This covers a visitor who worked anonymously and
then signed in (with the anonymous fallback on), and a deployment that moved
from anonymous use to accounts while visitors still hold their old cookie
(with the fallback on or off). There is no candidate without a valid cookie,
for an anonymous principal, or for the built-in sharedTeam (it has no
credential of its own, so nothing says whose browser work it is); a method
cannot set one itself. The built-in singleUser does get one: its deployment
has one person, so the cookie names that person's earlier anonymous work.
Nothing moves until the claim is triggered:
- Explicitly (the default): the app calls
POST /api/identity/claimwith a JSON body ({}) from its own pages. The answer is200 { status: 'claimed', moved }or200 { status: 'already-claimed' }, with aSet-Cookiethat drops the anonymous cookie. - Automatically:
OWNER_CLAIM_TRIGGER=autoclaims on the first route request that carries a pending claim, before the handler runs. The explicit routes are exempt, so they still report their own claim. Server Actions never trigger it. Prefer the explicit trigger where one browser can be shared by several people: whoever signs in first takes the anonymous work in it.
The anonymous cookie is a bearer credential. Whoever presents it can read
and edit that anonymous work, and, beside a signed-in account, claim it into
the account. On shared devices, clear it (a claim does) before the next person
signs in. The cookie is unsigned and not bound to the account, and it is the
claim candidate core reads. It is host-only, but a sibling subdomain under the
same registrable domain can set a Domain-scoped anonymous_id that the browser
may send first, so a hostile subdomain could plant which anonymous identity a
signed-in visitor claims. Serve OpenMAIC on a registrable domain of its own
(or where no untrusted party controls a sibling subdomain). Hardening the
cookie itself (a __Host- name on HTTPS deployments, refusing a request that
presents more than one anonymous_id) is a deferred item.
| Request | Result |
|---|---|
Not same-origin JSON (Sec-Fetch-Site other than same-origin, a foreign Origin, or any content type but application/json) | 403 CROSS_ORIGIN_REFUSED, nothing changes |
| An anonymous requester | 403 TARGET_ANONYMOUS |
| No anonymous cookie beside the account | 409 NO_PENDING_CLAIM |
| The anonymous owner was already claimed by another account | 409 ALREADY_CLAIMED_ELSEWHERE; the cookie is dropped |
| Either owner is being written (the claim could not take its locks in time) | 503 OWNER_BUSY with Retry-After; retry as it is |
| A claim the rules below refuse | 4xx with the rule's code; nothing changes |
What moves, in this fixed order (a host adds its own tables with
registerClaimParticipant, see below):
- Folders. A folder whose name the account already uses (compared
case-insensitively, like
createFolder) is merged into the account's; one whose id the account already uses for another name moves under a fresh id; the rest move as they are, after the account's own. Filing follows. - Courses (
stage_meta), including deleted ones. - Materials.
- Agent sessions and their session-list history.
- User skills. A handle the account already uses is renamed to the first
handle neither side holds (
my-notes-2,my-notes-3, ...). - Runtime sessions (the learner key). They are re-keyed as stored, so a session written by a newer version does not stop the claim.
- Asset entries (the per-owner partition), so a claimed course keeps rendering its media for every viewer.
- Legacy import bindings (temporary, with the one-way legacy browser import): a browser whose pre-server data the anonymous owner held is then the account's, so the import continues for it.
- Custom agents. An agent id the account already uses keeps the account's agent.
Quotas are not applied to what moves: the account keeps everything, and if it
is now above its asset, material, skill, agent or folder limit it cannot add more
until it is back under. The claim is recorded in owner_merges.
Rules: only an anonymous owner can be claimed, and only by a non-anonymous one; claiming the same pair again succeeds and does nothing; an anonymous owner already claimed by one account cannot be claimed by another; and there are no chains (a retired account cannot claim, and an owner that absorbed others cannot be claimed), so every retired id forwards in one step.
After a claim the anonymous id is retired. A request that still presents it writes nothing under it:
- Creates through
/api/persistence(documents, folders, assets, runtime sessions), the folder, course, material and skill-upload routes, and every other write through/api/persistenceanswer403 OWNER_RETIRED. - A write by id to a row that moved (deleting a skill, posting to an agent
session) answers
403 OWNER_RETIREDtoo. - Every such response carries the
Set-Cookievalues that drop the retired anonymous cookie (and those of any method that declaresissuesAnonymousOwners), so the browser gets a fresh anonymous owner on its next request. The library of the retired id reads as empty.
Work that started before the claim and runs on without a request (an agent run's course edits, generated media and new skills) follows the id to the account instead, so a course that was still generating when its author signed in lands in their account. An agent session created by a request that races the claim is written for the account, as if it had been created just before the claim.
Every write that creates or changes an owner's rows takes a per-owner PostgreSQL advisory lock (the identity lock) in shared mode as its transaction's first statement. A claim takes it in exclusive mode for both owners before touching a row. A fenced write racing a claim therefore either commits first and is moved, or waits and is refused; the tests show no deadlock and nothing left under the retired id for these writes.
Waits are bounded:
- A claim waits up to
OWNER_CLAIM_LOCK_WAIT_MS(default 5000) for the two identity locks. While it waits, PostgreSQL queues new writers of both owners behind it, so this is kept short. - A write waits up to
OWNER_WRITE_LOCK_WAIT_MS(default 30000) for its owner's lock. - An upload holds the lock while its bytes are written, so a claim waits for in-flight uploads, up to its bound.
- Running out of time is
503 OWNER_BUSYwithRetry-After, and nothing is written.
The asset collector does not take the identity lock; a collector pass racing a
claim over the same entries can make PostgreSQL abort one of them, and a
claim aborted that way also answers OWNER_BUSY.
A host registers participants for its own owner-keyed tables from
instrumentation.ts:
const { registerClaimParticipant } = await import('@/lib/persistence/owner-claims');
registerClaimParticipant({
name: 'course-notes',
order: 1000, // after core's 100-950; see lib/persistence/owner-claims.ts
rekey: async (tx, fromOwnerId, toOwnerId) =>
(
await tx.query('UPDATE course_notes SET owner_id = $2 WHERE owner_id = $1 RETURNING 1', [
fromOwnerId,
toOwnerId,
])
).rows.length,
});A participant runs inside the claim's transaction; if it throws, nothing any
participant did is kept. claimOwner(from, to) and
claimPendingOwner(principal) run a claim from host code.
A claim's source must be described as anonymous (describeStoredOwner,
through principalFromStoredOwner). Forwarding a retired id is core's own
(owner_merges); there is no host hook for it. owner_merges records claims
of anonymous owners only, because the write fences enforce retirement only for
ids described as anonymous: describeStoredOwner must keep describing an id
the same way, and a row retiring any other owner is refused when read. A host
that merges two signed-in accounts moves the rows itself (its own
participants) and refuses the merged-away account in its auth method. OWNER_WRITE_LOCK_WAIT_MS and
OWNER_CLAIM_LOCK_WAIT_MS are checked at startup.
Host extension hooks
A host can add product behavior at four points without forking a route. They
are registered like the owner auth methods: once, from instrumentation.ts
register(), and sealed on first use (a second call, or a call after the
server started using them, throws). A plain object or a class instance both
work; each hook is read once at registration and bound to the object passed.
A misspelled hook is refused rather than silently never called: a plain object
may carry only the known keys, and a class instance may carry no public method
other than a hook, so keep a host class's helpers private (#helper) or
register a plain object. With
nothing registered, every point behaves exactly as described above.
// instrumentation.ts, inside register(), next to configureOwnerAuthentication
const { configurePersistenceHooks, configureAssetByteStore } =
await import('@/lib/server/persistence-hooks');
configurePersistenceHooks({
name: 'my-host',
// Course creation, inside the transaction that creates the course.
async authorizeCreate(tx, actor) {
const retired = await tx.query('SELECT 1 FROM host_retired_owners WHERE owner_id = $1', [
actor.ownerId,
]);
return retired.rows.length ? { allow: false, message: 'account retired' } : { allow: true };
},
async onCreate(tx, actor, stageId) {
await tx.query('INSERT INTO host_library (owner_id, stage_id) VALUES ($1, $2)', [
actor.ownerId,
stageId,
]);
},
// What GET /api/stages lists.
library: {
name: 'owned-and-saved',
async list({ principal, queryable, ownedStageIds }) {
const saved = await queryable.query<{ stage_id: string }>(
'SELECT stage_id FROM host_saved_courses WHERE owner_id = $1',
[principal.ownerId],
);
return [...(await ownedStageIds()), ...saved.rows.map((row) => row.stage_id)];
},
},
// Upload admission (req.operation: 'create' | 'replace'), before any byte
// is stored or counted.
async beforeAssetAllocate(principal, req) {
const exhausted = await uploadBudgetExhausted(principal.ownerId, req.headers); // host code
return exhausted
? Response.json({ error: { code: 'UPLOAD_BUDGET' } }, { status: 429 })
: undefined;
},
});
// Where asset bytes live, for the request path and the collector alike.
configureAssetByteStore({
name: 'my-object-store',
create: () => createMyObjectByteStore(), // host code: an AssetByteStore
signsReadUrls: true, // required for ASSET_BYTE_EGRESS=redirect
});| Hook | Called | Contract |
|---|---|---|
authorizeCreate(tx, actor, stageId) | Once per created course, inside its transaction, after the course rows are written | Resolve { allow: true } or { allow: false, message? }. A refusal rolls everything back and answers 403 CREATE_REFUSED (on /api/persistence and POST /api/stages). |
onCreate(tx, actor, stageId) | Right after authorizeCreate allowed it, same transaction | Statements on tx commit with the course; a throw rolls the whole create back. |
library.list({ principal, queryable, ownedStageIds }) | GET /api/stages | Resolve at most 5000 stage ids (more is a 500). The route lists them in that order as the usual items, without duplicates, and drops every id the read path would refuse (deleted, unclaimed, or not addressable at all, such as .. or one containing NUL). folderId is shown only on the principal's own courses. |
beforeAssetAllocate(principal, req) | Every upload over /api/persistence/assets: POST /assets (req.operation: 'create') and PUT /assets/{id}/content ('replace', with the decoded req.assetId), after the owner is resolved and before the body is read | Resolve undefined to proceed or a Response to answer with it; nothing is stored and no quota is counted. Decide on operation / assetId, which are the storage handler's own routing; url is the request as received. Media an agent run generates on the server is not an HTTP upload and does not pass this hook; the per-owner quota still bounds it. |
configureAssetByteStore({ name, create, signsReadUrls? }) | Lazily, by the persistence route and by the asset collector | create({ queryable }) returns an AssetByteStore (@openmaic/storage) that keeps bytes outside the registry database and declares it with writesOutsideRegistryDatabase: true. The flag is trusted: a store that sets it but writes through the registry database can deadlock. Replaces the ASSET_S3_BUCKET switch; setting both stops the server. |
What counts as a create. A course is created by the transaction that
records its owner for the first time: the first save of a new stage id, from a
request or from an agent run. Saving, editing or renaming a course the owner
already holds is an update and calls neither create hook; so is a create that
loses a race to a concurrent create of the same id by the same owner.
actor.ownerId is always the owner. actor.source says who is writing:
'request' carries actor.principal, the principal the request resolved to;
'background' is an agent run writing on the owner's behalf after its request
ended, and has no principal because a run records only the owner id. A
background write is not a trusted caller: the owner started it, so apply the
same limits to it. On a background write a refusal reaches the agent as a fixed
"refused by this deployment" result; message is only sent to request clients.
Courses written before ownership rows existed are adopted by a one-time backfill when the server starts. That runs outside any request, so no create hook runs for them; the server logs how many it adopted.
What a library may list. Course reads are capability-by-id, so listing an
id hands it out: a provider lists another owner's course only when the
principal is entitled to know its id (it saved it, it was shared with it). The
route guarantees the rest: it never lists a course that GET /api/stages/{id}
would not serve.
Redirect egress. The built-in layers are unchanged (S3 signs; the
PostgreSQL column falls back to direct bytes). A configured store that does not
declare signsReadUrls: true stops the server at boot under
ASSET_BYTE_EGRESS=redirect, instead of being discovered by the first read. One
that declares it but turns out to have no signReadUrl is logged and served
with direct bytes, like the built-in fallback; the collector never signs.
Generation run hooks
Course generation runs on the server (POST /api/generation-runs, and the
headless POST /api/generate-classroom). A host that meters usage, enforces a
quota or keeps an audit trail registers its generation run hooks the same way,
from register(); the registration is validated and sealed like the hooks
above. With nothing registered, runs behave exactly as before.
// instrumentation.ts, inside register()
const { configureGenerationRunHooks } = await import('@/lib/server/generation-run-hooks');
configureGenerationRunHooks({
name: 'my-host',
// After every built-in check, inside the transaction that creates the run.
async authorizeStart({ principal, input, origin, tx }) {
const plan = await planFor(principal.ownerId, tx); // host code
return plan.exhausted
? { allow: false, status: 402, code: 'QUOTA_EXHAUSTED', message: 'No quota left' }
: { allow: true, attributes: { plan: plan.id } };
},
// Around every claimed execution, in whichever process claims it.
wrapExecution: (context, execute) => usageScope.run(context, execute), // an AsyncLocalStorage
classifyFailure: (error) =>
error instanceof QuotaExhaustedError // thrown by host code inside the scope
? { errorCode: 'QUOTA_EXHAUSTED', retryable: false }
: undefined,
onRunEvent: (event) => auditLog.write(event), // host code
});| Hook | Called | Contract |
|---|---|---|
authorizeStart(context) | Once per start on both routes, after the body, required-model, material, agent and run-limit checks, inside the create transaction (context.tx), before the run row exists, while the owner's starts are serialized: keep it fast and do no network I/O in it. context carries runId, principal, ownerId, input, request and origin ('generation-runs' or 'generate-classroom'). | Resolve { allow: true, attributes? } or { allow: false, status, code, message, headers? }. A refusal answers with that status (400-599), errorCode: code and error: message, and no run is created. attributes (string values, at most 8 KiB) are stored on the run, never shown to its owner, and handed to the other hooks. A malformed answer or a throw is a 500. |
wrapExecution(context, execute) | Around every claimed execution of a run: the first claim, a takeover after a lease went stale (takeover: true), the claim after an outline confirmation or a Retry, and one for a paused or completed run's media (kind: 'generation-run', with runId, ownerId as stored, currentOwnerId after account claims, stageId?, attributes). Also around each background extraction of an uploaded material (kind: 'material-extraction', with materialId). | Call execute once and answer what it answers. Throwing before execute refuses the execution: the run pauses at its step with the classified failure (its pending media fails instead, for a media-only execution), and the owner's Retry resumes it; an extraction fails. |
classifyFailure(error) | Before the built-in retry and error-code decisions of a run step, the outline stream's retries, the model fallback, a media item and a material extraction. The errors a failure wraps (cause, lastError, errors) are asked too. | Answer { errorCode, retryable, statusCode? } for the host's own errors, undefined otherwise. retryable: false skips the remaining automatic retries (the provider SDK's own retries of a retryable status included) and the fallback model and pauses the run at that step with errorCode. An answer also overrides the built-in rule for a provider's explicit quota refusal, which otherwise skips the automatic retries but may still use the fallback model. A media item keeps its Retry; an extraction records retryable. A throwing classifier is ignored. |
onRunEvent(event) | After a run's transition commits: started, outline-ready, scene-appended, paused, completed, ended (by a discard or a course deletion too). Each carries runId, ownerId (as stored), currentOwnerId (the account the run was claimed into since, resolved as for wrapExecution) and attributes. | Attribute usage to currentOwnerId. Called in commit order within a process, not awaited. The current owner is looked up for at most 5 s (else the stored owner is reported, with a warning), and at most 1000 notifications wait in a process: one that finds the queue full is dropped, with a warning that counts the drops; a throw or rejection is logged and ignored and never affects the run. Best effort, at most once: a process that stops between the commit and the call does not repeat it, so do not bill from it. The run's event log stays the record. |
Known gap. The extraction of materials attached to an agent workbench
session (lib/server/material-extraction/runner.ts) does not pass
wrapExecution or classifyFailure yet; only the extraction of uploaded
owner materials does.