Architecture

One process, one schema owner, one write path.

Siteplan runs as one Django project under ASGI: HTTP views, WebSocket consumers and a JSON API in the same process that owns the ORM. Two deployment processes come from the same image, and where an agent run executes is a policy — not a rewrite.

The decision

Why not a Django + FastAPI split.

The tempting shape is a realtime service in front of Django. It is rejected for a specific reason: there is no missing capability that justifies a second auth plane and a second migration consumer.

AlternativeVerdictReason
Django ASGI + Channels + Ninja Chosen One deploy, one schema owner, one auth plane. Reuses the existing HTMX, Alpine and Tailwind stack.
Django + separate FastAPI service Rejected Adds a second auth plane, a second migration consumer and a second deploy for no missing capability.
WSGI with polling or SSE only Rejected as primary SSE is one-way and still needs Redis across instances. Polling cannot push approvals. Kept as a documented degradation path.
Celery for agent runs Deferred Postgres SKIP LOCKED plus a Redis wake-up is enough for the first release, and Celery can replace the worker behind the same interface.

A split only on a concrete trigger

  • Agent runs need a materially different scaling policy, runtime or network zone.
  • A non-Python agent runtime is adopted and will not be reimplemented.
  • Sustained load exceeds roughly ten thousand concurrent sockets per instance.
  • Compliance requires provider credentials and the orchestrator in an isolated service.
The executor seam turns trigger one into a policy value. A remote executor is the mechanism a split would use, and it becomes reachable by adding a placement rather than by moving the run loop.
Topology

One image, two processes, two data services.

Postgres is the truth. Redis carries fan-out and wakes the worker. The browser talks HTTP and WebSocket to the same application.

System topology
BROWSER WORKSPACE COLUMN 1 · GLOBAL SHELL routes · pins · reports COLUMN 2 · CONTROL+STREAM chat · run logs · approvals COLUMN 3 · CANVAS GEN_UI · BROWSER · DIFF · LOGS HTTP · Django views, Ninja templates, HTMX, JSON over one session WebSocket · Channels auth · membership · replay · live DJANGO ASGI APPLICATION · uvicorn HTTP views templates + HTMX WS consumers auth · membership · replay Django Ninja JSON API canvas snapshot · actions append_event — the single write path seq · dedupe · grants · projections · post-commit fan-out Postgres · truth durable before fan-out Redis · fan-out carries only the seq POSTGRES event log · projections tenancy · agent config runs · objectives · artifacts SKIP LOCKED claim AGENT_WORKER · same image claims AgentRun · resolves runtime executes via an Executor placement appends events · writes run steps never reaches the auth plane
New apps: siteplan_org, siteplan_agent, siteplan_dynapp Redis is required from day one
The event log

One function writes events. Nothing else may.

Every write goes through append_event. That is not tidiness — it is what makes ordering, idempotency and authorization hold across instances.

  • A row lock on the topic serialises writers, so seq is gapless and monotonic across instances.
  • An idempotency key of (topic, client_event_id) returns the original event after a retry.
  • Agent grants are checked at append time, not at prompt time, so a compromised runtime cannot escalate by emitting an event.
  • Fan-out happens after commit and carries only the seq; subscribers re-read the row from Postgres.
def append_event(*, topic_id, kind, sender, payload,
                 run=None, caused_by=None, client_event_id=None):
    with transaction.atomic():
        topic = Topic.objects.select_for_update().get(pk=topic_id)

        if client_event_id:
            existing = TopicEvent.objects.filter(
                topic=topic, client_event_id=client_event_id).first()
            if existing:
                return existing

        if sender.actor_type == "AGENT":
            check_agent_grants(sender.agent, kind=kind, payload=payload)

        topic.last_seq += 1
        topic.save(update_fields=["last_seq"])
        event = TopicEvent.objects.create(
            topic=topic, seq=topic.last_seq, kind=kind, sender=sender,
            payload=payload, run=run, caused_by=caused_by,
            client_event_id=client_event_id)
        apply_projections(event)
        transaction.on_commit(lambda: fanout(topic_id, event.seq))
    return event

created_at is display metadata only. It is never used for ordering or resumption — wall-clock timestamps collide and cannot support dedupe.

Stream versus state

The log is the truth. Everything else is a projection.

Stream

TopicEvent

Append-only, and the record of what happened. Provider-neutral kinds such as TEXT, TOOL_CALL and UI_MUTATION, so an adapter replays normalised history.

State

Canvas · CanvasVersion

A cache of the current state at a version, keyed by a stable widget id. Rebuildable from zero with a projection rebuild command.

State

Artifact · Pin · Task

Projections of ARTIFACT_PUBLISHED, ARTIFACT_PINNED, ARTIFACT_UNPINNED and TASK_CHANGED — so a pin can be rebuilt and an artifact re-derived.

Resume without gaps or duplicates

The client stores its last seq per topic and sends it on connect.

1
Authorise
The consumer checks membership, then replays seq > n from Postgres in bounded pages.
2
Join, then go live
The group join happens before the replay read, so an event committed during replay is never lost.
3
Drop stale frames
Any live event at or below the last replayed seq is dropped. That closes the replay/live race.

Optimistic concurrency

A UI mutation carries base_version. If it does not match the canvas's current version, the append is rejected with a conflict event and the producer re-reads state.

Two agents cannot silently clobber one canvas. A conflict is an event, not an exception — so the losing producer can reconcile instead of guessing.
Large backlogs reset rather than replay. Beyond a threshold the client fetches a canvas snapshot instead of the whole history.
Agent runtime

Two axies that are easy to conflate: which engine, and where.

A provider adapter says which engine runs a session. It does not say where the session runs. Keeping them independent is what allows the same adapter to run in-process today and in a container or on a VM later — with no change to the adapter or the event pipeline.

Axis one · Engine

Runtime adapters

  • OpenAICompatibleRuntime streams chat completions with tool calling. The tool loop is ours, so every call is grant-checked and approval-checked before execution. Billing is pay per use.
  • CliAgentRuntime is one adapter per CLI — claude_code, opencode, hermes, commandcode. Each maps that tool's output stream to the same chunks. Billing is subscription.

"Support CLI agents" means one adapter per tool, not one generic flag. Start with the two tools the team actually uses.

Axis two · Placement

Executors

PlacementStatusIsolation
in_processbuiltnone
subprocessbuiltprocess, same uid
sandboxedspecifiedfilesystem namespaces
containerspecifiednamespaces + cgroups
remotespecifiedhost or VM
@runtime_checkable
class Executor(Protocol):
    placement: str
    def capabilities(self) -> PlacementCapabilities: ...
    def start(self, session: "RunSession") -> AsyncIterator[RunChunk]: ...
    async def cancel(self, run_id: uuid.UUID) -> None: ...
    async def teardown(self, run_id: uuid.UUID, epoch: int) -> None: ...

The same chunk stream everywhere

Executor.start returns an async iterator with the same shape as AgentRuntime.start, so the worker's translate loop is identical for every placement. The provider adapter never learns where it ran.

  • teardown is separate from cancel and must succeed even when the session already died — that is what prevents orphaned containers.
  • epoch numbers the attempt, so cleanup is idempotent and a stale attempt cannot delete a live one.
  • A boundary placement takes a serialisable envelope with a secret reference, never a plaintext secret.
Isolation

Local execution is a production placement, not a development scaffold.

Siteplan is installed on-premise rather than run as SaaS. An administrator who trusts the people on the box should not have to stand up a container runtime, a storage pool and an image pipeline before an agent can run.

That choice fixes the threat model, so it is stated plainly: a subprocess is containment, not a boundary. An agent holding a shell is arbitrary code execution as the worker's uid. What a trusting local placement still guarantees is non-collision and the right account — and each guarantee is required, not optional.

Guarantee
A working directory per workstream, leased per run

One directory per topic and requester, mode 0700. Two people in a room never share a checkout, and runs within one workstream serialise instead of interleaving writes.

Guarantee
A home root per user, pre-authenticated by the admin

A run never inherits the worker's credentials. Sharing a subscription is one account provisioned into several roots; a personal subscription is one user's root with their own account.

Guarantee
The isolation level is visible

The organization's placement policy and the run's recorded placement say which guarantees are in force, so an admin cannot believe they have a boundary they do not have.

A steering message is not a stop. A message aimed at a running turn steers it. Stopping the turn is a separate, deliberate act — the room's red hard stop, which cancels the row, frees the lease, and lets the engine author the terminal event.
Resolution

Placement and runtime are both first-match-wins chains.

Both resolve to a recorded value on the run, so an auditor can answer "which engine, and where" after any config change.

Runtime selection

  1. An explicit selection for this run, from the UI or the API.
  2. The active runtime binding for the topic and agent.
  3. The agent's runtime override.
  4. The definition's default config, then the lowest priority.
  5. An organization runtime policy.
  6. In future, an automatic router.

A manual switch appends a binding and a RUNTIME_CHANGED event, so the timeline shows where the engine changed. A switch never renumbers or rewrites existing events.

Placement resolution

  1. An operator force from settings — the incident kill switch.
  2. The environment profile.
  3. The organization policy.
  4. A validated hint on the runtime config.
  5. The default: in-process for the API provider, subprocess for CLI.
Validation fails closed. A CLI provider may not use in-process. A multi-tenant CLI run requires container or remote. An isolation floor may never fall back to a weaker placement.
Security

Two independent axes, because one is not enough.

Axis one

Capability

"May this actor use this verb." Coarse and auditable: topic.post, canvas.write, artifact.publish, artifact.pin, dataset.query, dynapp.write, task.assign, agent.manage, browser.open. Default-deny, expiring, revocable — resolved by one helper called from the append path, the API endpoints and tool dispatch.

Axis two

Data scope

"Which rows may this actor see." Resolved from the viewer at read time — the same membership union that defines room access — and never stored on an artifact, a pin or a grant. A membership change takes effect immediately.

RuleWhy it exists
No tool.invoke:sqlArbitrary SQL turns a prompt injection into a database read. A validated dataset spec is the only path.
Aggregates after row filteringA count computed before filtering leaks the existence of rows the viewer cannot read.
Secrets never in argv or the environmentBoth are readable on a shared host and inherited by child processes.
BROWSER content in a sandboxed iframeDeliberately without allow-same-origin, so the framed page runs on an opaque origin.
Approval bound to a payload hashAn approval cannot be replayed for a different action.
One hop for triggersAn agent may not trigger an agent, or the bill is unbounded by construction.
Roadmap

Sixteen slices, each independently verifiable.

Every step names real files and leaves the existing tests green. The first four — ASGI, the event log, the actor flag and the WebSocket consumer — are the foundation everything else stands on.

#StepWhat is verified
1ASGI skeletonExisting tests pass; /, /app/ and /cms/ still serve; a ping consumer echoes.
2Organization, team, room, topic, event logConcurrent appends produce unique monotonic seq; a duplicate client event returns the original; a fork creates a child with a required subject.
3Actor flagExisting auth tests pass, and agent users cannot log in.
4WebSocket consumerAnonymous closes 4401, non-member closes 4403, reconnect resumes with no gaps or duplicates.
5Agent domainClone copies configs and never secrets; grant expiry works; the approval state machine works.
6Generative UI and canvasInvalid payloads rejected; a patch updates one widget; every widget string is wrapped for translation.
7JSON APIOpenAPI serves; tenancy is enforced; actions re-resolve against stored widgets.
8Runtime and workerA fake provider drives a run to DONE; cancellation works; SKIP LOCKED does not double-claim; steps sum to the run cost.
9Executor seam and placementsA subprocess CLI run has no DATABASE_URL or REDIS_URL; a killed worker leaves no orphan.
10Dynamic appsCreate a model with a relation, persist records, enforce organization scoping.
11i18n and hardeningmsgfmt --check passes and no unwrapped UI strings remain.
12Objectives and tasksA task assigned to an agent creates a topic and a queued run; an objective with no linked work is visibly empty.
13Artifacts and pinsPromote a canvas to a team pin, pin the same artifact org-wide, resolve per viewer, confirm a guest sees neither.
14Datasets and data bindingTwo viewers of one artifact receive different rows from the same pin; a query outside scope renders a permission state.
15Participation and triggersAn agent is rostered, mentioned and fires on a cron; a trigger that tries to start a second run is refused.
16Tool mediation and agent APIA CLI placement performs a tool call through the same dispatch; a finished run's token is rejected; no executor opens a database connection.

Deploy delta: one REDIS_URL, an ASGI web command, one sync_dynamic_models build step, and one background worker from the same image.

Next: the protocol that lets an agent draw a report instead of writing HTML.