Runbook: local development without a carrier

How to run the whole system on a laptop — gateway, worker, admin console, and a fake Orange — with no Orange or MTN account and no SMS ever reaching a real handset.

This is the runbook for daily development, whether you are working on vsms or against it. getting-started.adoc is the longer, first-time walkthrough that builds the same thing by hand, one step at a time, and explains what each step proves; read that one when something here fails and you need to know which link broke. If you are integrating an application, ../integrating.md is what you want next.

The one command

just demo

Everything runs as a container, built from this checkout’s own source — compose.dev.yaml (see that file’s own header for the full design, including why it’s a third, separate compose file alongside compose.yml and compose.demo.yaml rather than a reuse of either). From a cold start this takes a few minutes (it builds every image). It brings up, in order: a scratch Postgres, the migrations, an App + Provider + Route + SenderId, the demo app’s service key, the console composer’s service key, an operator’s user row, sms-fake-orange, sms-fake-aws, sms-gateway, sms-worker (dispatch,scheduler,jobs,drain,hooks), the admin console, and (since just demo passes both --profile console and --profile demo) a WebhookEndpoint and demo-app — see "The demo app" below for why that pair is its own, separately-gated profile rather than unconditional.

just demo-status   # what's running
just demo-down     # stop everything, remove its own volumes only

Default ports (override via env if these collide with something already running on your machine — every one of the seven services in compose.dev.yaml's own build already races other parallel work on a shared Docker daemon in some environments, so collisions are real, not hypothetical):

Var	Default	What
VSMS_DEMO_PG_PORT	15433	Postgres, published on loopback
VSMS_DEMO_GATEWAY_PORT	8080	sms-gateway
VSMS_DEMO_ORANGE_PORT	8090	sms-fake-orange
VSMS_DEMO_AWS_PORT	8095	sms-fake-aws
VSMS_DEMO_CONSOLE_PORT	3100	the admin console

VSMS_DEMO_GATEWAY_PORT=18080 just demo

just demo resets to a genuinely fresh state on every up — compose.dev.yaml's own named volumes are wiped first (down -v), not a targeted DROP DATABASE/CREATE DATABASE the way the pre-containerisation version of this script used to do it. It is meant to be re-run, not kept alive for days.

Signed requests (VS-3)

The gateway answers only COSE_Sign1-signed requests and signs its answers, so it needs two Ed25519 server keys to boot. just demo and just demo-up run just dev-keys first, which writes a current and a next key to the git-ignored dev-keys/ (kept if already present, so a restarted stack keeps the identity a client pinned) and compose.dev.yaml mounts them read-only. They are dev keys. docker compose -f compose.dev.yaml run --rm -T sms-gateway server-keys prints the two public keys to pin.

just e2e-integration signs, demo-app signs with its own key (dev-keys/demo-app.pem, registered by the register-demo-app step) and the admin console signs in the browser (next section). Nothing in CI runs any of them against a live gateway.

Opening the console (VS-5b)

The console has no login. just demo registers its composer’s service key (dev-keys/console-composer.pem, the register-composer service) and the admin container derives the gateway’s two public sealing keys from the dev envelope keys it mounts. Open http://127.0.0.1:3100: it makes a key in your browser and shows a QR. There is no vaam admin phone in a dev stack, so pair it on the command line, with the k field of the QR (the page says where it is):

docker compose -f compose.dev.yaml --profile console run --rm --no-deps register-composer pair-console-key \
  --user demo@vsms.local --public-key "<the k field of the QR>"

The page notices within a couple of seconds and opens. The account is demo@vsms.local, an owner; the key lasts 30 days. docs/design/console-pairing.md has the design.

The demo app

just demo ends by waiting for demo-app (examples/node/demo-app — own README there) to run to completion and printing its log. It runs under its own demo Compose profile, not console — just demo/just demo-app pass --profile console --profile demo together, and --profile demo alone (no console at all) is itself a legitimate way to run this proof, since demo-app talks to sms-gateway directly and never through the admin console. It is not another fixture-seeding step: it signs every request with @vaam-apps/vsms-node (the same SDK an integrator would use, here built from this checkout) under its own Ed25519 key, sends one real otp-class message, polls it to a terminal state, and verifies the hooks worker role’s own signed webhook for it against a receiver it runs itself — the one thing nothing else in this stack proves end to end. Output from a run (the message id shortened):

[demo-app] webhook receiver listening on :9000 (POST /webhooks)
[demo-app] baseUrl=http://sms-gateway:8080 to=+237677123456 senderId=VSMS
[demo-app] signing as the registered key dNwD7AF7t3pLXDkQypoYiGf6eZcU5K4mso8D4SEjZwo
[demo-app] accepted messageId=cca64c4c... state=accepted encoding=gsm7 segments=1 operator=mtn estimatedCostXaf=0
[demo-app] state=accepted  (+3ms)
[webhook] VERIFIED  type=message.accepted messageId=cca64c4c... state=accepted
[demo-app] state=queued  (+1009ms)
[demo-app] state=submitted  (+2013ms)
[webhook] VERIFIED  type=message.submitted messageId=cca64c4c... state=submitted
[demo-app] state=delivered  (+4021ms)
[webhook] VERIFIED  type=message.delivered messageId=cca64c4c... state=delivered

=== timeline ===
  +     3ms  accepted
  +  1009ms  queued
  +  2013ms  submitted
  +  4021ms  delivered

=== webhook events received ===
  +   255ms  message.accepted messageId=cca64c4c... — verified
  +  2251ms  message.submitted messageId=cca64c4c... — verified
  +  4250ms  message.delivered messageId=cca64c4c... — verified

[demo-app] SUCCESS: messageId=cca64c4c... reached delivered with 3 verified webhook(s) of 3 received

(routed isn’t guaranteed to show up as its own line — dispatch polls every second, and a fast submit can move accepted → queued → routed → submitted between two polls, same as this real run did.)

Exit code 0 means exactly that: delivered and at least one signature-verified webhook. A non-zero exit — and just demo itself then fails, loudly, on that exit code — means one of the two didn’t happen, and the printed reason says which. Re-run it against an already-running stack with:

just demo-app

To see the sealed-webhook check genuinely fail closed rather than trust it: run demo-app with a deliberately wrong DEMO_WEBHOOK_AUDIENCE (its own README has the exact override). The message still reaches delivered — sending has no idea this receiver exists — but every received webhook is refused as unverified and the process exits non-zero.

Why there is no carrier

sms-fake-orange (backends/apps/sms-fake-orange) impersonates Orange Cameroon’s token and submit endpoints, and independently POSTs delivery receipts back to the gateway’s POST /dlr/{providerKey} route. It is a participant, not a response stub — "the SMS never arrived" is the absence of a later callback, which a mock that only answers the submit call can never model.

Nothing in the gateway or the worker knows it exists. The only difference from production is the orange_cm entry’s host.url in the dev profile (config/application-dev.yml). Every other code path — routing, submission, DLR ingestion, the state machine, webhooks — is the one that runs in production.

Never point a deployment at it. It logs a WARN saying so on every start, and no production compose file (deploy/docker-compose.yml) references it.

sms-fake-aws (backends/apps/sms-fake-aws) does the same for AWS End User Messaging SMS: it checks the worker’s signed SendTextMessage calls and posts SNS receipts it signs itself to /dlr/aws_sms. A second provider and catch-all route (seed-dispatch-aws) at Orange’s priority and weight make the demo split traffic between the two. The one difference from production on this side is in the gateway: this stack builds it with the fake-aws feature and the dev profile sets providers[code=aws_sms].settings.fake_sns_cert_base, so it fetches the signing certificate from the fake. Published images are built without that feature. aws-sms.adoc has the fault modes.

Where everything actually is

There is no .demo/ directory of pidfiles and logs any more — every process is a container, so docker already owns process supervision, and Compose’s own named volumes (vsms-dev_vsms_dev_pgdata, vsms-dev_vsms_dev_secrets, both scoped to the vsms-dev Compose project — see compose.dev.yaml's own header for why this file doesn’t reuse compose.yml's vsms project) own state:

docker compose -f compose.dev.yaml --profile console logs -f sms-gateway   # or sms-worker, sms-fake-orange, admin
docker compose -f compose.dev.yaml --profile console ps                     # same as `just demo-status`

docker compose ... logs <service> is where to look when something is wrong — a message that never leaves accepted is a worker problem (logs sms-worker), one stuck at routed is a fake-orange problem (logs sms-fake-orange).

The demo app’s key (register-demo-app) and the operator account (provision-user) hold no secret at the gateway and print none: a key is only ever its public half in the database, and the private halves are the git-ignored dev-keys/*.pem files. The one-shot steps' logs say what each registered:

docker compose -f compose.dev.yaml --profile demo logs register-demo-app   # the client and key it registered
docker compose -f compose.dev.yaml --profile console logs provision-user   # demo@vsms.local, with nothing to sign in with: pair a key (above)

Giving each developer their own credential

just demo registers one key for the demo app. Each service or developer should get their own against the same App (vsms-demo, the fixed slug compose.dev.yaml's own seed-demo-app step uses). Make a key (the private half never leaves you) and register its public half:

openssl genpkey -algorithm ed25519 -out .e2e/alice-key.pem
PUBLIC_KEY="$(openssl pkey -in .e2e/alice-key.pem -pubout -outform DER | tail -c 32 | base64 | tr '+/' '-_' | tr -d '=\n')"
docker compose -f compose.dev.yaml run --rm sms-gateway register-client \
  --app vsms-demo --client-name alice --label "billing service — alice" \
  --scope sms:send --scope sms:read --public-key "$PUBLIC_KEY"

register-client is idempotent on the key’s thumbprint, and --client-name gives the key its own principal (appc_vsms-demo_alice), so its rate-limit bucket and scopes are not shared. Point your application at http://127.0.0.1:${VSMS_DEMO_GATEWAY_PORT:-8080} with that key and the gateway’s two public keys (docker compose -f compose.dev.yaml run --rm -T sms-gateway server-keys) — see ../integrating.md.

Proving the whole chain

just e2e-integration

Rebuilds the stack (just demo), provisions a second independent client as an "external integrator", sends through ci/e2e-integration (a small Rust tool, not examples/rust/sms-send — see that tool’s own module doc for why it has to run inside the Compose network rather than as a host process against this stack specifically) over real HTTP, then polls GET /messages/{id} as the console’s own credential until that exact message reaches delivered. It exits non-zero at the first broken link, naming the step. e2e-integration.adoc explains what it proves and what it fakes.

examples/rust/sms-send is still the thing to hand someone who asks "what should my integration look like?" — it’s an integrator-facing example, meant to run bare against a real, single-address gateway, which is exactly what it’s for; it just can’t reach compose.dev.yaml's stack directly, for the reason ci/e2e-integration's own module doc explains.

Injecting failures

The happy path is the least interesting thing a fake carrier can do. Override sms-fake-orange's own command to run it with a fault mode:

docker compose -f compose.dev.yaml stop sms-fake-orange
docker compose -f compose.dev.yaml run --rm --service-ports sms-fake-orange \
  --bind-addr 0.0.0.0:8090 \
  --dlr-endpoint http://sms-gateway:8080/dlr/orange_cm \
  --sender-number +2370000 \
  --fault-mode seeded --seed 3

(stop first, so the fault-mode run can bind the same service network alias the stopped container was using; --service-ports republishes compose.dev.yaml's own loopback port mapping for it.)

Flag	What it simulates
--fault-mode seeded --seed <n>	A reproducible weighted mix: rejections, 429s, post-accept timeouts, and duplicate, out-of-order, unknown-reference, or submit-racing delivery receipts. Same seed, same draw.
--reject-tokens	Carrier credentials revoked mid-flight — every submit fails Permanent.
--dlr-delay-ms 30000	Slow delivery, for testing your own timeouts and polling.

The same weighted distribution drives backends/crates/sms-worker/tests/chaos_live_postgres.rs, which asserts invariants rather than outcomes: no message lost, nothing left claimable, and a message that went uncertain never submitted twice.

Running the tests

just check        # cargo check, with the build-concurrency cap
just test         # unit + in-process; the live suites stay ignored here
just test-live    # every live-Postgres suite — manages its own container
just all-checks   # fast host-toolchain subset -- NOT the whole gate
just ci           # the ENTIRE CI gate -- needs only docker/compose/just; see docs/runbooks/testing.adoc

test-live starts (or reuses) one labelled Postgres container and gives each test binary its own database. Do not run two test-live invocations concurrently on one machine — the container name is global and they will corrupt each other’s runs. just test-live-clean removes it.

Known gaps

• just demo is loopback-only. Every service publishes only on 127.0.0.1. Reaching it from another machine, a phone, or a container needs the port bindings in compose.dev.yaml changed.

• The first build on a machine can hit a real Docker BuildKit cache-mount race. just demo's own recipe forces COMPOSE_PARALLEL_LIMIT=1 for exactly this reason — see that recipe’s own comment in the justfile for the mechanism (several app/*/Dockerfile builder stages deliberately share one cargo-registry cache-mount id, which several genuinely-first, uncached builds racing at once can corrupt). A cache already warmed by a previous just demo doesn’t hit this.

Troubleshooting

Symptom	Cause
Gateway exits immediately at startup	No active orange_cm Provider/Route — checked before the listener binds, so this is a startup failure, never a first-request one. docker compose -f compose.dev.yaml logs seed-dispatch shows whether that one-shot step actually completed.
Messages stay accepted	The worker isn’t running, or isn’t running the dispatch role. docker compose -f compose.dev.yaml logs sms-worker.
Messages reach routed and stop	sms-fake-orange isn’t reachable at the orange_cm entry’s host.url in config/application-dev.yml. docker compose -f compose.dev.yaml logs sms-fake-orange.
Messages reach submitted and stop	The DLR never arrived. There is no notifyURL knob any more — Orange’s own docs register the DLR endpoint out of band, so sms-fake-orange is told where to post by its own --dlr-endpoint (FAKE_ORANGE_DLR_ENDPOINT) instead. Check it points at the internal http://sms-gateway:8080/dlr/orange_cm address in compose.dev.yaml, not a host-published port, then docker compose -f compose.dev.yaml logs sms-fake-orange to confirm it actually tried.
429 from everything	A rate-limit bucket. docker compose -f compose.dev.yaml restart sms-gateway to reset, or slow down.
The console shows a QR and never opens	The gateway has no ConsoleKey for this browser’s key yet: pair it with pair-console-key (above). A browser clock more than five minutes off the stack’s is refused the same way; the page says so past two minutes.
A hash-related failure after mixing runs	SMS_HASH_PEPPER differs between the processes that compute a hash. sms-gateway is the one service in compose.dev.yaml that carries it (sms-worker does not load security.* and is given none), pinned to a hardcoded demo value in that file — nothing detects a mismatch if you’ve run send_test_message or another gateway under a different one.
sms-gateway or sms-worker exits at start naming a variable, a flag or a YAML key	The YAML configuration (config/application.yml plus the dev profile, docs/architecture.md section 9.4) refused something before the process touched the database. A variable or flag from an earlier release is refused, not ignored: the message names the key that replaced it (ORANGE_CM_CLIENT_ID is now providers[code=orange_cm].credentials.client_id). Unset the variable in your shell, or move the value into config/application-dev.yml.