getknit/knit-spool

By getknit

Updated 6 days ago

Blinded store-and-forward relay for Knit's Internet plane — the reference spool daemon.

Image
Networking
Security
Message queues
0

121

getknit/knit-spool repository overview

knit-spool

The reference spool — a scoped, blinded store-and-forward relay for Knit's Internet plane.

It holds sealed frames for scope ids it cannot map to anyone, and forgets everything else.

Availability: images land here on the first v* release tag. Until then, build from a checkout — see Build your own below.

What a spool is

A spool holds, per conversation scope, a bounded set of end-to-end-sealed frames and a digest over them, streams new arrivals to connected subscribers, and heals divergence by digest anti-entropy.

It never learns node ids, message content, rosters, or delivery facts — it stores ciphertext for scope ids it cannot map to anyone. Spools never talk to each other: clients multi-home across several spools and union them, so no spool is load-bearing and a wiped spool is refilled by any one conversation member.

The protocol spec is the product. The normative spec lives in the Knit repo (docs/SPOOL_PROTOCOL.md); this daemon implements it, and its test suite pins the implementation to the spec's §13 vectors byte-for-byte. Third-party spool implementations are first-class — this image exists so nobody has to write one.

WireCBOR records over one WebSocket, wss://host/spool/v1 (?k= token on private spools)
StackKotlin 2.4.0 · Ktor 3.3.0 (CIO) · kotlinx-serialization CBOR · SQLite (WAL) · JDK 21
SeesScope ids, blob ids, ciphertext, sizes, timing
Never seesNode ids, plaintext, rosters, who read what, which spools a client also uses
OpsGET /healthz, GET /metrics (Prometheus text)
FootprintIdles in ~128–256 MB on the cheapest VPS tier (-Xmx256m default)
Platformslinux/amd64, linux/arm64
LicenseAGPL-3.0-or-later

Quick start

docker run -p 9470:9470 -v spool-data:/data -e SPOOL_POW_BITS=20 getknit/knit-spool:latest

The daemon persists to the /data volume, runs as uid 65532, and carries a /healthz HEALTHCHECK. It serves plain WebSocket — TLS terminates at a reverse proxy; the repo ships Caddy and nginx configs plus a compose file that issues and renews certificates for you.

Tags

Every release is tagged with its version, and a release that is not a prerelease also moves latest. Pin the version in production, or a @sha256: digest for the strict form. latest moves under you, and a restart on a moved tag brings back a daemon you never tested.

The same image lives in two registries

The release workflow builds one multi-arch manifest and pushes that manifest to both, so these are the same bytes:

RegistryImageNotes
GHCRghcr.io/getknit/knit-spoolCarries the build provenance attestation. No anonymous pull limit.
Docker Hubdocker.io/getknit/knit-spoolShorter to type. Anonymous pulls are rate-limited.

The provenance attestation is pushed to GHCR alone — it travels over the OCI referrers API, which Docker Hub supports unevenly. Verifying there covers this copy too, since both names resolve to the same digest:

gh attestation verify oci://ghcr.io/getknit/knit-spool:<version> --repo getknit/knit-spool

Configuration

Environment variables only; invalid values refuse to start, and an unrecognized SPOOL_* name is logged as a probable typo. Defaults follow the spec's §12 constants.

VariableDefaultMeaning
SPOOL_PORT9470listen port
SPOOL_TOKENunsetbearer token; unset = public spool
SPOOL_DATA_DIR/data in the imageunset = in-memory; set = SQLite at $DIR/spool.db
SPOOL_POW_BITS0PoW difficulty for unknown scopes (spec suggests 20; 0 = off)
SPOOL_MAX_BLOB65536max sealed-blob bytes
SPOOL_MAX_SCOPES64max scopes held
SPOOL_MAX_FRAMES1000per-scope frame-cap ceiling
SPOOL_MAX_TTL_MS604800000per-scope TTL ceiling (7 d)
SPOOL_MAX_RECORD131072max CBOR record bytes (must fit SPOOL_MAX_BLOB + 512)
SPOOL_MAX_PULL64max blob ids per pull
SPOOL_MAX_ATTACH_BYTES16777216per-scope attachment byte quota (§6.5); 0 turns attachments off
SPOOL_MAX_A_CHUNK49221max sealed attachment-chunk bytes
SPOOL_MAX_AGET32max chunks per aget; an over-long request is truncated, never refused
SPOOL_MAX_BYTES268435456payload watermark; over it the least-active scope is shed; 0 = unlimited
SPOOL_SWEEP_MS60000sweeper cadence (expiry, cache pruning, watermark)
SPOOL_STATUS_MS300000status log line cadence (5 min); 0 = off
SPOOL_TRUST_PROXYfalsehonor the proxy-appended X-Forwarded-For hop for per-IP limits
SPOOL_MAX_CONNS_PER_IP16connection cap per client IP
SPOOL_RATE_RECORDS50records/s per connection (burst 4×)
SPOOL_RATE_PUSHES10pushes/s per connection (burst 4×)
SPOOL_RATE_NEW_SCOPES6new scopes/min per IP (burst 4×)
SPOOL_LOG_LEVELINFOroot log level

Operating

GET /healthz for liveness, GET /metrics for Prometheus text (token-gated with ?k= on private spools; the shipped proxy configs seal it off from the internet, so scrape it from inside your network). The bearer token rides in the query string, so keep it out of proxy access logs — the shipped configs already do.

Every SPOOL_STATUS_MS the daemon logs one status line, which is the docker logs -f view of a spool with no Prometheus in front of it. On a metered link, watch knit_spool_egress_bytes_total — fan-out means one push leaves as (subscribers − 1) copies, and on cheap VPS tiers the monthly transfer allowance binds long before CPU or memory does.

Build your own

The route to take if you have modified the daemon:

git clone https://github.com/getknit/knit-spool && cd knit-spool
docker build -t knit-spool .
docker run -p 9470:9470 -v spool-data:/data -e SPOOL_POW_BITS=20 knit-spool

Dockerfile compiles from source; Dockerfile.dist is what the release workflow publishes — the same runtime stage over a distribution built ahead of time, which is how the arm64 image avoids an emulated compile.

Conformance

The repo ships a conformance suite that validates any spool implementation — this one or a third party's — over a live connection. TAP on stdout, a MUST tally on stderr, exit 0 only when every MUST check passed.

Licensed AGPL-3.0-or-later. A best-effort hobby project, shipped as-is.

Tag summary

Content type

Image

Digest

sha256:5844ef0a2

Size

142.3 MB

Last updated

6 days ago

docker pull getknit/knit-spool