# thunderstrike

thunderstrike is a key-value database server. It speaks TLS 1.3 only,
encrypts every value at rest, and does its disk and network I/O through
io_uring. It ships as one Docker image for x86-64 and ARM64, `docker.io/sbansal793/thunderstrike:0.1.6`,
and the image carries the kit of files that runs it:

| File | What it is |
|---|---|
| `compose.yaml` | The server with its data volume, secrets, limits and seccomp profile, a `tls-init` job that makes a CA and the server's certificate, and a `sample` client. |
| `.env.example` | Every setting Compose reads. Copy it to `.env`. |
| `docker/seccomp-io_uring.json` | Docker's default seccomp profile plus the three io_uring system calls the server needs. |
| `README.md` | This guide. |
| `LICENSE` | The terms you may use thunderstrike under. |

To get them, ask the image to write them into an empty directory:

```
mkdir thunderstrike && cd thunderstrike
docker run --rm -v "$PWD:/out" docker.io/sbansal793/thunderstrike:0.1.6 kit /out
```

The files are owned by whoever owns the directory, and `kit` never
overwrites one that is already there.

## What you need

- **Docker with Compose v2.** Docker Engine on Linux, or Docker Desktop on
  macOS and Windows.
- **A Linux kernel of 6.8 or newer** on the Docker host. The server checks
  this at startup and refuses older kernels. On macOS and Windows, upgrade
  Docker Desktop's Linux VM if needed. A newer container base image does
  not change the host kernel.
- **A supported CPU.** x86-64 with AVX2 (Intel Haswell, AMD Zen 1 or
  newer), or ARM64 v8.2 with the Crypto Extensions (AWS Graviton2 or newer,
  Ampere Altra, Apple Silicon, Raspberry Pi 5).
- **Memory.** Each derived reactor locks its arena in RAM: automatically
  4 GiB with one reactor, or 2 GiB per reactor with two or more. Plan at
  least 4 GiB plus overhead for the default allocation. Use 4 GiB per
  reactor to store 512 MiB values.
- **Disk.** ext4 or XFS under Docker's data directory.

## Quick start

```
cp .env.example .env                                    # 1. settings: read it
docker compose -f compose.yaml run --rm configure        # derive reactors and publish their ports
mkdir -p docker/secrets docker/tls
openssl rand -base64 32 > docker/secrets/passphrase     # 2. the keystore passphrase
chmod 600 docker/secrets/passphrase
docker compose run --rm tls-init                        # 3. a CA and the server's certificate in docker/tls/
docker compose up -d                                    # 4. the server
docker compose ps                                       #    wait for "healthy"
docker compose run --rm sample                          # 5. write, read back, verify
```

The first command that needs the image (`tls-init`) pulls it. Steps 2 and 3
make the two things the server will not start without. Both are files on
the host, and `.env` names their paths.

The sample client writes values of 256 B, 4 KiB, 64 KiB, 2 MiB and 16 MiB
on 8 connections for 30 seconds. It reads every one back, compares it byte
for byte, and exits 0 only with no mismatch and no error.

## Settings: `.env`

Compose reads `.env` beside `compose.yaml`, or variables from your shell,
which take precedence. **Sizes are plain byte counts**: `4294967296`, not
`4g`. A size with a suffix is refused at start with the number to use.

| Variable | Default | |
|---|---|---|
| `THUNDERSTRIKE_IMAGE` | this release | The image to run. |
| `THUNDERSTRIKE_CORES` | `2` | Physical-core budget, even and at least 2. CPU topology determines the reactor count (see [Cores](#cores-shards-and-ports)). |
| `THUNDERSTRIKE_TLS_PORT` | `7380` | Starting client port. Remaining reactor ports and Docker's mapping are generated automatically. |
| `THUNDERSTRIKE_ARENA_BYTES` | automatic | Locked memory per reactor: 4 GiB with one reactor, 2 GiB each otherwise. Set `4294967296` for 512 MiB values. |
| `THUNDERSTRIKE_BLOB_PREALLOC_BYTES` | `0` | Disk each logical shard reserves at creation. Disabled by default; files grow as needed. A nonzero value is multiplied by 1,024 on a new database. |
| `THUNDERSTRIKE_HOST_PASSPHRASE` | `./docker/secrets/passphrase` | The file holding the keystore passphrase on one line. |
| `THUNDERSTRIKE_HOST_TLS_DIR` | `./docker/tls` | Where `tls-init` keeps the CA (`ca.crt`, `ca.key`) and the server's certificate (`server.crt`, `server.key`). The server is given only `server.crt` and `server.key`. |
| `THUNDERSTRIKE_TLS_NAMES` | `localhost` | The DNS names `tls-init` issues the certificate for, separated by spaces or commas. |
| `THUNDERSTRIKE_HOST_DATA` | `data` | The database: `data` is a named volume (`thunderstrike_data`); a path such as `/srv/thunderstrike` is a bind mount. |

A container memory limit, if you set one, must exceed the arena bytes
times the **derived reactor count**, plus a few hundred MiB. `CORES` is
the physical-core budget, not a reactor or software-thread count.
The whole arena is in memory from boot, also with `THUNDERSTRIKE_MLOCK=0`,
so a limit below it fails at start, not later under load: the server
refuses with `... of arena are made resident at boot ..., but this
container's memory limit is ... MiB`, naming `THUNDERSTRIKE_ARENA_BYTES`
and `THUNDERSTRIKE_CORES`. Lower one of them or raise the limit. Making
the arena resident takes time at boot, and the boot log reports it
(`golf arena — ... resident in N s`): about 0.3 s per GiB where memory is
already backed, and up to 8 s per GiB on a cloud VM touching its memory
for the first time.

## The passphrase

The passphrase unlocks the keystore that holds the keys every value is
sealed under. **Losing the passphrase or the data volume loses every
value**, so back them up together. There is no default: without a
passphrase the server refuses to start, and a wrong one stops it at boot
with `keystore open failed: Kek(AuthFailed)`.

Give it as a file (`THUNDERSTRIKE_HOST_PASSPHRASE` with Compose, or a
read-only mount plus `THUNDERSTRIKE_PASSPHRASE_FILE` with `docker run`).
`THUNDERSTRIKE_PASSPHRASE` works too, but `docker inspect` shows it.

## TLS

TLS is mandatory, and it works one way: **clients trust a certificate
authority (CA), and the server presents a certificate that CA issued.** The
server speaks TLS 1.3 only, with **Ed25519** keys only. It refuses to start
with a self-signed certificate, a key that is not its certificate's, or a
certificate outside its dates, and it logs the date its certificate expires.

**Make the CA and the server's certificate.** Put the names clients will
connect by in `.env` (`THUNDERSTRIKE_TLS_NAMES`), then:

```
docker compose run --rm tls-init
```

The first run writes four files into `THUNDERSTRIKE_HOST_TLS_DIR`, owned by
whoever owns that directory:

| File | What it is | Who gets it |
|---|---|---|
| `ca.crt` | The CA's certificate, valid for 10 years. | Every client: it is what they trust. |
| `ca.key` | The CA's key, mode 0600. It signs certificates. | No one. Keep it secret and backed up; only `tls-init` reads it. |
| `server.crt` | The server's certificate, issued by the CA for your names, valid for 825 days. | The server. |
| `server.key` | The server's key, mode 0600. | The server, and nothing else. |

The first name is the certificate's common name, and every name is a DNS
subject alternative name. Clients must connect by one of them.

**Renewing**, or changing the names: `tls-init` keeps the CA and issues a
new certificate from it, so clients change nothing.

```
rm docker/tls/server.crt docker/tls/server.key
docker compose run --rm tls-init
docker compose up -d --force-recreate thunderstrike
```

`tls-init` never overwrites: while `server.crt` or `server.key` is there,
it does nothing.

**Your organisation's own CA** works the same way. Either put its
certificate and Ed25519 key in `THUNDERSTRIKE_HOST_TLS_DIR` as `ca.crt` and
`ca.key` before running `tls-init`, which then issues from it; or have it
issue the server's certificate yourself, and put that in `server.crt`
(followed by any intermediate certificates) with its key in `server.key`.
Every certificate in the chain, the CA's included, must be Ed25519. Public
CAs such as Let's Encrypt do not issue Ed25519 certificates.

**Mutual TLS**: put a client CA in a file, mount it into the container,
and set `THUNDERSTRIKE_TLS_CLIENT_CA` to its path there. Clients then
present an Ed25519 identity that chains to it.

## Cores, shards and ports

**Set a physical-core budget and one starting port.** For example:

```env
THUNDERSTRIKE_CORES=8
THUNDERSTRIKE_TLS_PORT=7380
```

- **A logical shard is a persistent store**: its own files and key range.
  The server always uses **exactly 1,024 logical shards**, independently of
  the core count. It records and validates that count on disk. A directory
  with another count is refused and needs data migration; never edit its
  count record to force it open.
- **A reactor group uses two hardware threads (logical CPUs)**: one runs
  the foreground reactor; the other is shared by its background workers.
  This does not mean only two software threads. The default group starts
  one reactor, seven maintenance lanes and eight journal workers, with
  additional sync fan-out and kernel workers as needed.
- **`CORES` means physical cores visible to the process.** On a machine
  with two usable hardware threads per physical core, 8 cores supply 8
  reactor/helper pairs. With one hardware thread per core, 8 cores supply
  4 pairs. The server reads actual sibling relationships; CPU IDs need
  not be adjacent. It respects CPU affinity and refuses allocations that
  exceed the available physical cores or CPU quota.
- **The public core budget is even**, from 2 through 1,024. Reactor counts
  themselves may be odd: 6 physical cores without SMT supply 3 reactors.
  On mixed or restricted CPU sets, complete sibling pairs are used first,
  then single-threaded cores are paired; an unpaired hardware thread is
  left unused and reported at startup.
- **Each reactor owns a contiguous group of logical shards**, one arena
  and one port, `starting_port + reactor_index`. For `R` derived reactors,
  each gets `floor(1024 / R)` logical shards, and the first `1024 % R`
  reactors get one extra. No shard or reactor count is entered separately.

| Physical cores (`CORES`) | Usable hardware threads per core | Derived reactors | Shards per reactor | Ports from 7380 |
|---|---|---|---|---|
| `2` | 1 | 1 | 1,024 | `7380` |
| `2` | 2 | 2 | 512 | `7380-7381` |
| `8` | 1 | 4 | 256 | `7380-7383` |
| `8` | 2 | 8 | 128 | `7380-7387` |

The topology is the one visible inside the Docker host or VM. Run:

```
docker compose -f compose.yaml run --rm configure
```

This detects the reactor count and writes `compose.override.yaml` with
the exact matching host/container ports. Docker Compose loads that file
automatically. **There is no port range to enter.** It preserves an
override file the user wrote themselves, refusing to overwrite it.

After changing the core budget, starting port or available CPU topology,
stop cleanly, run `configure` again, then run `docker compose up -d`.
A missing or stale generated mapping refuses startup before stores open.
After a crash, recover with an allocation producing the previous reactor
count and stop cleanly before changing it: journals belong to reactors.
Logical shards and their key hash stay fixed when reactor ownership changes.

**The application supplies one address**, such as `db:7380`. The client
asks for `DISCOVER`, which supplies the 1,024-shard layout and the actual
ports serving its ranges. For each key it computes `CRC32C(key) % 1024`,
finds the returned range and uses a connection to that reactor. The client
does not need the core count, CPU topology or a port range from the user.
All generated ports must be reachable on the same host and same port
numbers; clients receive these destinations from the server.
The routing epoch is persisted and advances when ownership or ports
change, so a client can distinguish its cached map from the new layout.

## Durability

**Every acknowledged write survives a crash or a power cut.** Each
write's journal record is on disk before its acknowledgement. There is
nothing to set.

An orderly stop (`docker compose stop`) loses nothing.

## Large values

Values up to 512 MiB and more are first-class.

- **Memory.** One SET needs two copies of its value in the arena: set
  `THUNDERSTRIKE_ARENA_BYTES=4294967296` to store 512 MiB values.
- **Disk.** Size the volume at the peak live bytes divided by 0.75, plus
  one 4 GiB chunk if you store values of 512 MiB or more. When the disk
  fills, writes get `StorageExhausted` and the server keeps serving reads.

## Data and backups

The data volume is mounted at `/var/lib/thunderstrike`, and the store lives
in `data/` beneath it, the keystore included.

- **Named volume or host directory.** `THUNDERSTRIKE_HOST_DATA=data` (the
  default) uses the volume `thunderstrike_data`.
  `THUNDERSTRIKE_HOST_DATA=/srv/thunderstrike` bind-mounts a host directory;
  the container takes ownership of it for uid 10001 at its first start. On
  Docker Desktop keep the named volume: a host directory goes through the
  VM's file sharing.
- **Backup.** Stop the server (or snapshot the volume with the filesystem),
  copy the whole volume, and keep the passphrase with it. A copy taken
  while the server writes is not consistent.

```
docker compose stop thunderstrike
docker run --rm -v thunderstrike_data:/v:ro -v "$PWD:/b" debian:bookworm-slim \
  tar -C /v -czf /b/thunderstrike-backup.tgz .
docker compose start thunderstrike
```

## Stopping, restarting, upgrading

`docker compose stop` sends SIGTERM. The server stops accepting, drains
every in-flight request, checkpoints the index and exits 0. `compose.yaml`
gives it two minutes to do so (`stop_grace_period`); with `docker run`,
use `docker stop -t 120`. A SIGKILL is a crash (see
[Durability](#durability)).

**To upgrade**: back up first. Unpack the new kit beside this one, copy
`.env` (then set its `THUNDERSTRIKE_IMAGE` line to the new release),
`docker/secrets/` and `docker/tls/` into it, and run `docker compose up -d`
there. The data lives in the `thunderstrike_data` volume, not in the kit's
directory, so the new release opens it.

## Health and logs

`docker compose ps` shows `healthy` once every derived reactor has finished
database recovery and its port is listening.
The check reads the container's socket table rather than connecting, and it
allows two minutes for a large store to reopen. `docker compose logs -f
thunderstrike` follows the server's log.

## Running without Compose

The same server with `docker run`, using the passphrase file and TLS
identity from the quick start:

```
docker volume create thunderstrike_data
published_ports=$(docker run --rm --env-file .env docker.io/sbansal793/thunderstrike:0.1.6 ports)
docker run -d --name thunderstrike \
  --env-file .env \
  --ulimit memlock=-1 \
  --security-opt seccomp=docker/seccomp-io_uring.json \
  --stop-timeout 120 \
  -p "$published_ports" \
  -v thunderstrike_data:/var/lib/thunderstrike \
  -v "$PWD/docker/secrets/passphrase:/run/secrets/passphrase:ro" \
  -v "$PWD/docker/tls/server.crt:/run/tls/server.crt:ro" \
  -v "$PWD/docker/tls/server.key:/run/tls/server.key:ro" \
  -e THUNDERSTRIKE_PASSPHRASE_FILE=/run/secrets/passphrase \
  -e THUNDERSTRIKE_TLS_CERT=/run/tls/server.crt \
  -e THUNDERSTRIKE_TLS_KEY=/run/tls/server.key \
  docker.io/sbansal793/thunderstrike:0.1.6
```

Three of those options are not optional:

- **`--security-opt seccomp=docker/seccomp-io_uring.json`.** The server
  cannot run without io_uring, and Docker's default seccomp profile refuses
  it. The profile is Docker's default with one rule added that allows
  `io_uring_setup`, `io_uring_enter` and `io_uring_register`. Operations
  submitted through a ring are not filtered by seccomp, so a container
  allowed io_uring can reach files and sockets its other rules would refuse.
- **`--ulimit memlock=-1`.** The arena is locked in memory, and Docker's
  default limit is 8 MiB. `THUNDERSTRIKE_MLOCK=0` runs unlocked instead, at
  the cost of latency spikes if the arena is swapped. Unlocked, the arena
  still takes its full size in memory from boot.
- **A volume at `/var/lib/thunderstrike`.** That is the database.

To make the CA and the server's certificate without Compose:
`docker run --rm -v "$PWD/docker/tls:/out" docker.io/sbansal793/thunderstrike:0.1.6 tls-init /out`. Mount
only `server.crt` and `server.key` into the server, as above, never the
directory: `ca.key` stays out of it.

## Which machines run which image

One tag holds two images, `linux/amd64` and `linux/arm64`, and Docker
pulls the one for the machine. Docker on macOS and Windows runs Linux
containers in a VM, so the same two cover every machine:

| Machine | Image |
|---|---|
| x86-64 Linux servers, Windows (Docker Desktop, WSL 2), Intel Macs | `linux/amd64` |
| ARM servers (Graviton, Ampere), Apple Silicon Macs, Windows on ARM | `linux/arm64` |

The release may also carry a `-v4` tag: amd64 only, compiled for AVX-512.
It is faster on CPUs that have AVX-512 and stops with SIGILL (exit status
132) on those that do not.

On Docker Desktop the VM's memory is the limit (Settings, Resources), and
the VM's disk and network sit between the server and the hardware: use it
for development, and measure performance on Linux.

## Security

- The entrypoint starts as root only to read the secrets, stage the TLS
  key in memory, and take ownership of the data directory. It then drops to
  the `thunderstrike` user (uid and gid 10001) for good, with no
  capabilities and `no_new_privs`, and starts the server, which becomes
  PID 1. `--user 10001:10001` skips the root step if your secrets and
  directories are readable by that user.
- The image has no shell: `docker exec <container> sh` fails. Read the
  logs with `docker logs`, check health with `docker inspect`, and reach the
  data by mounting the volume into another container (as the backup does).
- Keep `docker/secrets/`, `docker/tls/ca.key`, `docker/tls/server.key` and `.env` out of version
  control and out of any image.

## Troubleshooting

| Symptom | Cause |
|---|---|
| `docker compose ps` shows `Restarting` | The server refused to start, and Compose starts it again. `docker compose logs thunderstrike` says why: one of the rows below. |
| `io_uring probe failed: NotSupported { feature: "io_uring" }` | The container runs under Docker's default seccomp profile. Add `--security-opt seccomp=docker/seccomp-io_uring.json`. |
| `RLIMIT_MEMLOCK is 8192 KiB and the arena needs ...` | Add `--ulimit memlock=-1`. If Docker refuses that (`error setting rlimit type 8`), its daemon's own limit is lower: run it with `LimitMEMLOCK=infinity`, or set `THUNDERSTRIKE_MLOCK=0`. |
| `Linux kernel 6.8 or newer is required` | Upgrade the Docker host's kernel. On macOS or Windows upgrade Docker Desktop's Linux VM; pulling a new container cannot change its kernel. |
| `no keystore passphrase` | `THUNDERSTRIKE_HOST_PASSPHRASE` names a missing or empty file. |
| `keystore open failed: Kek(AuthFailed)` | The passphrase is wrong for this volume's keystore. |
| `TLS is mandatory: set THUNDERSTRIKE_TLS_CERT and THUNDERSTRIKE_TLS_KEY` | No identity was given. Run `docker compose run --rm tls-init`, or put a certificate and key in `THUNDERSTRIKE_HOST_TLS_DIR`. |
| `... is not readable by uid 10001` | The container was started with `--user`. Make the file readable by that user, or let it start as root. |
| `... is not a plain integer` | A size was given with a suffix. Use bytes: 4 GiB is `4294967296`. |
| `TLS cert ... is signed by itself (or is a CA's)` | The certificate is self-signed; an older `tls-init` made such certificates. Remove `server.crt` and `server.key`, run `tls-init`, and give clients the `ca.crt` it writes. |
| `TLS cert ...: certificate 1 has expired` | Renew it: see TLS. |
| A client fails certificate verification | It connects by a name the certificate does not carry (`THUNDERSTRIKE_TLS_NAMES`), or it does not trust `ca.crt`. |
| Clients reach the first port, then fail | A published port differs from the container's. Publish `7380+c` as `7380+c` for every core. |
| `THUNDERSTRIKE_CORES must be an even physical-core budget from 2 through 1024` | Set an even physical-core budget the Docker host/VM can supply, then run `configure`. |
| `physical cores ... available` or `process CPU capacity` | Reduce `CORES` or increase the container's available CPU set/quota. |
| `Docker's published ports are missing or stale` | Run `docker compose -f compose.yaml run --rm configure`, then `docker compose up -d`. |
| `requires exactly 1024 logical shards` | The directory was written with another key-routing modulus. Migrate its data into a new directory; do not edit its count record. |
| `journal ... needs recovery with the previous reactor count` | Restart with the previous core count, stop cleanly, then change it. |
| The container exits with status 132 (SIGILL) | The `-v4` image on a CPU without AVX-512. Use the main tag. |
| `StorageExhausted` | The volume is full. |

## License

thunderstrike is proprietary software of SHORTSQUEEZE PRIVATE LIMITED,
licensed under the THUNDERSTRIKE END USER LICENSE AGREEMENT in `LICENSE`. You
may download and run it free of charge, for any lawful purpose, including in
production and as the database behind your own applications and services.
It is not open source. The agreement does not allow modifying, reverse
engineering, reselling or redistributing it, or offering it as a hosted
database service in its own right. It comes with no warranty and is
governed by the laws of India. By downloading or using thunderstrike you
accept the agreement; `docker run --rm docker.io/sbansal793/thunderstrike:0.1.6 license` prints it.

The image includes third-party components under their own licenses. Their
notices are in the image, in `/usr/local/share/thunderstrike/third-party/`
(the programs' components) and `/usr/share/doc/` (the base system's).
