requirements
hydrant is written in rust and requires the rust toolchain (including cargo), make, cmake for some dependencies. you will also need the clang toolchain and the wild linker.
outbound https and secure websocket connections use the operating system’s certificate trust store, not a bundled root list. on linux, install your distribution’s ca-certificates package; the docker image already includes it.
building from source
cargo build --releasethe binary will be at target/release/hydrant.
to build with optional features (e.g. backlinks):
cargo build --release --features backlinkssee build features for the full list.
running
set the required environment variables and run the binary:
export HYDRANT_DATABASE_PATH=./hydrant.db
./target/release/hydrantsee configuration for all available variables. if a .env file exists in the working directory it will be loaded automatically.
to stop it, send SIGTERM or SIGINT (ctrl-c). hydrant saves its firehose cursors, writes out and syncs the database journal and closes the database, waiting up to 30 seconds for that, then exits with 0. running backfills are dropped and start over from the queue on the next start. firehose messages still queued in memory are dropped too, but the saved cursor never passes a message that wasn't applied, so the next start reads them again. the next start also replays the journal into memory, since the storage engine doesn't flush its write buffers on close. a second signal exits right away without closing the database, which hydrant recovers from like a crash.
reverse proxying
it is highly recommended to run hydrant behind a reverse proxy (like nginx or caddy) if you intend to expose the XRPC or event stream APIs to the public. hydrant's API includes several management endpoints that do not require or support authentication. you MUST NOT expose these management endpoints to the public internet.
management endpoints are only served on unix: binds by default, so a tcp bind (the default 0.0.0.0:3000,[::]:3000) only serves the public endpoints. to manage hydrant, add a socket to HYDRANT_API_BIND, eg. 0.0.0.0:3000,unix:/run/hydrant/api.sock, and talk to it with curl --unix-socket /run/hydrant/api.sock http://localhost/filter. to serve them on tcp like before, set HYDRANT_API_TCP_MANAGEMENT=true and keep them private yourself. see configuration.
public endpoints (safe to proxy)
/xrpc/*: XRPC endpoints./stream: hydrant's ordered event stream./stream/head: the newest durable event ID, for detecting when a replay has caught up./stats: general database statistics./health//_health//version//_version: health check; returns the running version as JSON.
management endpoints (keep private)
only served on unix: binds unless HYDRANT_API_TCP_MANAGEMENT=true.
/repos: explicit repository tracking/resyncing/untracking./redactions: operator erasure of a whole repository or one record's bodies./filter: management of NSID filter patterns./ingestion: manual control over component lifecycle (crawler, firehose, etc.)./crawler/sources: management of crawler relays./firehose/sources: management of firehose relays./pds/tiers: rate-limit tier assignments./db/train//db/compact: database maintenance tasks.*/cursors: cursor management./debug/*: introspection and testing endpoints. these are on their ownDEBUG_PORTlistener on127.0.0.1, not onAPI_BIND.