how values are read

• everything is read once at startup, so changing a value means a restart. the ones fixed when the database is created also need a fresh database.
• the real environment wins over .env. .env lines are KEY=value, quotes optional, and the keys still need the HYDRANT_ prefix.
• embedded hydrant (hydrant_start, see embedding) takes the same keys from its json settings, so all of this applies there too. ENABLE_DEBUG and DEBUG_PORT are binary only.
• a value that doesn't parse stops startup with an error naming the variable and the value, like invalid HYDRANT_FIREHOSE_WORKERS `4GiB`: want a whole number. values are trimmed first, and an empty value counts as unset, so it gets the default. hydrant logs the config it ended up with at startup (hydrant configuration:).
• numbers are plain decimal integers, so 4GiB, 1_000 or 1e9 are errors.
• bools take true, false, 1 or 0, lowercase. yes, on or TRUE are errors.
• durations use humantime and need a unit: 3s, 5min, 2h, 3d. a bare 300 is an error, only a bare 0 works without one.
• the fixed-choice ones (DATA_COMPRESSION, VERIFY_SIGNATURES, BACKFILL_STRATEGY, ...) only take the values listed for them, lowercase.
• the ones that take none take it in any case.
• lists are comma separated, entries trimmed, empty entries dropped. a bad entry in RELAY_HOSTS, SEED_HOSTS, CRAWLER_URLS, RATE_TIERS or TIER_RULES is dropped with a warning in the log, and the rest of the list is used. PLC_URL and FILTER_EXCLUDES are stricter, a bad entry there is an error.
• values that parse but would crash hydrant or spin a loop are errors too: FIREHOSE_WORKERS, DB_WORKER_THREADS, BACKFILL_CONCURRENCY_LIMIT and BACKFILL_SPILLS (indexer), DB_EVENTS_MEMTABLE_SIZE_MB and DB_RECORDS_MEMTABLE_SIZE_MB of 0, a zero CURSOR_SAVE_INTERVAL, SEED_POLL_INTERVAL or OFFLINE_HOST_RETRY_INTERVAL, DB_MAX_JOURNALING_SIZE_MB below 64 and DB_REPOS_MEMTABLE_SIZE_MB below 2.
• BACKFILL_PROXIES with anything in it is an error in an indexer.

fixed when the database is created

the storage engine saves each keyspace's options the first time it creates that keyspace, and every later open uses the saved copy. so these only apply to a fresh DATABASE_PATH, or to a keyspace that didn't exist yet (like one a newly enabled build feature adds). on an existing database, changing them does nothing, and nothing tells you:

• DB_REPOS_MEMTABLE_SIZE_MB, DB_EVENTS_MEMTABLE_SIZE_MB, DB_RECORDS_MEMTABLE_SIZE_MB
• DB_RECORDS_BLOOM_FILTERS
• DATA_COMPRESSION, and the zstd dictionaries written by POST /db/train. a dictionary only gets used by a keyspace created while its dict_<name>.bin file is already in the database folder, so restarting after training doesn't pick it up on an existing database.
• the events block layout EPHEMERAL picks (see below)

these two are saved too, but checked: flipping one on an existing database is a startup error, so keep the old value or use a fresh path.

• EPHEMERAL
• ONLY_INDEX_LINKS

and one is sticky in a different way: FULL_NETWORK=true writes full mode into the database's filter on every start, and FULL_NETWORK=false doesn't write anything, so the database stays in full mode. switch it back with PATCH /filter ({"mode": "filter"}), see filter management.

DB_RECORDS_MEMTABLE_SIZE_MB (32, 128 full network) and DB_EVENTS_MEMTABLE_SIZE_MB (32, 192 full network) are how much of the records and events keyspaces is held in memory before it's flushed to a table on disk. a backfill writes its records in steps of BACKFILL_CHUNK_RECORDS, so these memtables flush at their configured size. so their size trades memory against compaction cpu directly: smaller memtables mean more, smaller tables that compaction merges more often, and bigger ones hold more in memory. on a one-core sync of about 950 repos, halving both from 32 to 16 MB cost about 30% more cpu for 40 MB less peak memory, and doubling them to 64 MB saved about a fifth for 75 MB more. raising them helps a box with memory to spare that spends a lot of cpu compacting during a big sync, but only on a fresh database: fjall saves each keyspace's options when it's created, so changing them later does nothing.

core

• DATABASE_PATH./hydrant.dbpath to the database folder, relative to the working directory. it holds the journal, the keyspaces, the lock file and any trained dict_*.bin. only one process can open it at a time. a new path is the only way to get a fresh database, which is what the fixed at creation settings need
• RUST_LOGinfo,fjall=warn,lsm_tree=warnlog filter directives (e.g., debug, hydrant=trace). tracing env-filter syntax, bad directives are skipped. at info every firehose message already opens a log span, so warn is cheaper on a busy indexer. embedded: an empty string isn't treated as unset, so "" means plain info with the storage engine's logs back on, and only the first hydrant_start in a process sets the filter, later ones keep it
• API_BIND0.0.0.0:3000,[::]:3000comma-separated list of <ip>:<port> socket addresses and unix:<path> sockets to bind the API server to. literal IPs only (hostnames not resolved). when both an ipv4 and ipv6 entry share the same port, the v6 listener is set to v6-only to avoid bind collision, and a lone [::]:<port> listens dual-stack. unix sockets get API_SOCKET_MODE. a stale socket left by a crashed hydrant is replaced, but one that is still being listened on is not. the socket is set up in a short-lived .hydrant-<random>/ directory next to it and renamed into place, so its directory has to be writable and its path about 20 characters shorter than the os limit (104 on macOS, 108 on linux). none runs without the API, and an empty value is a startup error. embedded hydrant has no default and requires it
• API_SOCKET_MODE0600octal permissions for unix: sockets in API_BIND. owner-only by default since the API has no auth, 0660 also lets the socket's group in. anything that isn't octal up to 0777 is a startup error
• API_TCP_MANAGEMENTfalsealso serve the management endpoints on tcp binds. by default they're only served on unix: binds, since they have no auth and the socket's permissions are what guards them, so with only tcp binds there is no management API.
• GET_REPO_CONCURRENCY_LIMIT2how many public com.atproto.sync.getRepo requests can build a repo at once. each one reads every record of the repo and rebuilds its tree in memory. a request past the limit gets a rate limited error straight away, it doesn't wait. the slot is released once the repo is built, before it's streamed to the client, so slow clients can keep more repos than this in memory. getRepo only works on databases that store every collection's record bodies (no FILTER_COLLECTIONS, EPHEMERAL or ONLY_INDEX_LINKS)
• ENABLE_DEBUGfalseenable debug endpoints. binary only
• DEBUG_PORTfirst tcp API_BIND port + 1, or 3001port for debug endpoints (if enabled), on 127.0.0.1. binary only

indexing mode

• FULL_NETWORKfalse (indexer), true (relay)if true, discover and index all repos in the network. it does three things. it picks the full network defaults for everything else (24 firehose workers, 64 backfill tasks, plc.directory, bigger memtables and so on, each still overridable). it makes bare CRAWLER_URLS entries default to list_repos, and the default crawler sources become your relay hosts. and it writes full mode into the database's filter on every start, which false never undoes (see above). it doesn't decide whether backfills fetch whole repos, FILTER_COLLECTIONS does. a relay with FULL_NETWORK=false drops to the small filtered defaults (8 workers, plc.wtf) but stays ephemeral
• EPHEMERALfalse (indexer), true (relay)indexer: no records are stored. record bodies ride inline in events and are deleted with them after EPHEMERAL_TTL, so there's no getRecord and no history, and the write path skips the record head lookup. needs the indexer_stream feature, and can't be combined with ONLY_INDEX_LINKS, HISTORY_TTL or the backlinks feature. relay: not read at all, relay events always expire. its only effect there is that HISTORY_TTL is rejected unless you also set EPHEMERAL=false. immutable per database: the setting is persisted on first open and a later flip is rejected at startup, so keep the previous setting or use a fresh database path. it also picks the events keyspace's block size and compression when that keyspace is created
• EPHEMERAL_TTL60min, 3d (relay)how long to keep events before deletion. a worker checks once a minute and drops whole storage tables older than the cutoff, so a bit more than the ttl stays on disk. each time the cutoff moves it also flushes the events memtable first, so in ephemeral mode the events memtable gets flushed about once a minute regardless of DB_EVENTS_MEMTABLE_SIZE_MB. when built with jetstream, retained Jetstream replay metadata is pruned on the same schedule. can change between restarts, see retention for what shrinking or growing it does
• HISTORY_TTLunsetindexer only, permanent databases only. how long replaced record versions stay in history so replays of older events can still show them. humantime duration like 2h, unset keeps them forever. a bad value is a startup error. entries go when compaction next rewrites them, not at the deadline, and nothing triggers that compaction on its own, so old data at the bottom of the tree can sit past the ttl until POST /db/compact. does nothing with ONLY_INDEX_LINKS, which writes no history. can change between restarts. see retention
• ONLY_INDEX_LINKSfalseindexer only. if enabled, record blocks are not stored, only the index (records, counts, events) is kept, and no history is written. getRecord, listRecords, and getRepo will return errors. the event stream and Jetstream stream still work, but create/update events will not include record values. rejected in relay builds. immutable per database: the setting is persisted on first open and a later flip is rejected at startup, so keep the previous setting or use a fresh database path

filter

these set the database's stored filter, the same one PATCH /filter edits (see filter management). setting one at all replaces that whole set on every start, so it wins over whatever was changed through the API since, and an empty value (HYDRANT_FILTER_COLLECTIONS=) clears the set. leave it unset to keep what the database already has. narrowing a filter doesn't delete records that are already stored.

• FILTER_SIGNALScomma-separated list of NSID patterns to use for the filter (e.g. app.bsky.feed.post,app.bsky.graph.*). in filter mode an unknown repo only gets picked up from the firehose when a commit touches a signal collection, so with no signals the firehose skips every repo you haven't added yourself. the default lightrail crawler also needs signals and idles without them. it sends each signal to lightrail as the collection name exactly as written, and lightrail rejects wildcards, so a pattern like app.bsky.* still matches firehose commits but lightrail never finds any repos for it (the crawler just logs an error for that signal and moves on). list exact NSIDs as signals if you want lightrail discovery. ignored in full mode
• FILTER_COLLECTIONScomma-separated list of NSID patterns to use for the collections filter. empty = store all. records outside it are dropped before they're written anywhere, live and in backfill. it also picks how backfill fetches a repo: with any collections set, a backfill streams the repo and keeps only what matches (or uses sparse fetches, see BACKFILL_STRATEGY), and with none it loads the whole repo, which is what BACKFILL_MEMORY_BYTES budgets
• FILTER_EXCLUDEScomma-separated list of DIDs to exclude from indexing. checked on every firehose message and every write, before anything else. an entry that isn't a valid did is a startup error

firehose

• RELAY_HOSTwss://relay.fire.hose.cam/ (indexer), empty (relay)URL of a single firehose source. takes the pds:: prefix too. ignored whenever RELAY_HOSTS is set
• RELAY_HOSTScomma-separated list of firehose sources. if unset, falls back to RELAY_HOST, and if set to an empty string there are no configured sources at all, even with RELAY_HOST set. prefix a URL with pds:: to mark it as a direct PDS connection (e.g. pds::wss://pds.example.com), which turns on host authority checks and RATE_TIERS pacing for it. bare URLs are treated as relays. each source is one websocket and one task, and its cursor is saved every CURSOR_SAVE_INTERVAL. configured sources may point at private or local addresses, unlike seeded ones. defaults to empty in relay mode, PDSes are expected to be seeded via SEED_HOSTS or the firehose management API
• SEED_HOSTShttps://bsky.network (relay)comma-separated list of base URLs to call com.atproto.sync.listHosts on at startup and periodically. hydrant adds every non-banned host as a PDS firehose source. the first pass runs before any source connects, so a slow seed host delays startup. seeded hosts don't count toward NEW_HOST_LIMIT, and each one costs a websocket, a task and a cursor write per save, so memory and writes grow with the host count
• SEED_POLL_INTERVAL30minhumantime interval between completed listHosts polls. set to none (any case) to seed once at startup. 0s is an error
• ENABLE_FIREHOSEtruewhether to ingest relay subscriptions. false keeps the sources from connecting, but their tasks, the shard threads, seeding and the offline retry loop all still run. can also be toggled at runtime through /ingestion.
• FIREHOSE_WORKERS8 (24 full network)number of firehose shards. messages go to a shard by a hash of their repo's did, so one repo's messages stay in order, and each shard is its own thread with a queue of 64 messages. an indexer runs two sets of shards, one to validate and one to write records, so 2n threads, and a relay runs n. a shard doesn't wait on identity lookups: it holds a repo's messages while a shared lookup pool resolves it and goes on with the others. every commit that emits events still goes through one shared lock, so more shards don't add write throughput past that. backfill results also drain through the shards, so they run even with the firehose off. at least 1
• FIREHOSE_MAX_FAILURES15consecutive connect or stream failures before hydrant gives up on a source. the wait between tries doubles from 15 seconds and is capped at a minute, so 15 is about 14 minutes of retrying. failures only reset after a minute connected. a PDS that hits it is marked offline. given-up sources come back through OFFLINE_HOST_RETRY_INTERVAL. 0 and 1 both give up after the first failure
• OFFLINE_HOST_RETRY_INTERVAL30minhow often sources that were given up on get restarted, which also resets their failure count. none (any case) disables it. 0s is an error, it would restart dead hosts in a hot loop
• CURSOR_SAVE_INTERVAL3sechow often a background thread saves every firehose cursor, checkpoints counts and writes the journal buffer out to the os. it's a write to the os, not an fsync, so it protects against a crash of hydrant but not a power loss. after a crash hydrant re-reads up to this much of the firehose, which is cheap since already applied commits are skipped. every source's cursor is rewritten each tick, changed or not, which adds up on a relay with thousands of seeded sources. 0s is an error, it would save in a hot loop

identity & verification

• PLC_URLhttps://plc.wtf, https://plc.directory (full network)base URL(s) of the PLC directory, comma-separated. lookups rotate through them and move to the next one on a rate limit, a server error or a connection error. any invalid entry is a startup error. lookups refuse private and loopback addresses, so a PLC mirror on localhost won't work. each lookup can take up to 30 seconds and blocks the firehose shard that asked
• IDENTITY_CACHE_SIZE1000000max entries in the in-memory identity cache, an LRU of resolved did documents (pds endpoint, handle, signing key). the same number also caps two smaller caches the API fills (verified handles and identity reconcile cooldowns). a miss on the firehose path is a PLC lookup on that shard's thread, so a small cache on a full network firehose means a lookup on almost every commit. entries have no ttl, they leave on eviction or when hydrant invalidates them (a new account, a host mismatch, a signature failure). 0 doesn't turn it off, the cache rounds it up to a tiny minimum. starts empty on every start
• VERIFY_SIGNATURESfullsignature verification level: full, backfill-only, or none. full checks every live commit's signature against the repo's signing key (an identity lookup on a cache miss), and also turns on MST inversion for every live commit, see VERIFY_MST. backfills are verified under full and backfill-only. relay builds have no backfill, so backfill-only is the same as none there. without it a relay or PDS can forge commits
• VERIFY_MSTtrue (false when VERIFY_SIGNATURES isn't full)run MST inversion on live commits: rebuild the commit's tree changes in reverse and check they land on the previous root, which copies the commit's blocks once more. does nothing under VERIFY_SIGNATURES=full, since full verification already runs it. it only matters with backfill-only or none
• VERIFY_CIDStruereject firehose and backfill CAR blocks whose content does not hash to their claimed CID, which costs a sha256 over every block. without this, signature verification does not cover the MST: a PDS or relay can forge nodes that map a key to a different record. record bodies are hashed before they are stored either way
• REV_CLOCK_SKEW300seconds a live commit's rev timestamp may be ahead of hydrant's clock before the commit is dropped. 0 drops commits from any PDS whose clock runs slightly fast, and a negative value drops nearly everything. too large lets a far-future rev through, and since later commits with lower revs count as stale, that repo's real commits get dropped until the clock catches up

backfill

• ENABLE_BACKFILLtruewhether queued repos get backfilled. with false the queue still fills (from the crawler, the firehose and /repos) but nothing starts, and live commits for queued repos pile up on disk until they're backfilled. can also be toggled at runtime through /ingestion
• BACKFILL_CONCURRENCY_LIMIT16 (64 full network)maximum number of concurrent backfill tasks. a task holds its slot for the whole repo: identity lookup, fetch, verification, the write, and the wait for its shard to apply live commits that arrived meanwhile. the queue is taken in random order, not first come first served. one PDS gets at most PER_PDS_CONCURRENCY or this minus one, whichever is lower, so another PDS always has room. every repo download also holds a thread from a shared pool of 512 that the database calls use too, so values in the hundreds start slowing everything else. 0 is an error, turn backfill off with ENABLE_BACKFILL=false instead
• BACKFILL_MEMORY_BYTES4294967296 (4 GiB)memory that unfiltered backfills may hold at once. each one counts what its car holds at its peak: a car in memory about 6x its size, since every block stays there until the last step, and one that spilled past BACKFILL_SPILL_BYTES about 64 bytes a block for its index plus 16 MiB for one step. a backfill that would go past it waits, except the oldest one running, which always goes ahead, so peak memory stays near this plus one repo. "unfiltered" means exactly FILTER_COLLECTIONS is empty, any collections at all (even in full network) means backfills stream and keep only what they want, so they don't count and this does nothing. a waiting backfill keeps its connection open and its REPO_FETCH_TIMEOUT running, so a long wait can turn into a timeout and a retry
• BACKFILL_CHUNK_RECORDS1000records a full backfill writes per step. each step is its own commit on the repo's indexer shard, so live events there wait at most one step behind a big repo, and the commit sequencer is held for one step at a time. an operator erase and a deleted account's wipe go in steps of this size too, an erase at most 4096 entries a step. 0 writes each repo in one step. a backfill stopped or crashed partway resumes from what its finished steps stored, but in ephemeral mode nothing is stored to compare against, so the redo emits the records of the finished steps again
• BACKFILL_SPILL_BYTES33554432 (32 MiB)how big a whole car gets in memory before its backfill moves it to a file. a car in memory peaks at about 6x its size, so 32 MiB is about 200 MiB, while a spilled one holds about 64 bytes per block for its index plus one step: so a car of a few hundred MiB takes tens of MiB instead of GiBs. every spilled block is hashed against its cid as it lands and again when it's read back. 0 spills every whole car
• BACKFILL_SPILL_DIRspill in the database directorywhere spilled cars go. each file is unlinked as soon as it's made, so only the open backfill keeps it, and a spill a crash left before that (named {pid}-{n}.car) is removed at start. other files there are left alone
• BACKFILL_SPILLS4how many backfills may spill at once. a spill always leaves 256 MiB free on its disk: it checks before it starts and again every time all spills together wrote 64 MiB, and once less is left the backfill fails and is retried like a failed fetch, with nothing written. a backfill holds its slot until its last step is in, and one waiting for a slot holds what it has read so far in memory with its REPO_FETCH_TIMEOUT still running, so a long wait can turn into a timeout and a retry
• BACKFILL_STRATEGYautobackfill strategy: full, sparse-filter or auto. full always fetches the whole repo with getRepo. with FILTER_COLLECTIONS set it keeps only the matching records, but still downloads, hashes and decodes every block. sparse-filter instead walks just the parts of the repo's tree that can hold the filtered collections, fetching nodes with getBlocks in rounds, and falls back to getRepo if that fails (but not on a rate limit or connection error, which retries the repo later). auto is sparse-filter, except it switches to getRepo for shallow trees when the first probe didn't already cover what's needed. with no FILTER_COLLECTIONS both sparse modes act like full
• PER_PDS_CONCURRENCY10max requests in flight to one PDS (per scheme, host and port), shared by backfill, crawler checks and PDS listings. a whole getRepo download holds one for its full length, and a sparse backfill can hold up to 4 at once. it also caps backfill tasks per PDS, see BACKFILL_CONCURRENCY_LIMIT, and a repo whose PDS is full gets put back and tried again later. 0 acts like 1
• MAX_CAR_BODY_BYTES2147483648 (2 GiB)largest response accepted from a PDS during backfill, measured after decompression. it covers getRepo and the sparse requests. a repo over it fails and retries on backoff (up to an hour apart) forever, it isn't skipped. for unfiltered backfills it bounds the worst case per task (about 6x this), for filtered ones it bounds the download, not memory
• REPO_FETCH_TIMEOUT5mintotal time allowed for each backfill request, from connecting until the whole body is read. applies to every getRepo download and every sparse request. a slow PDS that can't send a big repo in time never finishes it, while a long timeout lets a stalled PDS hold a backfill slot that long

crawler

• CRAWLER_URLSrelay hosts as list_repos (full network), by_collection::https://lightrail.microcosm.blue (filter)comma-separated list of [mode::]url crawler sources. mode is list_repos, relay_first, or by_collection (dashes work too), bare URLs use the default mode (list_repos with full network, by_collection otherwise). list_repos pages a relay's repo list, by_collection asks for repos holding each FILTER_SIGNALS collection and does nothing without signals. relay_first is an explicit coverage-over-moderation opt-in. an entry with an unknown mode or bad URL is dropped with a warning. set to empty string to disable the configured sources. sources added through /crawler/sources are saved in the database and still start
• ENABLE_CRAWLERtrue if full network or sources configuredwhether to actively query the network for unknown repositories. since CRAWLER_URLS has a default in every mode, unset means on unless CRAWLER_URLS is empty. leave it unset (or empty) for that default. with false the crawler tasks still exist and wait. can also be toggled at runtime through /ingestion
• CRAWLER_MAX_PENDING_REPOS2000 (25000 full network)persistent discovery lookahead retained in the pending queue before the crawler pauses. this does not increase active backfill task concurrency. it's checked before each page of discovered repos is queued, so the queue can go over by one page (up to 2000 repos), and while paused the crawler rechecks every 5 seconds
• CRAWLER_RESUME_PENDING_REPOS1000 (12500 full network)pending-repo count at which discovery resumes. not checked against the max, and a value at or above it just resumes as soon as the queue is back under the max

stream

these apply to every event stream subscriber: /stream, Jetstream and subscribeRepos. each subscriber runs on its own thread with a 500 item output queue.

• STREAM_REPLAY_CHUNK_SIZE0 (auto)how many events a replay reads from the database per step. auto is 250, half the output queue. values above 500 act like 500, and a step never asks for more than the queue has room for. smaller steps mean more database iterators set up per event
• THROTTLED_STREAMSjetstream,relaythe streams whose websocket subscribers have their replay from a cursor capped, comma separated: indexer (/stream), jetstream (/subscribe), relay (subscribeRepos). empty caps none. the live tail and catching up after falling behind it are never capped, and neither are in-process subscribers (Hydrant::subscribe and friends)
• STREAM_REPLAY_SPEED10how fast a capped replay may go, as a multiple of what keeping up with the live stream costs that subscriber: new events a second (averaged over about 10 seconds) times the bytes its replay sends per event read. events a Jetstream filter drops count 32 bytes each, about what reading them costs in cpu next to sending one. at least 1
• STREAM_REPLAY_FLOOR_BYTES10000000bytes a second a capped replay may always use, however quiet the live stream is. the default is about 80 Mbit/s per replaying subscriber, some 15k Jetstream events a second. each subscriber gets its own budget, so many replaying at once are bounded by how many connect, not by this
• STREAM_PENDING_EVENT_LIMIT4096live events a subscriber can hold in memory while its output queue is full. past it, events wait in the shared live buffer (512), and once that laps the subscriber catches up from the database instead. on /stream that catch-up skips the identity and account events it fell behind on, since those are never stored. higher means more memory per slow subscriber and fewer catch-ups. 0 acts like 1
• STREAM_SEND_TIMEOUT30sechow long a subscriber may block delivery (a full queue, or a websocket write of up to 64 frames or 64 KiB at once that doesn't finish) before it gets a consumer-too-slow error and is disconnected. a stuck subscriber keeps its thread and its buffers until then. 0s disconnects anyone who falls even a little behind

database

the storage engine's knobs. the memtable, bloom filter and data compression ones are fixed when the database is created. the keyspaces not named here have fixed memtable sizes (4 to 16 MB each) that no setting changes.

• CACHE_SIZE128the block cache, in MB. one cache shared by every keyspace, holding decompressed blocks. a miss reads the block from disk again and decompresses it again. some index and filter blocks are pinned in memory outside the cache, so they don't count against it. can change between restarts
• DATA_COMPRESSIONzstdcompression for stored data blocks: zstd, lz4 or none. covers the record, history, repo, event and backlink keyspaces, but the top levels of some are left uncompressed on purpose. smaller tables mean less disk and less compaction I/O, at the cost of CPU on every flush, compaction and cache miss. fixed when the database is created
• JOURNAL_COMPRESSIONlz4compression for the write-ahead journal: lz4, zstd or none. only values of 4 KiB and up get compressed, so most writes (repo state, counts, cursors, typical records) go in raw whatever this says. can change between restarts
• DB_WORKER_THREADS4 (8 full network)background threads that flush memtables and compact. with more than one, the first thread never compacts so it stays free for flushes, which leaves n-1 for compaction. with one, a long compaction holds up flushes, and writes stall once 4 memtables wait to flush. POST /db/compact doesn't use these threads, it compacts every keyspace at once. at least 1
• DB_MAX_JOURNALING_SIZE_MB400 (1024 full network)soft cap on journal files on disk. when it's over, keyspaces still holding the oldest journal get flushed early so it can go. it's only checked when a flush happens and the active journal is past about 64 MB, so it isn't a hard limit. unflushed journal is replayed into memory on every open, so this bounds startup time too. at least 64
• DB_REPOS_MEMTABLE_SIZE_MB16 (48 full network)in-memory write buffer for the repos keyspace, and half of it (rounded down to whole MB) for repo_metadata. each keyspace can hold its active memtable plus up to 4 waiting to flush before writes stop, so memory per keyspace can reach 5x this. bigger means fewer, larger flushes and more to replay at startup. at least 2, so repo_metadata doesn't get a zero size one. fixed when the database is created
• DB_EVENTS_MEMTABLE_SIZE_MB32 (192 full network)write buffer for the stream events keyspace (and jetstream_events and relay_events when built). event ids only go up, so flushed tables are never rewritten and this mostly sets their size and count. in ephemeral mode and in relays the ttl worker flushes it about once a minute anyway, so it only matters past a minute's worth of events. at least 1. fixed when the database is created
• DB_RECORDS_MEMTABLE_SIZE_MB32 (128 full network)write buffer for the record heads keyspace, and also for backlinks when built with that feature. record writes look up the current head first, so recent records found in the memtable skip a disk read. no effect in relay builds. at least 1. fixed when the database is created
• DB_RECORDS_BLOOM_FILTERSfalsebuild a bloom filter on the bottom level of the record heads keyspace too. false doesn't mean no bloom filters, the upper levels always have them, only the bottom level, which holds most of the data, goes without. without it, every lookup of a record that isn't stored yet reads index and data blocks from that level, and every first write of a record does that lookup, so backfilling new repos into a big database pays for it. true costs about 10 bits per record in disk and cache. fixed when the database is created
• MAINTENANCE_TIMEOUT6hhow long POST /db/compact and POST /db/train keep ingestion paused before giving up. 0 or 0s means no limit. on timeout ingestion resumes, but the compaction or training keeps running in the background, since it can't be cancelled. a second compact request is refused until that compaction ends, a second training request isn't

rate limiting

• NEW_HOST_LIMIT50how many new hosts can be added via com.atproto.sync.requestCrawl per UTC day. none (any case) means unlimited. seeded hosts and ones added through the management API don't count
• RATE_TIERScomma-separated list of named rate tier definitions in name:base/mul/hourly/daily[/account_limit] format (e.g. trusted:5000/10.0/18000000/432000000/10000000). the per second limit is max(base, accounts × mul), with hourly and daily caps on top. it paces firehose events from pds:: sources and backfill requests to that PDS. the optional account limit prevents new accounts from being created on a PDS once reached, and leaving it off (or giving one that doesn't parse) means no account limit at all. built-in tiers (default, trusted) are always present and can be overridden. in indexer builds default has the same limits as trusted (5000/s), so pacing rarely kicks in for backfill there. in relay builds it's 50/s with a 100 account limit
• TIER_RULEScomma-separated ordered list of glob rules in pattern:tier_name format (e.g. *.bsky.network:trusted). rules are evaluated in order, first match wins. explicit API assignments via PUT /pds/tiers take precedence, and the default tier is the final fallback (defaults to trusted in indexer mode). a rule naming a tier that doesn't exist falls back to default. uses standard glob wildcards (*, ?) matched against the PDS hostname

compatibility settings

hydrant still reads these, so env files that set them keep working, and logs a warning at startup when one is set but does nothing:

• ENABLE_BACKLINKSfalsenothing uses it. backlinks are on exactly when hydrant is built with the backlinks feature, see build features, and then they index every record whatever this says
• BACKFILL_PROXIESunsupported. any value is a startup error in indexer builds, and relay builds ignore it with a warning