pgvector HNSW Index Tuning for Production Workloads
I switched our vector search to pgvector HNSW indexes six months ago and cut p99 latency by 40 percent. The defaults work for prototypes but they will bite you in production if you do not understand the trade-offs between m, ef_construction, and ef_search. This post covers the tuning decisions I made, the benchmarks that drove them, and the maintenance windows I wish I had planned for.
How HNSW indexing works in pgvector
HNSW builds a multi-layer graph where each vector connects to m neighbors on each layer. The top layers are sparse for fast navigation. The bottom layer is dense for accuracy. During a search the algorithm enters at the top layer, greedily walks toward the query vector, then drops down layer by layer until it reaches layer zero. The ef_search parameter controls how many candidates the algorithm keeps in its priority queue at each step. Higher ef_search means better recall and slower queries. The index is built once at creation time using ef_construction which plays the same role but during the build phase.
Tuning m and ef_construction for recall vs speed
The two knobs that matter most at build time are m and ef_construction.
m is the maximum number of connections per node per layer. Typical values are 16, 32, or 64. Higher m increases recall and index size. It also slows down inserts because the algorithm has to find and link more neighbors. I run m=32 for our 768-dimension embeddings. Going to 64 added 15 percent index size and 20 percent build time for only a 1 percent recall gain on our test set.
ef_construction controls the candidate list size during index build. The pgvector default is 64. I set it to 200 for the initial build. That gave me 99.2 percent recall at ef_search=100 versus 96 percent with the default. The build took 3x longer but that is a one-time cost. If you are rebuilding nightly, keep it lower. If you build once and query for months, crank it up.
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops)WITH (m = 32, ef_construction = 200);Do not guess. Run a recall benchmark on a held-out sample of your actual data. I use a small script that queries the exact top-k with a brute-force scan then compares the HNSW results at various ef_search settings.
Creating indexes concurrently to avoid locks
CREATE INDEX takes an ACCESS EXCLUSIVE lock by default. That blocks writes. On a table with 10 million rows the build took 45 minutes. Zero writes for 45 minutes was not an option.
Postgres 15 added CREATE INDEX CONCURRENTLY support for HNSW. It works but there are catches. The build runs in two phases. First it builds the index without blocking writes. Then it waits for a moment when no transactions hold conflicting locks to swap the index in. That second phase can stall if you have long-running transactions. I killed a deploy once because a stray analytics query held a lock for an hour.
CREATE INDEX CONCURRENTLY ON documents USING hnsw (embedding vector_cosine_ops)WITH (m = 32, ef_construction = 200);Monitor pg_stat_progress_create_index to see the phase. If the “tuples done” counter stops moving, check pg_locks for the blocker. I now schedule concurrent builds during our lowest traffic window and set idle_in_transaction_session_timeout = 5min to prevent stuck transactions.
Benchmarking IVFFlat vs HNSW on your dataset
IVFFlat is the older algorithm. It partitions vectors into clusters and scans a subset of clusters at query time. It uses less memory and builds faster. It also degrades badly when the query distribution shifts.
I ran a benchmark on 2 million 768-dim vectors from our production corpus. Hardware: db.r6g.2xlarge, 8 vCPU, 64 GB RAM, Postgres 16.
| Index | Build Time | Index Size | p50 Latency | p99 Latency | Recall@10 (ef_search=100) |
|---|---|---|---|---|---|
| IVFFlat (lists=1000) | 8 min | 4.2 GB | 12 ms | 45 ms | 0.87 |
| HNSW (m=32, ef_construction=200) | 42 min | 6.8 GB | 4 ms | 9 ms | 0.99 |
HNSW won on latency and recall. IVFFlat won on build time and disk. If you have a strict disk budget or you rebuild indexes daily, IVFFlat is still viable. For a serving path that must stay fast under load, HNSW is the right call.
One caveat: HNSW memory usage grows with the number of vectors. The graph lives in shared buffers. If your working set exceeds RAM, you will see random I/O and latency spikes. I keep shared_buffers at 50 percent of RAM and monitor pg_statio_user_indexes for index hit ratio. Below 99 percent is a warning sign.
Monitoring index size and maintenance overhead
HNSW indexes do not support REINDEX CONCURRENTLY yet. A full REINDEX takes the same exclusive lock as the initial create. Plan for that. I run a nightly ANALYZE to keep statistics fresh for the planner. That is cheap and lock-free.
Index bloat is real. Deletes and updates create dead tuples in the index just like in tables. pgvector does not have a native vacuum for the graph structure. The workaround is to rebuild periodically. I rebuild quarterly during a maintenance window. The rebuild script:
- Creates a new index concurrently with a
_newsuffix. - Waits for it to become valid.
- Drops the old index.
- Renames the new one.
CREATE INDEX CONCURRENTLY documents_embedding_hnsw_newON documents USING hnsw (embedding vector_cosine_ops)WITH (m = 32, ef_construction = 200);
-- wait for pg_index.indisvalid = true
DROP INDEX CONCURRENTLY documents_embedding_hnsw;ALTER INDEX documents_embedding_hnsw_new RENAME TO documents_embedding_hnsw;Track index size with this query. Alert if it grows more than 20 percent over baseline.
SELECT pg_size_pretty(pg_relation_size('documents_embedding_hnsw')) AS index_size, pg_stat_get_last_vacuum_time('documents_embedding_hnsw'::regclass) AS last_vacuum;When not to use HNSW
Skip HNSW if:
- Your dataset fits in memory and you need exact results. Use a brute-force scan or
ORDER BY embedding <-> query_vector LIMIT kwith no index. - You have fewer than 100k vectors. The overhead is not worth it.
- You cannot tolerate any write latency during index build. Even concurrent builds add CPU pressure.
- You are on Postgres 14 or older. Concurrent HNSW requires 15+.
If you are building a FastAPI service around this, the async driver choice matters. I covered the sync vs async trade-offs in FastAPI Async vs Sync: Benchmarks & When to Use Each and the SQLAlchemy 2.0 migration gotchas in Fix: AsyncSession Has No Attribute ‘query’ in SQLAlchemy 2.0.
FAQ
What is a good starting ef_search value for pgvector HNSW?
Start with ef_search = 100. Measure recall on your validation set. Increase by 50 until recall plateaus or latency exceeds your SLO.
Does HNSW support filtered metadata queries?
Yes. Use the WHERE clause before the vector operator. Postgres will filter first then run the ANN search on the reduced set. Performance depends on selectivity.
How much RAM does an HNSW index need?
Roughly 1.5x the index size on disk for the graph plus the vector data. For 10M 768-dim vectors at m=32 expect 15-20 GB. Size shared_buffers accordingly.
Can I use HNSW with pgvector on a read replica?
Yes. Build on the primary. The replica will replay the WAL. Query the replica with SET hnsw.ef_search = 100; per session.
Key Takeaways
- Set
m=32andef_construction=200as a baseline for 768-dim embeddings then benchmark. - Always use
CREATE INDEX CONCURRENTLYon live tables and watch for lock contention. - HNSW beats IVFFlat on latency and recall but uses more disk and RAM.
- Plan for quarterly rebuilds because
REINDEX CONCURRENTLYis not supported. - Monitor index hit ratio. If it drops below 99 percent, increase
shared_buffersor add read replicas.
Working on something similar?
If you're building backend or AI systems and want a second set of senior eyes, let's talk.