bloom

Package bloom provides a Bloom filter virtual table — a space-efficient probabilistic structure for set-membership testing. Classic use case: filter a stream of candidates against a known set before paying the cost of an exact join.

CREATE VIRTUAL TABLE recent USING bloom(size=100000, p=0.01);
INSERT INTO recent(word) VALUES ('hello');
SELECT present FROM recent WHERE word = ?;  -- returns 1 if present.

The bit array is persisted to a shadow table named <vtab>_storage on the same schema. Reads and writes go through SQLite’s incremental BLOB API (gosqlite.org.Conn.OpenBlob), so the bit array survives database/sql.DB.Close and reconnects. The shadow table is dropped automatically when the virtual table is dropped.

Positional and named forms are both accepted:

  CREATE VIRTUAL TABLE name USING bloom(100000, 0.01, 7);
  CREATE VIRTUAL TABLE name USING bloom(size=100000, p=0.01, k=7);

  - size=N (default 100) — expected element count. Used to size the
    bit array.
  - p=0.01 (default 0.01) — target false-positive probability (0 < p < 1).
  - k=N — number of hash functions. Default: optimal for the chosen p,
    `round(-log2(p))`.

The vtab declares one visible column (present BOOL) and one HIDDEN column (word TEXT) used for INSERTs and WHERE-clause membership tests:

INSERT INTO name(word) VALUES (?);                -- add to filter
SELECT present FROM name WHERE word = ?;          -- test membership

For a typed Go handle over this vtab — Create / Add / AddMany / Contains / Drop, mirroring vec.Table and fts.Index — see Filter.

import (
    sqlite "gosqlite.org"
    "gosqlite.org/ext/bloom"
)

if err := bloom.Register(conn); err != nil { ... }

For pool-wide auto-registration via sqlite.Driver.ConnectHook:

import _ "gosqlite.org/ext/bloom/auto"

Ported from ncruces/ext/bloom with a Go-native blob-IO path.

Full API: pkg.go.dev/gosqlite.org/ext/bloom