mule/engines/sqlite

The SQLite engine — the port of Oban.Engines.Lite, built on sqlight (the Erlang esqlite NIF).

Concurrency model

SQLite gives the engine a single connection, and Mule’s runtime calls the engine from many processes (producers, the Stager, plugins, the peer). So new spawns a runner process that owns every use of the connection: each engine operation ships its whole body to the runner as a closure and waits for the reply. That serialization is also the engine’s mutual exclusion — the multi-statement operations (unique insert’s check-then-act, the bulk transitions’ select-then-mutate, leader election’s reap-then-upsert) are atomic because nothing else can touch the connection between their statements. It replaces both Postgres devices wholesale: no advisory locks (so insert_job never returns ConflictLocked) and no FOR UPDATE SKIP LOCKED (so update_job’s JobLockedOrNotFound only ever means not-found).

The runner is linked to the process that called new and the reply wait is a monitored process.call — if the runner dies, callers panic rather than hang.

Representation

The schema (see mule/engines/sqlite/migration) stores timestamps as INTEGER epoch microseconds — unixepoch('subsec') is the database clock, so due-times and lease expiry never depend on the BEAM clock — and args/meta/tags/errors/attempted_by as TEXT holding JSON, queried with the JSON1 functions. Unique containment matching uses the json_each scan from lite.ex’s json_contains fragment; there are no GIN-index analogs, so those matches scan (acceptable at SQLite scale).

Pair the engine with the isolated notifier (SQLite has no LISTEN/NOTIFY) — within one BEAM node it relays every notification the runtime needs. reindex is a no-op: SQLite indexes do not accrue Postgres-style bloat, and REINDEX locks the whole database for no benefit here.

Usage

let assert Ok(connection) = sqlight.open(“mule.db”) let assert Ok(_) = migration.migrate(connection) let engine = sqlite.new(connection)

pub fn new(
  connection connection: sqlight.Connection,
) -> engine.Engine