mule/plugins/lifeline

The Lifeline plugin: rescue jobs orphaned in executing. When a node (or a producer) crashes mid-run, the jobs it was executing are left in executing forever — no process survives to ack them. On a timer Lifeline finds rows whose attempted_at is older than rescue_after and moves each back to available to run again, or to discarded once its attempts are spent. A Stager-shaped actor — the rescue SQL lives in the engine (engine.rescue_jobs, per driver); this plugin just drives it on its interval.

Leader-only, so the rescue runs once cluster-wide: the leader check reads config.peer.is_leader() (a pure cell lookup) — the Postgres lease peer elects one node, and single-node / in-memory uses peer.always_leader. rescue_after should comfortably exceed your longest expected job runtime, or a still-running job could be rescued out from under its executor.

Types

pub type Message {
  Rescue
}

Constructors

  • Rescue
pub type State {
  State(
    config: config.Config,
    rescue_after: duration.Duration,
    interval: duration.Duration,
    self_subject: process.Subject(Message),
  )
}

Constructors

Values

pub fn default_interval() -> duration.Duration

How often a rescue pass runs — Elixir’s default (lifeline.ex interval: :timer.minutes(1)).

pub fn default_rescue_after() -> duration.Duration

How long a job may sit in executing before it is considered orphaned — Elixir’s default (lifeline.ex rescue_after: :timer.minutes(60)).

pub fn plugin() -> plugin.Plugin

Package the Lifeline as a runtime Plugin for MuleSpec.plugins with Elixir’s defaults: rescue after 60 minutes, sweeping every minute.

pub fn plugin_with(
  rescue_after rescue_after: duration.Duration,
  interval interval: duration.Duration,
) -> plugin.Plugin

plugin with explicit settings: rescue_after is how long a job may sit in executing before it is considered orphaned, and interval how often a rescue pass runs.

pub fn start(
  config config: config.Config,
  rescue_after rescue_after: duration.Duration,
  interval interval: duration.Duration,
) -> Result(
  actor.Started(process.Subject(Message)),
  actor.StartError,
)
Search Document