Background jobs with Sidekiq

Sidekiq is a popular system for running background and cron jobs in Ruby on Rails apps. Roadster provides built-in support for running background jobs with Sidekiq via the Sidekiq.rs crate, which provides a Rust interface for interacting with a Sidekiq server (e.g., a Redis server).

Below is an example of how to register a worker and enqueue it into the job queue.

Service configs

Various properties of the Sidekiq worker service can be configured via the app's config files. The most important fields to configure are the following:

service.worker.sidekiq.num-workers: The number of Sidekiq worker tasks that can run at the same time.
service.worker.sidekiq.queues: The names of the worker queues to handle.
service.worker.sidekiq.redis.uri: The URI of the Redis database to use as the Sidekiq server.

[service.worker.sidekiq]
num-workers = 2
balance-strategy = "round-robin"
queues = ["default"]

[service.worker.sidekiq.redis]
# A hard-coded value can be provided to connect to a local server for local development.
# Production values should be provided via a more secure method, such as an environment var
# or an `AsyncSource` that fetches from an external secrets manager.
uri = "redis://localhost:6379"

[service.worker.sidekiq.periodic]
stale-cleanup = "auto-clean-stale"

See the config struct for the full list of fields available.

Worker configs

In addition to the service-level configs, each worker has various configurable values that can be provided by implementing the Worker::worker_config method. Any configs not provided in this implementation will fall back to the values provided in the app config.

[service.worker.enqueue-config]
queue = "default"

[service.worker.worker-config]
timeout = true
max-duration = 60000
max-retries = 25

Example

Worker definition

// Implement the `Worker` trait
#[async_trait]
impl Worker<AppContext, String> for ExampleWorker {
    type Error = roadster::error::Error;
    type Enqueuer = roadster::worker::SidekiqEnqueuer;

    // Optionally provide worker-level config overrides
    fn worker_config(&self, _state: &AppContext) -> WorkerConfig {
        WorkerConfig::builder()
            .retry_config(RetryConfig::builder().max_retries(3).build())
            .timeout(true)
            .max_duration(Duration::from_secs(30))
            .build()
    }

    async fn handle(&self, _state: &AppContext, args: String) -> Result<(), Self::Error> {
        info!("Processing job with args: {args}");
        Ok(())
    }
}

Register the worker with the processor

fn build_app() -> RoadsterApp<AppContext> {
    RoadsterApp::builder()
        // Use the default `AppContext` for this example
        .state_provider(Ok)
        // Register the Sidekiq worker service
        .add_service_provider(move |registry, state| {
            Box::pin(async move {
                let processor = SidekiqProcessor::builder(state)
                    // Register the `ExampleWorker` with the sidekiq service
                    .register(ExampleWorker)?
                    // Example of registering the `ExampleWorker` to run as a periodic cron job
                    .register_periodic(
                        ExampleWorker,
                        PeriodicArgs::builder()
                            .args("Periodic example args".to_string())
                            .schedule(cron::Schedule::from_str("* * * * * *")?)
                            .build(),
                    )?
                    .build()
                    .await?;
                registry.register_service(
                    SidekiqWorkerService::builder().processor(processor).build(),
                )?;
                Ok(())
            })
        })
        .build()
}

Enqueue a job for the worker

async fn example_get(State(state): State<AppContext>) -> RoadsterResult<()> {
    // Enqueue the job in your API handler
    ExampleWorker::enqueue(&state, "Example".to_string()).await?;

    Ok(())
}

Keyboard shortcuts

Roadster