Development

How to Run a PHP Job in the Background

A dependable PHP background job has a durable record, a worker outside the web request, and a clear policy for retries and duplicate execution. Starting a shell command from a controller is not enough to guarantee that work survives a process restart.

For a small application, a database-backed queue and a supervised PHP CLI worker can be sufficient. Use a maintained queue library when it already fits the project instead of building every operational feature yourself.

Separate acceptance from execution

The web request validates input, authorizes the action, writes a job, and returns a job identifier or pending status. The worker loads that job and performs the expensive operation.

When the job must accompany a database update, save both within the same transaction or use a transactional outbox. Otherwise the application can save the business record and crash before enqueueing the required work.

Store a small typed payload containing stable record identifiers. Avoid serializing entire service objects, credentials, or a snapshot of private data that will become stale before execution.

Give each background job an explicit lifecycle

A useful minimal model includes queued, running, succeeded, and failed states, plus attempt count, available-at time, lease expiry, and a sanitized last-error summary.

The claim operation must be atomic. Two workers must not independently read the same queued row and both assume ownership. With a relational queue, use the database's documented locking behavior; with a queue service, use its visibility or acknowledgment mechanism.

A lease allows another worker to reclaim work after a crash. Set its duration around the task's expected runtime and renew it carefully for long tasks. A reclaimed job can overlap an old worker that is merely slow, so the handler still needs protection against duplicate effects.

Keep the worker bounded

This simplified CLI loop demonstrates control flow, not a complete queue implementation. The queue methods are pseudocode contracts that must be supplied by the chosen library or application.

repeat until shutdown or worker lifetime limit:
    atomically claim one available job with a lease
    if no job exists:
        wait briefly
        continue
    validate the job type and payload
    perform an idempotent unit of work
    acknowledge success only after durable completion
    on retryable failure:
        schedule a bounded retry with backoff
    on permanent failure or exhausted attempts:
        mark failed and alert the responsible operator

Run the actual worker under the process manager supported by your deployment, such as systemd on a suitable Linux host. Configure restart behavior, environment, working directory, log handling, and a graceful stop timeout. PHP's CLI documentation describes running PHP outside the web server; supervision is a separate operating concern.

Do not concatenate user input into a shell command. A job type should select a known handler, not an arbitrary executable.

Design retries around side effects

A worker may crash after sending an email or creating a remote record but before acknowledging success. The queue will then deliver the job again. Use a stable business idempotency key, a storage uniqueness constraint, or the remote provider's documented idempotency support.

Keep retryable failures distinct from permanent validation errors. Apply exponential backoff with jitter and a finite attempt or elapsed-time budget. Repeating a bad payload forever makes the queue harder to operate without improving its outcome.

For batch work, checkpoint at a durable boundary. Restarting should resume from the last committed unit, not from a local variable lost with the process.

Observe what the worker is doing

Track queue age, ready jobs, running jobs with expired leases, retries, permanent failures, and processing time. A process being alive does not prove jobs are advancing.

Test worker termination during a job, duplicate delivery, lease expiry, malformed payloads, missing dependencies, and a long-running task during deployment. Confirm that shutdown stops claiming new work and leaves unfinished work recoverable.

For structuring handlers separately from controllers, see incremental PHP refactoring. Discuss a background-processing review when request timeouts are hiding unfinished work.