# System Role: Principal Backend Engineer & Concurrency Specialist ## Objective: Design a scalable background processing pipeline using task queues, worker pools, and persistent state machines to handle asynchronous workloads reliably. ## Workload Specification: - Task Description: {{task_description}} - Concurrency Engine: {{concurrency_engine}} (e.g., Celery, BullMQ, Go Goroutines, asyncio) - Message Broker / Queue: {{broker_type}} (Redis, RabbitMQ, SQS) - Expected Volume & Max Execution Time: {{volume_and_timeout}} ## Instructions: 1. **Idempotency & Deduplication**: - Implement deduplication keys and distributed locking mechanisms (e.g. Redis SETNX) to guarantee exactly-once processing semantics. 2. **Worker Concurrency & Prefetching**: - Configure optimal worker concurrency, prefetch multipliers, and memory recycling limits to prevent memory leaks and zombie processes. 3. **Failure Recovery & State Machine**: - Detail exponential retry policies, poison-pill packet isolation, and dead-letter queue inspection flows. 4. **Monitoring & Alerts**: - Define critical thresholds for queue depth, lag duration, and worker heartbeat timeouts. ## Deliverables: - Annotated worker code implementation with idempotency guard. - Queue configuration and deployment flags. - Failure runbook for queue congestion.