Task queue backup

Incident Report for Crunch.io

Resolved

No indication of any remaining queue or latencies.

Posted Jul 01, 2020 - 12:07 PDT

Update

No workers are blocked and tasks in queue are being processed normally; queues are diminishing.

Posted Jul 01, 2020 - 09:52 PDT

Monitoring

Some server processes became unresponsive, causing tasks to be “queued” but not processed in a timely manner.
We have restarted those processes and added additional workers to address the queue.

Posted Jul 01, 2020 - 08:32 PDT

This incident affected: Crunch.io Web Application and API, Dataset Streaming, and Individual Dataset Access.