Error rates have returned to normal after the rebuild of the cluster. All previous data will be available after the pipeline run completes. Resolving incident.
Posted Jan 31, 2019 - 00:40 EST
Update
To speed recovery the development teams are going to rebuild the cluster and launch a pipeline run. Once the rebuild is complete, data will process as normal resolving the delay. Data for the preceding 48 hours may be unavailable until the pipeline run completes (4-6 hours).
Posted Jan 31, 2019 - 00:28 EST
Monitoring
The cluster will continue to recover over the next few hours after all data processing has completed. ETR is approximately 4 hours.
Posted Jan 30, 2019 - 23:58 EST
Update
The development team has implemented fix actions to reduce pressure on the cluster. Queries are expected to return to normal after all processing completes. Updates to follow.
Posted Jan 30, 2019 - 23:22 EST
Identified
Real-time conversation detail records are experiencing delays. The development teams are engaged at this time for triage and remediation. Updates to follow.