Error rates have returned to normal after the rebuild of the cluster. All previous data will be available after the pipeline run completes. Resolving incident.
Posted 5 months ago. Jan 31, 2019 - 00:40 EST
Update
To speed recovery the development teams are going to rebuild the cluster and launch a pipeline run. Once the rebuild is complete, data will process as normal resolving the delay. Data for the preceding 48 hours may be unavailable until the pipeline run completes (4-6 hours).
Posted 5 months ago. Jan 31, 2019 - 00:28 EST
Monitoring
The cluster will continue to recover over the next few hours after all data processing has completed. ETR is approximately 4 hours.
Posted 5 months ago. Jan 30, 2019 - 23:58 EST
Update
The development team has implemented fix actions to reduce pressure on the cluster. Queries are expected to return to normal after all processing completes. Updates to follow.
Posted 5 months ago. Jan 30, 2019 - 23:22 EST
Identified
Real-time conversation detail records are experiencing delays. The development teams are engaged at this time for triage and remediation. Updates to follow.