We're no longer seeing errors from the PagerDuty endpoint and have confirmed escalations are polling successfully, meaning escalation messages (created in response to `/inc escalate`) are working once more.
We've followed up with PagerDuty to understand what caused this, and improved our retrying logic to make us more robust to future flakiness.
Posted Dec 17, 2022 - 19:35 UTC
Investigating
We're seeing a specific PagerDuty endpoint that we use to fetch escalation log entries fail, which is preventing us from updating escalations with recent data.
Creating escalations via `/inc escalate` is still working, and PagerDuty auto-create is unaffected. But the escalation message presented to the channel after `/inc escalate` may show stale data, even while the escalation has been ack'd.
Posted Dec 17, 2022 - 18:55 UTC
This incident affected: 🔌 Integrations (PagerDuty).