Some Dashboards Unavailable
Incident Report for Fin
Resolved
We have seen improved performance following our fixes, and heard from affected customers that they have not been experiencing dashboard failures anymore.
Posted Nov 10, 2022 - 19:55 UTC
Monitoring
A fix has been implemented and released for a problem that caused some dashboards to be unavailable for some users.
Most users should now be able to refresh the Fin Console and affected dashboards should load.

The issue allowed transient query failures to be cached and reused for long periods of time, even after the underlying query became healthy.

The cache time has been reduced to 10 minutes, which substantially reduces the amount of time that this problem can occur. While a query failure may still be cached for this duration, users now have the ability to refresh the page after 10 minutes and can expect that the dashboard will load at that time.

We are continuing to investigate ways to further improve the user experience around transient failures.
Posted Nov 08, 2022 - 23:38 UTC
Identified
Fin is aware that some dashboards are failing to load for some users in our Console application.
We have identified and deployed a mitigation measure, and are working to confirm that the issue is fully resolved.

Some users have been able to work around the failures by changing a filter value such as daterange.
Posted Nov 07, 2022 - 16:00 UTC
This incident affected: Analytics Data.