Between 07:33 UTC and 21:38 UTC on 16 Sep 2023, some parts of the VALD platform experienced persistent issues trying to connect to Azure SQL Database in East US.

All users will now have access to VALD products and services as per normal.

If you or a client are still encountering any issues, please reach out to support@vald.com

Microsoft have advised they are manually rebooting affected instances, which should bring the databases back online. We are seeing promising signs of recovery for of our infrastructure.

Remediation efforts from Microsoft are ongoing - Microsoft has suggested that some affected storage and compute nodes have started to recover.

We are expecting to see some VALD services begin recovering soon.

Updates will follow as portions of our infrastructure are confirmed to have healed.

This incident started at 07:33 UTC on 16 Sep 2023; Microsoft has now confirmed that they are working to return the affected compute and storage resources to a healthy state.

The latest update from Microsoft provides no further insight as to how their remediation efforts are progressing.

The VALD support team suggests clients use our applications offline data capture functionality. Some clients have confirmed that toggling a VPN to Europe or Australia is a viable short-term workaround.

The latest update from Microsoft suggests that the scope of the issue also includes compute resources, not just storage. We are exploring potential mitigation strategies.

Users outside the North and South American regions should be unaffected by this incident.

We've been alerted that our SQL database instance in the Azure East US region is one of a number currently performing at a degraded state. VALD users in or near the US region might experience long loading times across our apps or even time-outs. We'll post updates as more information becomes available.