DocsBot AI - Chat APIs down – Incident details

All systems operational

Chat APIs down

Resolved
Major outage
Started 3 months agoLasted about 3 hours

Affected

Website

Operational from 3:45 AM to 5:12 AM, Degraded performance from 5:12 AM to 6:19 AM

Chatbots

Major outage from 3:45 AM to 5:40 AM, Partial outage from 5:40 AM to 6:19 AM

Updates
  • Resolved
    Resolved

    This incident has been resolved thanks to our workaround. We will continue to monitor our cloud provider incident to learn the root cause.

  • Monitoring
    Monitoring
    We implemented a temporary workaround for DNS resolution while our cloud provider is still experiencing a global outage and are currently monitoring the result. Bots seem to be working, though it may take a bit of time for server load and response times to decrease.
  • Identified
    Identified

    Our API cloud provider has confirmed the DNS resolution incident affecting many of their regional datacenters:
    "We are currently experiencing a partial outage in our Chicago, Dallas, Frankfurt, London, Los Angeles, Seoul, Singapore, Sydney, Tokyo, Toronto location. Our network engineers are actively working with our upstream providers to restore connectivity as quickly as possible. This has been effecting DNS resolution and overall network connectivity at this time."
    https://status.vultr.com/

    We are awaiting their resolution. When back online our chat APIs should also be restored.

  • Investigating
    Investigating

    We are currently investigating this incident. It appears to be DNS related with our API cloud provider.