Keeping Cloud Services Alive in the Face of Catastrophic Power Failure

When the recent storms on the east coast knocked out power to Amazon’s data center and took down Netflix, Pinterest, and Instagram, Panopto was affected too. None of our customers experienced any loss of data during the outage, but our cloud-hosted video platform was inaccessible for several hours.

In the aftermath of the outage, Panopto’s engineering team set out to address the root of the problem, so that future catastrophic power failures wouldn’t result in connectivity loss to our service. Late last week, we rolled out an update that protects against these types of interruptions and further reduces the chance of data loss during outages.

The update involves mirroring our video platform across Amazon’s availability zones. This means that as new information is uploaded to our servers, we continually replicate it to a standby system that sits on separate machines in a different location on a different power grid. If connectivity to the primary system is lost for any reason (anything from hardware failure to systemic power outages), we seamlessly redirect client traffic to the standby system. So even if Amazon’s primary and backup generators fail again, you’ll still be able to access the Panopto VCMS website, view videos, record new content, do live webcasts, and administer your system.

This is an important advancement for Panopto, and we believe is the standard that customers should expect from their online video platform and lecture capture providers. The update we rolled out today was relatively straightforward based on the multitenant architecture of our system. And since Panopto can scale to demand elastically while still managing all customers in one shared cloud, this is a natural evolution of our technology.

Other providers may not be able to make this shift so easily. Specifically, providers whose architecture is based on the use of dedicated virtual machines rather than a multitenant architecture will be prevented from mirroring their systems across availability zones, presenting their customers with a single point of failure.

If you’re currently using another cloud-based online video platform or lecture capture system, call your vendor and ask them how they guarantee uptime during a data center outage. If they don’t, give us a call. We’d love to discuss how Panopto can provide your organization with a reliable cloud-based platform for video.

-Ari Bixhorn

Published: September 04, 2012