If You Don’t Have Electricity, You’re Gonna Have a Bad Time

By: Tony DePrato | Follow me on Twitter @tdeprato

 

Many schools are still self-hosting the majority of their information systems instead of using cloud-based solutions, co-location solutions, or data centers offering more traditional hosting.

The question has to be asked, are these schools taking the proper steps to protect their data, their systems, and the integrity of their entire architecture?

Recently, my ITBabble co-host Patrick went to a fairly popular tech conference and stumbled upon two schools that are regularly affected by power outages in their self-hosted data centers. These schools reported to Patrick that they did not only have power failures, but lacked business continuity plans.

Without a business continuity plan (BCP), business operations come to a halt when power outages happen. For example, if accounting and payroll systems are offline, employees would not be paid on time. In some states, this BCP failure would result in the school receiving a fine for missing payroll.

After hearing this story, I decided to write a couple of blogs posts focused on risk management.

Systems Need Electricity

If your school is going to host any core system internally, then the school needs to plan for power outages. There are very few places in the United States immune to power outages. I compiled and condensed a spreadsheet to demonstrate how common outages are based-on 2017 data. The spreadsheet below represents this data.


Link to Spreadsheet: https://docs.google.com/spreadsheets/d/e/2PACX-1vSd-zIhs8sNH7doMsrnwkEGMtRwGkbDWnc0Q8GSpNUEECmSyKd6bu-ldjbI1P6oAzU6Rc40Ye5NXQEu/pubhtml

Link to Original Data Source: https://www.eia.gov/electricity/data/eia861/


Since the data is available for previous years, those who are self-hosting should download the historical data and confirm the median number of minutes per year their region is impacted by power outages. They should also find the median amount of time per outage they are impacted.

From that analysis, a school can determine what their requirements are to be reasonably prepared for a power outage.

Being Reasonable

Risk management does not require a school to prepare for a threat that is historically unlikely to occur. For example, and school in Utah would not draft a plan for a hurricane.

In the official data, power outages are reported by state and by provider. If a school determines their median value and frequency is 60 minutes 3 times a year, then they need to ensure all their critical systems can run for 60-120 minutes when the power is out. This is enough time to shut everything down and protect system integrity. However, it is not enough time to execute a complex process like payroll.

If a school is internally running services like payroll, I would suggest the power backup standard be set to the task execution time, of the most critical task. For example, if payroll takes 4-5 hours to complete every Friday, then the systems needed for payroll need to be powered for 4-5 hours in the event of an outage. This often means the majority of the core infrastructure needs to also be up and running for 4-5 hours.

Supporting the Business Continuity Plan

Aside from setting-up backup power systems and standards, here are some tasks that the IT department should be completing (or overseeing) to support the overall BCP in the event of a long term power outage:

  1. Creating hard copies of core demographic data for all students, parents, and employees
  2. Creating hard copies of all schedules, including after school programs and transportation
  3. Creating hard copies of phone lists (phone trees)
  4. Organizing all hard copy backups into folders that are available in multiple locations; shredding outdated documents
  5. Creating portable backups of data needed for core school operations; this includes a full backup of all databases in at least two formats
  6. Testing all electronic backup systems responsible for power and data backups
  7. Connecting with a similar school, and forming a good relationship, for additional tech support when emergencies happen; this should be symbiotic
  8. Contracting and maintaining a portable 4G-5G router for ad-hoc network deployment
  9. Building an offsite data backup system that is immune to any local threats; this data can be compressed
  10. Annually reviewing all risk management policies and procedures with a team from every major department

If a school is self-hosting, they should be mostly immune to power outage threats with proper planning.

If the equipment needed to provide backup power is not available, or simply too expensive, then reconsidering cloud services would be the next step in mitigating future problems.

 

About Tony DePrato

about.me/tonydeprato
This entry was posted in management, Tony DePrato and tagged , , , . Bookmark the permalink.

Leave a reply! The IT Babble Team Need Feedback.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s