Data Center Explained – Electrical Power

This article is a followup to my overview article posted earlier. It’s a high-level view of all things in a data center. Click here for the overview.

Electricity is the fuel of IT equipment. The electrical power system is probably the least understood system of the data center.

Why is electrical system needed?

IT equipment needs it. As infrastructure in data centers, electricity must be available continuously. That means multiple sources and multiple paths to the IT equipment is a must. This redundancy avoids outage when something fails. Most data center outages are expansive in terms of lost revenue and cost to recover. They are measured in thousands of dollars per MINUTE. A quick search shows an average of $9,000 per minute.

What is it?

In a nutshell, an electrical system in a data center is a chain of specialized equipment to take electrical power from utility companies and distribute it to computers. Here is a simplified, major electrical power equipment and their relationships:

Utility Power – The primary source of electric power from a utility company.
Meter – The primary source of “how much electricity is used?” the utility company owns this device.
Generator – Used as a backup to utility power. Typically runs on diesel fuel and switched on automatically when utility power is absent.
Switchgear – Switches the source power from the utility to generator when necessary. There are various types and commonly also known as Static Transfer Switch (STS), Automatic Transfer Switch (ATS), etc.
UPS – Uninterruptible Power Supply. In conjunction with the batteries, provide temporary electrical power. Their capacity is typically measured in minutes or hours.
PDU – Power Distribution Unit. This has two main functionality, 1) change the voltage from UPS (480V) to voltage levels computers use (240V). This functionality of also known as “Transformer,” 2) distribute electrical power by providing ‘whip’, ‘run’, ‘receptacle’ near the cabinets where rPDU gets connected. It also includes breakers for each, and sometimes this is handled by Remote Power Panel (RPP, mostly in older systems but still very common). Not shown in my diagram.
rPDU – Rack Power Distribution Unit. This is a large version of a power strip commonly used at home. It provides lots of outlets and usually has breakers to protect equipment power surge.
Computer – Main IT gears needed to run software applications.

While the generator and the UPS provide some backup electrical power, most data centers also completely replicate the system mentioned above. So it more like this diagram:

As you can see in the diagram, the computers take at least 2 electrical power connections. Recently, each source is color-coded and blue and red seems to be common to indicate A vs B source.

Who’s involved?

The facility team takes care of the electrical system. Most of the equipment is installed during building construction, and there are some periodic expansions and maintenance done to keep up with the overall addition of computer equipment.

The computer equipment can be plugged in either by the facility team or the IT team. Ideally, a specific outlet on the rPDU is pre-determined by checking the actual usage data. It will avoid ‘prematurely tripping the breakers on the rPDU. How this is achieved will be covered in another article.

Here are some of the main vendors:

How does it work?

This is very complicated. I’ll cover this in another article but I will mention one thing: one-line diagram

Sample one-line Diagram

Every data center has an electrical one-line diagram. It represents how all this equipment is connected. Unfortunately, this is not kept up to date, and only a few in the facility team member know about it.

Electrical power is a shared system, and everything connecting to the system impacts every equipment that’s connected.

More to come later. It’s going to get really geekie.