A Coordinated Electric System Interconnection Review—the utility’s deep-dive on technical and cost impacts of your project.

Challenge: Frequent false tripping using conventional electromechanical relays
Solution: SEL-487E integration with multi-terminal differential protection and dynamic inrush restraint
Result: 90% reduction in false trips, saving over $250,000 in downtime

The three operating regions you have to design to

Device Output vs voltage Response Best suited to Main limitations
Mechanically switched capacitor or reactor Proportional to voltage squared Seconds; discrete steps; limited switching operations per day Steady-state reactive supply, voltage profile, loss reduction No dynamic capability; step voltage change on switching; capability collapses when most needed
Static var compensator Capacitive branches proportional to voltage squared A few cycles; continuously controllable Continuous control where cost matters and deep voltage support is not the driver Square-law capability loss; harmonic filters are part of the plant and interact with the network
STATCOM Approximately proportional to voltage — constant current capability One to two cycles closed loop; converter response faster still Voltage stability margin, weak interconnections, fast disturbance recovery, flicker and unbalance compensation Higher capital cost; converter losses; adds a converter and its control dynamics to the network
Synchronous condenser Governed by machine capability and excitation Excitation response in the hundreds of milliseconds; inherent inertial response instantaneous System strength and inertia, short-circuit contribution, black start support Rotating plant with maintenance and losses; slower controlled response than a converter
STATCOM with energy storage Reactive as a STATCOM, plus real power within the storage rating As STATCOM for reactive; real power limited by storage Where a real power deficiency is part of the problem Cost and complexity of the storage; different failure and maintenance profile
Stage Keentel scope Outcome
Screening Series-compensation proximity, radial contingency review, UIF and short-circuit ratio screening, frequency scans Early identification of which SSO types apply
Detailed studies PSCAD/EMTDC EMT studies with vendor real-code models, impedance-based and frequency-scan analysis Clear evidence of stability margins or risk
Model quality EMT model review, benchmarking against positive-sequence models, model acceptance support Models that system operators accept
Mitigation Control retuning recommendations with vendors, SSDC and bypass logic requirements, protection and monitoring specifications Practical, coordinated solutions
Interconnection support SSR and SSO study reports for ISO and utility requirements, response to reviewer comments Fewer delays in the interconnection process

Resilience: designing CDUs to the facility Tier objective


Liquid cooling makes heat removal more critical, not less. Cold plates hold only seconds of thermal buffer. If TCS flow stops, processors throttle or shut down very quickly. Resilience design must therefore cover:


  • Pump redundancy: N+1 pumps at minimum, with automatic changeover.
  • CDU redundancy: N+1 CDUs on a common TCS header, or 2N CDUs serving A and B manifolds for fault-tolerant designs.
  • Power to CDUs: dual-corded CDUs fed from A and B sources, with pumps and controls on UPS power so coolant keeps flowing through a utility failure and generator start. This is the liquid-cooling equivalent of continuous cooling.
  • Facility water continuity: chilled water or condenser water must also continue, through thermal storage, UPS-backed pumps or a high enough FWS temperature that the TCS can coast on stored thermal mass for the transfer period.
  • Concurrent maintainability: isolation valves so any CDU, pump, filter or manifold section can be serviced without stopping the others.
  • Leak management: leak detection, drip trays, automatic isolation valves and procedures for fluid spills.


Electrical engineering considerations


CDUs and liquid-cooling plants are electrical loads with their own design requirements:


  • Pump motors and VFDs. Large CDUs use variable-frequency drives on their pump motors. These introduce harmonic current that must be included in the facility's harmonic study and IEEE 519 evaluation.
  • Critical mechanical load on UPS. Putting CDU pumps on UPS increases UPS and battery sizing. The mechanical UPS load must be included in redundancy and ride-through calculations.
  • Feeder and protection design. Dual feeds, automatic transfer, selective coordination and arc-flash labelling apply to CDU power distribution just as they do to IT power.
  • Generator transient response. CDU pumps restarting together after a transfer create motor inrush on the generators. Staged restarts and soft acceleration reduce the step load.
  • Load dynamics. AI training loads can swing rapidly between high and low power. Cooling controls must track those swings without temperature overshoot, and electrical systems must handle both the IT and the cooling load changes.


Commissioning a liquid-cooled data hall


  1. Flush and clean the TCS piping and manifolds to the specified cleanliness, using high-purity flush water.
  2. Fill and vent with the specified coolant, verifying glycol concentration and chemistry.
  3. Pressure test for leaks at design and test pressures, including quick-disconnect fittings.
  4. Verify sensors and controls: flow meters, temperature sensors, pressure transmitters, leak detection and BMS points.
  5. Load test with heat load banks connected through the cold-plate interfaces or rack manifolds, across the operating range.
  6. Prove the heat balance on both loops at several load steps, and the approach temperature against the design.
  7. Test failure scenarios: pump failure, CDU failure, power transfer to generator, valve failure and loss of facility water, confirming temperatures stay within the IT vendor's limits.


How Keentel Engineering helps


Keentel Engineering supports data center owners, developers and EPCs with the electrical engineering that liquid-cooled facilities depend on, and coordinates with the mechanical engineer of record and cooling vendors on the integrated design.


Data Center Tiers I–IV Explained: What Concurrent Maintainability and Fault Tolerance Mean for MEP and Electrical Engineers

Data Center Tiers I–IV comparison showing redundancy, concurrent maintainability and fault tolerance.
Calendar icon. D

November 14, 2026 | Blog

Part A — Data Center Tiers at a Glance

Every data center owner eventually asks the same question: how much can this facility tolerate before the IT load goes dark? The four-level Tier classification is the industry's most widely used answer. It describes the site infrastructure (power, cooling, fuel, controls and the spaces that house them), not the servers, storage or network, and it judges that infrastructure against two tests. Can every component and every distribution path be taken out of service for planned work without affecting the IT load? And can any single unplanned failure or event happen without affecting it?

Those two tests sound simple. In practice they reach into every breaker, valve, control power supply, fuel pump and cable tray in the building. A design with twice the UPS modules it needs can still fail both tests because of one shared switchboard bus, one paralleling controller or one chilled-water header valve. That is why Keentel Engineering approaches Tiers as a systems-engineering problem rather than an equipment count. A Tier III or Tier IV facility is only as reliable as its weakest component, and successful delivery needs coordinated electrical, mechanical and controls engineering from concept through integrated systems testing.

This guide is written for the people who have to turn a Tier objective into drawings, equipment, studies and test scripts. It explains what each Tier really requires, corrects several widely repeated simplifications, works through the numbers that drive equipment selection, and maps Tier design onto US codes and standards: the National Electrical Code (NEC, NFPA 70), NFPA 75, 76, 110, 111, 72, 13, 2001 and 855, NFPA 70E, IEEE 1584, IEEE 3006.x, ASHRAE thermal guidelines and ASHRAE 90.4, UL equipment standards, and the FERC and NERC activity now shaping large-load interconnection.

Companion reading. Keentel's earlier article on data center Tiers introduces the four levels and the certification landscape. This guide goes deeper into what Tiers mean for MEP and electrical engineers: sizing, availability mathematics, concurrent maintainability reviews, code interaction and commissioning. Related Keentel blogs cover data center load calculations, the anatomy of a 1 GW campus, RMU versus MV switchgear for data centers, 800 VDC power architectures and data center rack engineering.

Who this guide is for

  • Owners, developers and colocation operators deciding which Tier objective to pursue, and what it will cost in capital, space, energy and operations.
  • Electrical and mechanical engineers of record who must deliver concurrent maintainability or fault tolerance in every operating mode, not only in normal operation.
  • Controls, BMS and EPMS integrators, whose systems are often the hidden single point of failure in otherwise redundant plants.
  • Commissioning agents and operators, who prove and then preserve the Tier through Level 1 to Level 5 testing and day-to-day operation.
  • Utilities and transmission planners serving large data center loads at 69 kV to 345 kV and above.

The four Tiers in one table

Attribute Tier I Tier II Tier III Tier IV
Performance objective Basic capacity Redundant capacity components Concurrently maintainable Fault tolerant
Capacity components (UPS, generators, chillers, pumps, cooling units) N N+1 (redundant components) N+1 (redundant components) Enough that losing any one system still leaves N; commonly 2N, 2(N+1) or distributed redundant
Distribution paths to the critical load One One At least two: one active, one alternate At least two, simultaneously active
Planned maintenance Requires IT shutdown Component work possible; path work requires shutdown Any component or path element can be removed on a planned basis without IT impact Same as Tier III
Single unplanned failure Likely IT outage Component failure usually covered; path failure causes outage Not required to be tolerated; many failures cause an outage Must not affect the IT load, including the consequences of the failure
Compartmentalization Not required Not required Not required Required
Continuous cooling Not required Not required Not required Required
Typical uses Small rooms, test, development, edge Small and medium enterprise, regional Enterprise, colocation, cloud, healthcare, finance Trading, government, payment networks, highest-consequence workloads

Fact check: what the common summaries get right and wrong

Short summaries of the Tiers circulate widely. Most are directionally right but oversimplified in ways that cause real design errors. Keentel's review of the most common claims:

Common claim Keentel assessment
"Tier III has multiple independent distribution paths." Partly right. Tier III needs redundant distribution paths, but only one needs to be active; the second can be an alternate path used during maintenance. Many Tier III designs make both active (A and B feeds to dual-corded IT), which is good practice but more than the minimum.
"Tier III is typically N+1." Correct for capacity components, but N+1 alone does not make Tier III. Without a second distribution path and maintainable switchgear, controls, fuel and piping, an N+1 facility is Tier II.
"Tier IV is typically 2N or 2N+1." A common implementation, not a requirement. The Tier Standard is performance based: it requires fault tolerance, not a particular notation. Distributed redundant and some block redundant designs can meet the objective with less installed capacity than 2N.
"Tier IV withstands a single unplanned failure without impacting IT." Right, and it must also tolerate the consequences of that failure, such as fire, smoke or water damage in one location. That is why compartmentalization and continuous cooling are explicit Tier IV requirements, and why topology alone is not enough.
"Tier II maintenance may require shutdown of certain systems." More precisely: redundant components can be serviced while others carry the load, but any work on the single distribution path (switchboard, UPS output bus, distribution boards, cables) requires an IT shutdown.
"Tier I has limited protection against equipment failure." Tier I still requires a UPS, dedicated cooling and an engine generator. It is not "no protection"; it is no redundancy.
Availability percentages (99.671%, 99.741%, 99.982%, 99.995%) define the Tiers. Not part of the current Tier Standard. These legacy figures summarized observed site performance decades ago. The Tiers are defined by topology and performance objectives. Keentel calculates project-specific availability instead (Worked Example 4).
Utility feeds count towards the Tier. No. The Tier Standard treats on-site engine generation as the reliable power source and utility power as an economic alternative. Dual utility feeds add value but do not replace generator capacity and redundancy.
Any engineer or contractor can certify a facility as Tier III. Only the Uptime Institute certifies Tiers. Others design to Tier objectives. Equivalent frameworks with their own terminology are ANSI/TIA-942-C Rated-1 to Rated-4 and ANSI/BICSI 002 Classes F0 to F4.

Eight findings for decision makers

1. The step from Tier II to Tier III is about the path, not the equipment. It is the second distribution path and the maintainability of everything on it that delivers concurrent maintainability.

2. The step from Tier III to Tier IV is about failure and its consequences. Fault tolerance, compartmentalization and continuous cooling are the additions; the capacity count can stay similar.

3. Redundancy has diminishing returns unless common-cause failures are removed. In Worked Example 4, two independent paths cut expected unavailability from about 200 minutes a year to under 0.1 minute in theory, but a 5% common-cause share brings it back to about 10 minutes.

4. Utilization drives cost. A 2N UPS system can never be loaded above 50%; 3-to-make-2 distributed redundancy allows about 67%; 4-to-make-3 about 75%. The difference is millions of dollars of equipment at 10 MW and above.

5. The generator plant is the Tier's power source. Generator rating type, fuel storage (a commonly cited minimum of 12 hours on site), fuel delivery paths and paralleling controls must meet the same Tier logic as the UPS.

6. Controls are the most frequently missed single point of failure. One PLC, one DC control power panel or one network switch can defeat an otherwise maintainable topology.

7. US code compliance and Tier objectives are separate obligations. The NEC and NFPA standards are mandatory; the Tier is a performance objective that sits on top. Neither replaces the other.

8. The Tier is proven in commissioning and kept in operations. Integrated systems testing proves the topology; operational practice and human factors decide whether it holds over 20 years.

Key terms

Term Meaning
N The capacity needed to carry the design critical load, with no spare.
N+1 N plus one redundant capacity component of the same type.
2N Two complete, independent systems, each able to carry the full design load.
2(N+1) Two independent systems, each with its own redundant component. Often written 2N+2 in specifications; sometimes loosely called 2N+1.
Distributed redundant Three or more systems, each loaded below its rating, so that the loss of any one leaves the others able to carry the load (for example, 3-to-make-2 or 4-to-make-3).
Block redundant / catcher Several primary systems backed by a shared reserve system that "catches" the load of a failed system through static or automatic transfer switches.
Capacity component Equipment that provides capacity: UPS module, generator, chiller, pump, cooling unit, cooling tower, fuel pump.
Distribution path The route that delivers power or cooling to the critical load: switchgear, buses, breakers, transfer switches, cables, pipes, valves.
Concurrent maintainability Every capacity component and distribution element can be removed from service on a planned basis without affecting the critical load.
Fault tolerance A single unplanned failure or event, and its consequences, does not affect the critical load.
Compartmentalization Physical separation of redundant systems and paths so that one event cannot affect both.
Continuous cooling Uninterrupted heat removal from the critical space through a utility failure and generator start sequence.
Single point of failure (SPOF) Any element whose failure alone interrupts the critical load.
Availability (A) The fraction of time a system is able to perform its function, commonly A = MTBF / (MTBF + MTTR).
Keentel Engineering data center tiers technical figure or engineering formula 1
Figure 1. Simplified electrical topologies for each Tier objective. Real designs include medium-voltage switchgear, multiple transformers, static switches, maintenance bypasses and mechanical systems arranged to the same logic.

Part B — Data Center Tiers I to IV: What They Mean for MEP and Electrical Engineers

How each Tier objective translates into power, cooling, fuel, controls and fire protection design; how to size the equipment and quantify what redundancy buys; and how to prove the result in commissioning, using US codes and standards.

1. Where the Tiers come from, and who decides

The Tier classification system was developed by the Uptime Institute in the 1990s to give owners, designers and operators a common language for data center site resilience. Today it is published as the Tier Standard: Topology, which defines the four Tier objectives, and the Tier Standard: Operational Sustainability, which covers how a facility is staffed, maintained and managed. "Tier" and "Tier Certification" are Uptime Institute terms, and only the Uptime Institute certifies facilities against them.

Certification has three distinct steps, and owners often confuse them:

Certification What is assessed When it happens What it proves
Tier Certification of Design Documents (TCDD) Drawings and specifications Before construction documents are complete The design, on paper, meets the Tier objective
Tier Certification of Constructed Facility (TCCF) The built facility, through on-site demonstrations After construction and commissioning The facility, as built, performs to the Tier objective under maintenance and failure scenarios
Operational Sustainability Staffing, maintenance, training, procedures, planning During operation The facility is operated in a way that preserves its Tier performance

A design certification does not certify the building. Many facilities are marketed as "Tier III design" without ever demonstrating the constructed facility. Owners and tenants who depend on the Tier should ask which certification applies.

Two other frameworks engineers will meet

ANSI/TIA-942-C (2024) is the Telecommunications Industry Association's data center infrastructure standard. It defines Rated-1 to Rated-4 levels across telecommunications cabling, architectural, electrical and mechanical systems. The levels are conceptually close to Tiers I to IV, but the requirements and the assessment model differ, and conformity can be self-assessed or assessed by third-party bodies.

ANSI/BICSI 002 is the BICSI design and implementation best-practices standard. It describes availability Classes F0 to F4, where F0 is a single-path facility without a generator and F4 approximates fault tolerance. It is a design guide rather than a certification scheme, and it is particularly useful for its detailed electrical, mechanical and cabling guidance.

Objective Uptime Institute ANSI/TIA-942-C ANSI/BICSI 002
Basic capacity Tier I Rated-1 Class F1 (F0 below it)
Redundant components Tier II Rated-2 Class F2
Concurrently maintainable Tier III Rated-3 Class F3
Fault tolerant Tier IV Rated-4 Class F4

The alignment is conceptual. A facility that meets TIA-942 Rated-3 should be described as Rated-3, not as Tier III; a facility designed to Tier III objectives but not certified should be described as "designed to Tier III objectives". Precise language avoids contractual disputes with tenants and lenders.

Topology versus performance

The modern Tier Standard is performance based. It states objectives (concurrent maintainability, fault tolerance) and tests the design and the facility against them. It does not prescribe a particular one-line diagram, equipment brand or redundancy notation. Two designs with completely different architectures can both meet Tier III, and a design that looks "2N" on a sketch can fail Tier IV because of a shared control system. The engineer's job is to demonstrate the objective, component by component and path by path, not to pick a notation.

How Keentel Engineering helps here: Keentel helps owners choose the right framework and the right objective before design begins. We prepare a Tier basis-of-design document that states the objective for each system (electrical, mechanical, fuel, controls, fire protection), the redundancy philosophy, the operating modes that must be supported and the codes that apply, so every discipline designs to the same target and the certification reviewer sees a coherent intent.

2. Redundancy notation and what it really costs

Redundancy is always stated relative to N, the capacity needed to carry the design critical load. The notation is simple; its consequences for equipment count, loading and efficiency are not.

Configuration Systems Each system rated for Maximum normal loading Loading after one system fails Tolerates
N 1 100% of load 100% Outage Nothing
N+1 1 (with a spare module) 100% + one module N/(N+1) 100% of N modules One module failure; not a bus or path failure
2N 2 100% of load 50% 100% Loss of a complete system or path
2(N+1) 2 (each with a spare) 100% + one module Below 50% Below 100% Loss of a system plus a module in the surviving system
3-to-make-2 (distributed redundant) 3 50% of load 66.7% 100% Loss of any one system
4-to-make-3 (distributed redundant) 4 33.3% of load 75% 100% Loss of any one system
Block redundant (catcher) n primaries + 1 reserve Each primary carries its block; the reserve matches the largest block Primaries up to 100% of their rating Reserve picks up one failed block Loss of any one primary system

The maximum loading in normal operation follows from the requirement that the surviving systems must carry the full load after one fails. For a distributed redundant design with n systems, each rated S, carrying load P:

Keentel Engineering data center tiers technical figure or engineering formula 2

That gives the familiar limits: 50% for 2N (n = 2), 66.7% for 3-to-make-2 and 75% for 4-to-make-3. In practice designers keep a margin below these limits for load growth, measurement error and imbalance between systems.

Worked Example 1: UPS architecture for a 10 MW data hall

Given. Critical IT load 10 MW at unity power factor (modern UPS modules are rated in kW at power factor 1.0). UPS modules of 1,250 kW. Compare N+1, 2N, 2(N+1), 3-to-make-2 and 4-to-make-3.

N. 10,000 ÷ 1,250 = 8 modules.

Configuration Modules Installed UPS capacity Normal loading per system Loading after worst single failure
N+1 (one system) 9 11,250 kW 10,000 ÷ 11,250 = 88.9% 100% (8 modules)
2N 2 × 8 = 16 20,000 kW 5,000 ÷ 10,000 = 50.0% 100%
2(N+1) 2 × 9 = 18 22,500 kW 5,000 ÷ 11,250 = 44.4% 88.9%
3-to-make-2 3 × 4 = 12 15,000 kW 3,333 ÷ 5,000 = 66.7% 100%
4-to-make-3 4 × 3 = 12 15,000 kW 2,500 ÷ 3,750 = 66.7% 3,333 ÷ 3,750 = 88.9%

Observations.

  • 2N needs 16 modules; either distributed redundant design needs 12, a saving of four 1,250 kW modules and their batteries, switchgear, space and cooling. At an illustrative installed cost of about $450 per kW of UPS capacity, that is roughly $9.0 million for 2N against $6.8 million for 3-to-make-2.
  • With discrete module sizes, 4-to-make-3 does not save modules here: 3.33 MW per system rounds up to three 1,250 kW modules. Module size should be chosen together with the architecture.
  • N+1 is the cheapest, but it protects only against a module failure. The UPS output switchboard and everything downstream remain single points of failure. That is Tier II territory unless a second distribution path is added.
  • The normal loading of 2N systems (50% or less) puts UPS modules in the lower part of their efficiency curve. Modern double-conversion modules are efficient across a wide range, and many offer high-efficiency or multi-mode operation, but the loss difference is still worth calculating over a 15 to 20 year life.

How Keentel Engineering helps here: Keentel compares redundancy architectures on equal terms: equipment counts, installed capacity, utilization, efficiency at expected loading, footprint, failure scenarios, maintainability and life-cycle cost. We recommend module sizes and system groupings together, because a good architecture with the wrong module size can lose most of its advantage.

3. Tier I: basic capacity

A Tier I facility has dedicated space for IT equipment, a UPS to ride through power sags, spikes and short interruptions, dedicated cooling equipment that runs outside office hours, and an engine generator to carry the load through extended utility outages. It has no redundant capacity components and a single, non-redundant distribution path.

Electrical. Typically a single utility service (often 13.8 kV, 12.47 kV or 480 V depending on size), one generator, an automatic transfer switch (ATS), one UPS system and one distribution path through switchboards, PDUs and remote power panels to the racks. Any planned work on the generator, ATS, switchboard, UPS or distribution requires an IT shutdown, and any failure of them causes one.

Mechanical. N cooling units or chillers, single piping and controls.

What Tier I is not. It is not "no protection". It is a real data center infrastructure with a UPS, generator and dedicated cooling, designed for a business that can schedule downtime. Server rooms in office buildings often fail even Tier I because they lack a generator or dedicated cooling.

Where it fits. Test and development environments, small businesses with tolerant workloads, and edge sites where the application is replicated elsewhere.

How Keentel Engineering helps here: For smaller facilities and edge sites, Keentel delivers right-sized electrical designs that meet the NEC, NFPA 110 and NFPA 75 cleanly at minimum cost, with a clear upgrade path (spaces, conduits, switchgear sections and tie provisions) so the owner can move to a higher objective later without tearing out what was built.

4. Tier II: redundant capacity components

Tier II adds redundant capacity components: N+1 UPS modules, engine generators, chillers, pumps, cooling units and similar equipment. The distribution path remains single.

Electrical. Generators paralleled on a common bus with one spare unit; UPS modules in an N+1 parallel system; but one generator paralleling switchboard, one main switchboard, one UPS input and output board, and one set of PDUs. A failed UPS module or generator no longer drops the load, and individual modules can be serviced. But work on any bus, breaker, static switch, cable or panel in the path still means an IT shutdown, and a fault on any of them still causes an outage.

Mechanical. N+1 chillers, pumps and cooling units on a single piping system with single valves and headers.

The hidden trap. Tier II designs often look impressive on an equipment schedule. The weakness is the path. A UPS output switchboard bus fault, a failed paralleling controller or a cracked chilled-water header takes everything down regardless of how many spare modules exist.

Where it fits. Small and medium enterprises and regional facilities where a few planned maintenance windows a year are acceptable.

How Keentel Engineering helps here: Keentel's single-point-of-failure review of Tier II facilities identifies which path elements carry the most risk and which inexpensive changes (bypass provisions, a tie breaker, a second static switch, a temporary-generator connection, a looped header) remove the largest share of outage exposure without a full Tier upgrade.

5. Tier III: concurrently maintainable

Tier III is the most widely specified objective for enterprise, colocation and cloud facilities, because it removes the large majority of planned downtime at a moderate cost premium. The requirement is easy to state and hard to deliver: every capacity component and every element of every distribution path can be removed from service on a planned basis without affecting the critical load.

What the requirement covers

Engineers tend to check the obvious items (UPS modules, generators, chillers) and miss the rest. A Tier III review must cover, at minimum:

System Items that must be concurrently maintainable
Medium-voltage power Utility switchgear and generator switchgear buses, breakers, ties, protective relays and their DC control power, potential transformers, cable sections, MV/LV transformers
Generator plant Each generator, its starting batteries and chargers, paralleling switchgear buses and breakers, master controls, load-shed and load-add logic, radiators and ventilation
Fuel Bulk tanks, transfer pumps, filters, polishing systems, day tanks, supply and return piping, valves, level controls
Low-voltage power Switchboards, buses, main and tie breakers, ATS or closed-transition transfer schemes, static transfer switches, UPS input and output boards, maintenance bypasses, PDUs, remote power panels, busway
UPS and storage UPS modules and systems, battery strings or cabinets, battery breakers, battery monitoring, static bypass
Mechanical Chillers, cooling towers, dry coolers, pumps, heat exchangers, CRAH/CRAC units, coolant distribution units (CDUs), headers, piping sections, valves, makeup water, water treatment
Controls BMS, EPMS, generator and switchgear PLCs, network switches, control power supplies, sensors that drive automatic actions
Ancillary Fire protection interfaces that trip equipment, emergency power off (EPO) circuits, ventilation for battery and generator rooms
Electrical design patterns that deliver Tier III
  • Two distribution paths to the critical load. One active and one alternate is the minimum. Most modern designs use two active paths, A and B, feeding dual-corded IT equipment.
  • Single-corded loads. Each needs a point-of-use or rack-mounted automatic transfer switch, or a static transfer switch upstream, so it can be fed from either path. Without one, every single-corded load is a concurrent maintainability failure.
  • Maintainable switchgear. Main-tie-main or double-ended substations; isolation devices on both sides of every component; maintenance bypasses for each UPS system; and the ability to move load between sources by closed transition or by a planned, brief transfer the IT load can ride through.
  • Isolation points. Every component must be isolatable without de-energizing the alternate path. A breaker that cannot be racked out without opening a shared bus is a classic failure.
  • Control power. Separate DC control power for A-side and B-side switchgear, or a maintainable arrangement, and redundant PLCs for generator paralleling and transfer logic.
  • Generator plant. N+1 generators with a paralleling bus that can be maintained in sections, or generators assigned to independent paths.
The Tier III limitation

Concurrent maintainability protects planned work. It does not require the facility to survive an unplanned failure while one path is out of service, nor a failure whose consequences (fire, smoke, water) affect both paths at one location. In a Tier III facility running on a single path during maintenance, any failure on the remaining path can cause an outage. That exposure is managed by limiting maintenance windows, scheduling work carefully and, often, adding features from Tier IV practice where risk justifies them.

A practical concurrent maintainability review

Keentel uses a structured, component-by-component method:

1. List every component on the one-line diagrams, mechanical flow diagrams, fuel diagrams and controls architecture, including isolation devices and control power.

2. For each component, write the maintenance scenario: what is isolated, which devices are opened, where the load goes, which protective settings change, and which control functions are affected.

3. Check capacity in the scenario: the surviving equipment must carry the full design load at design ambient conditions.

4. Check the alternate path: every element of the path that carries the load in that scenario must be available.

5. Check controls and monitoring: automatic functions still needed during maintenance must still work.

6. Record gaps and fixes, then re-run the scenarios affected by each fix.

The output is a scenario matrix that becomes the basis for methods of procedure (MOPs) and for integrated systems test scripts.

How Keentel Engineering helps here: Keentel performs concurrent maintainability reviews at schematic, design development and construction document stages, across electrical, mechanical, fuel, controls and fire protection interfaces. Typical findings include shared DC control panels, single fuel headers, unisolatable chilled-water valves, single paralleling PLCs and single-corded loads without transfer switches, all of which are cheap to fix on paper and expensive after construction.

6. Tier IV: fault tolerant

Tier IV requires that any single unplanned failure of a capacity component or distribution element, or any single event, does not affect the critical load. It builds on Tier III with three explicit additions.

Fault tolerance

The facility must have multiple independent systems so that, after the loss of any one system, the remaining capacity still carries the design load. The facility must also detect the failure and respond automatically, isolating it and keeping the load supported without depending on operator action. In electrical terms that means independent A and B systems (or a distributed redundant set) from the generator plant to the IT input, with no shared bus, controller, control power source or network whose failure takes down both.

Compartmentalization

Redundant systems and their distribution paths must be physically separated so that a single event, such as a fire, arc flash, explosion, smoke release or water leak, cannot affect both. Typical measures:

  • Separate electrical rooms for A and B switchgear, UPS and batteries, with rated fire separations.
  • Separate cable and pipe routes, including where they enter the data hall; a shared corridor or tray is a classic compartmentalization failure.
  • Separate generator enclosures or yards, fuel systems and day tanks.
  • Separate mechanical plant sections and piping routes.
Continuous cooling

Cooling must continue without interruption through a utility failure and the generator start and transfer sequence. Air-cooled IT equipment heats the room quickly when fans and chillers stop, and liquid-cooled processors have even less thermal margin. Continuous cooling is usually delivered by UPS-backed CRAH or fan wall units and pumps, chilled-water thermal storage, UPS-backed CDUs for liquid cooling, or a combination.

Worked Example 2: why continuous cooling matters

Given. A 10,000 sq ft data hall with a 15 ft clear height and 2 MW of IT load. The air in the room is about 150,000 cu ft. Air's volumetric heat capacity is roughly 0.018 BTU per cubic foot per °F.

Air heat capacity. 150,000 × 0.018 ≈ 2,700 BTU/°F.

Heat release. 2,000 kW × 3,412 BTU/kWh ÷ 60 ≈ 113,700 BTU per minute.

Air-only temperature rise rate. 113,700 ÷ 2,700 ≈ 42 °F per minute.

The building structure, raised floor, racks and IT hardware absorb heat too, so measured rises are much slower than this air-only figure. But the order of magnitude explains why cooling interruptions of one to three minutes during a chiller restart can push inlet temperatures beyond the ASHRAE allowable range in high-density rooms. Chiller restart after a power interruption can take several minutes unless the chillers have fast-restart capability.

Thermal storage sizing. To carry 3,061 tons of cooling (Worked Example 7) for 10 minutes from a chilled-water tank with a 12 °F usable temperature difference:

Keentel Engineering data center tiers technical figure or engineering formula 3

V ≈ 3,061 × 12,000 × (10 ÷ 60) ÷ (8.34 × 12) ≈ 61,000 gallons, before allowances for stratification and tank efficiency. The pumps and air handlers that move that water must themselves be on UPS power.

Common Tier IV failures in practice
  • A single EPMS or BMS network ring whose failure causes both generator plants to lose their load-add logic.
  • A and B cables that share a tray or a penetration for a few feet.
  • A shared chilled-water header or a single makeup-water connection.
  • Fire suppression release on one side that trips equipment on both sides through a common EPO circuit.
  • A and B UPS systems synchronized through a shared load bus synchronization controller with no independent fallback.

How Keentel Engineering helps here: Keentel performs fault tolerance reviews that go beyond the one-line diagram: failure-mode and consequence analysis for each component, compartmentalization walk-downs on 3D models and in the field, control system dependency mapping, and automatic response checks. We confirm that every single failure is detected, isolated and ridden through without operator action.

7. The power source: engine generators as the Tier's foundation

The Tier Standard treats the on-site engine generator plant as the facility's reliable source of power. Utility power is valuable and is usually the cheapest source, but it is outside the owner's control, so it does not count towards the Tier objective. A Tier III facility with two utility feeds and an undersized generator plant is not Tier III; a facility with one utility feed and a properly redundant, maintainable generator plant can be.

Generator ratings

Generator sets are rated by the manufacturer under ISO 8528-1 duty classes. The commonly used US terms map approximately as follows:

Rating Typical meaning Tier implication
Emergency standby power (ESP) Variable load during a utility outage, with annual run-hour limits (often around 200 hours) and a limited average load factor Not acceptable on its own for Tier III or IV where it carries runtime limits
Limited-time running power (LTP) Constant load for a limited number of hours per year Generally not suitable as the Tier basis
Prime power (PRP) Variable load for unlimited hours, with an average load factor limit (commonly 70%) and some overload capability Acceptable when applied within its limits, often by derating
Continuous operating power (COP) Constant load for unlimited hours Acceptable
Data center continuous power (DCC) A manufacturer rating for unlimited hours at varying data center load, defined by each manufacturer Accepted where the manufacturer's rating carries no runtime limit

The principle behind the Tier Standard's position is straightforward: in a long utility outage the generators are the facility's only source, so their rating must allow unlimited running at the design load. Engineers should confirm the rating basis in writing with the manufacturer and state it in the specification.

NEC and NFPA requirements that apply regardless of Tier
  • NEC Article 700 (emergency systems), 701 (legally required standby) and 702 (optional standby) classify generator-supplied loads. Life-safety loads (egress lighting, fire alarm, some smoke control) are typically Article 700; the IT load is normally Article 702 optional standby. Each class has its own transfer time, separation and selective coordination rules.
  • NEC Article 708 (critical operations power systems, COPS) applies only where a government authority designates a facility as critical to public safety or national security. It imposes risk assessment, physical security, commissioning and selective coordination requirements. It is not automatic for commercial data centers.
  • NFPA 110 governs emergency and standby power system installation, performance, maintenance and testing. It classifies systems by Type (maximum time to restore power, for example Type 10 = 10 seconds), Class (minimum running time without refueling, for example Class 48 = 48 hours) and Level (Level 1 where failure could cause loss of life or serious injury; Level 2 otherwise).
  • NFPA 111 covers stored electrical energy emergency and standby power systems, relevant where batteries or other stored energy act as the emergency source.
Worked Example 3: generator plant sizing for the 10 MW hall

Given. 10 MW critical IT load. UPS efficiency 96%. Battery recharge allowance 10% of IT load. Mechanical load on the design day 3,000 kW. Other house loads 300 kW. Generators rated 2,500 kW (3,125 kVA at 0.8 power factor) on a data center continuous basis, derated 3% for altitude and 4% for intake air temperature at the site per the manufacturer's curves.

Generator load.

Load kW
UPS input at full IT load: 10,000 ÷ 0.96 10,417
Battery recharge allowance 1,000
Mechanical plant, design day 3,000
House and ancillary loads 300
Total 14,717

Effective unit rating. 2,500 × 0.97 × 0.96 ≈ 2,328 kW.

Unit count. 14,717 ÷ 2,328 = 6.3, so N = 7 and N+1 = 8 units. With all eight running, each carries 14,717 ÷ (8 × 2,328) ≈ 79% of its derated rating.

Checks the count alone does not cover.

  • Step loading. The UPS rectifiers walk in their load gradually, but mechanical loads restart in blocks. Each generator typically accepts a first load step of a fraction of its rating (often on the order of 25% to 50%, engine dependent) with acceptable frequency and voltage dips. A 25% step on a 2,328 kW unit is about 580 kW. Load-add sequencing must be designed accordingly.
  • UPS interaction. Rectifier input filters, walk-in rates and generator voltage regulator response must be coordinated so the UPS accepts the generator source without repeated transfers to battery.
  • Paralleling bus and controls. For Tier III, the paralleling switchgear must be maintainable in sections; for Tier IV, a single bus or master controller failure must not disable the plant.
  • Where N is counted. If generators are assigned to A and B paths in a 2N arrangement, each side needs N units (plus spares as required), which roughly doubles the plant. Many Tier IV designs use N+1 or N+2 generator plants with fault-tolerant paralleling instead.

How Keentel Engineering helps here: Keentel sizes generator plants from first principles (load profiles, step loading, motor starting, UPS walk-in, altitude and temperature derating) and designs the paralleling switchgear, load-add and load-shed logic, and protection so that the plant meets the Tier objective in every operating mode. We review the manufacturer's rating basis and transient performance data, and we model generator and UPS interaction where rapid load swings are expected.

8. Availability mathematics: what redundancy actually buys

The availability percentages quoted for each Tier are legacy figures from early observational studies. They are not part of the current Tier Standard. For reference, the arithmetic behind the usual quotes is:

Keentel Engineering data center tiers technical figure or engineering formula 4
Legacy figure Availability Annual downtime
Tier I 99.671% (1 − 0.99671) × 8,760 ≈ 28.8 h
Tier II 99.741% ≈ 22.7 h
Tier III 99.982% ≈ 1.6 h (about 95 min)
Tier IV 99.995% ≈ 0.44 h (about 26 min)

Some sources quote 22 hours for Tier II; the arithmetic gives 22.7 hours. Either way, these figures are not design targets. A project-specific availability analysis is much more useful, because it compares real design options with real component data.

The core equations

Availability of a repairable component from its mean time between failures (MTBF) and mean time to repair (MTTR):

Keentel Engineering data center tiers technical figure or engineering formula 5

Components in series (every one must work):

Keentel Engineering data center tiers technical figure or engineering formula 6

Independent redundant paths in parallel (any one is enough):

Keentel Engineering data center tiers technical figure or engineering formula 7

k-out-of-n identical modules (at least k of n must work), as in an N+1 UPS system:

Keentel Engineering data center tiers technical figure or engineering formula 8

A common-cause (beta-factor) model, where a fraction β of each path's unavailability U is shared by both paths:

Keentel Engineering data center tiers technical figure or engineering formula 9
Worked Example 4: single path, N+1 and two paths

Given. Illustrative MTBF and MTTR values for a five-element distribution path. Real studies use IEEE 3006.8 and 3006.9 data, manufacturer data and the owner's own records.

Component MTBF (h) MTTR (h) Availability
Utility service and MV switchgear 200,000 8 0.9999600
MV/LV transformer 300,000 72 0.9997601
LV switchboard 250,000 8 0.9999680
UPS system 100,000 6 0.9999400
PDU 500,000 4 0.9999920

Single path (series). The product is 0.9996201, an expected unavailability of about 3.3 hours (about 200 minutes) a year. The transformer dominates because of its long repair time, which is why on-site spares and maintenance bypasses matter as much as redundancy.

N+1 at module level. For UPS modules with A = 0.99952 (MTBF 50,000 h, MTTR 24 h), eight modules needed:

Configuration Availability of module group Expected unavailability
8 of 8 (N) 0.99617 about 2,014 min/yr
8 of 9 (N+1) 0.9999917 about 4.3 min/yr
8 of 10 (N+2) 0.99999999 about 0.007 min/yr

N+1 makes the module group very reliable. It does nothing for the bus, static switch and output board those modules share.

Two independent paths. Ideal parallel combination of two such paths: 1 − (0.0003799)² ≈ 0.99999986, about 0.08 minutes a year.

With 5% common cause. U_sys ≈ (0.95 × 0.0003799)² + 0.05 × 0.0003799 ≈ 1.91 × 10⁻⁵, about 10 minutes a year, more than a hundred times the ideal figure.

Keentel Engineering data center tiers technical figure or engineering formula 10
Figure 2. Expected unavailability for the configurations in Worked Example 4 (illustrative data). Redundancy is powerful, but shared controls, shared spaces and human error claw back most of the theoretical gain unless they are engineered out.

Lessons.

1. Redundancy at the component level does not fix path weaknesses.

2. Independent paths are extremely effective in theory; common-cause failures decide the real result. This is the quantitative reason for compartmentalization, independent controls and disciplined operations in Tier IV.

3. Repair time matters. Spares strategy, maintenance bypasses and vendor response time are availability levers as powerful as extra equipment.

Standards. IEEE 3006.5 (probability methods), 3006.7 (reliability of 7×24 continuous power systems), 3006.8 (analyzing the reliability of power distribution systems) and 3006.9 (collecting data for reliability analysis) replace the legacy IEEE 493 (Gold Book), which remains a useful source of historical component data. Fault-tree analysis, reliability block diagrams and Monte Carlo simulation are the usual tools for full facilities.

How Keentel Engineering helps here: Keentel builds reliability block diagram and fault-tree models of competing architectures using IEEE 3006 methods, owner data and manufacturer data, including common-cause and repair-time sensitivity. The output is a defensible comparison of expected outage frequency and duration per architecture, which owners can weigh against capital and operating cost.

9. UPS systems and energy storage

The UPS is the bridge between a utility disturbance and the generator plant, and the last line of power conditioning before the IT load. Tier design affects the UPS in three ways: the redundancy architecture (Section 2), the ride-through energy, and the maintainability and isolation of every UPS element.

UPS topologies and Tier fit
UPS arrangement Notes for Tier design
Double-conversion (online) modules in N+1 parallel systems The workhorse of Tier II and Tier III; requires maintenance bypass and isolation for each module and the system
2N systems with static transfer switches (STS) for single-corded loads Common Tier IV pattern; the STS itself must be maintainable and its failure modes analyzed
Distributed redundant (3-to-make-2, 4-to-make-3) Higher utilization; needs careful load assignment so no single system loss overloads the others
Block redundant (catcher) A reserve system catches a failed primary through transfer switches; transfer reliability and controls independence are critical
Rack-level or row-level battery backup with 48 VDC or 800 VDC distribution Emerging in hyperscale and AI designs; Tier analysis must address the shared upstream paths and battery shelf maintainability

UPS equipment in the US is listed to UL 1778. Lithium-ion battery systems for UPS service are typically evaluated with UL 1973(batteries) and, for energy storage systems, UL 9540 with UL 9540A fire-propagation test data. NFPA 855 governs the installation of stationary energy storage systems above threshold quantities, including separation, ventilation, explosion control and fire protection, and the NEC (Articles 480 and 706) covers battery and energy storage system wiring.

Worked Example 5: battery ride-through energy for 5 minutes at 10 MW

Given. 10 MW IT load per UPS side (a 2N design means each side must carry the full load alone). Required ride-through 5 minutes at end of life. Inverter (DC to AC) efficiency 96%. Aging factor 1.25 (battery sized so 80% end-of-life capacity still meets the load). Temperature factor 1.11 for operation below the rated temperature. Design margin 1.10.

DC energy at beginning of life.

Keentel Engineering data center tiers technical figure or engineering formula 11

E = 10,000 × (5 ÷ 60) ÷ 0.96 ≈ 868 kWh; × 1.25 ≈ 1,085 kWh; × 1.11 ≈ 1,204 kWh; × 1.10 ≈ 1,325 kWh per side.

Observations.

  • A 2N design needs about 1,325 kWh on each side, about 2,650 kWh in total, for a 5-minute runtime. That is a significant fire-code quantity; NFPA 855 thresholds and separation rules will apply.
  • Battery current is large: a 1,250 kW module at an end-of-discharge DC voltage around 400 V draws roughly 1,250 ÷ 0.96 ÷ 400 ≈ 3,260 A, which drives DC breaker, busway and cable sizing and DC arc-flash assessment.
  • Five minutes is a common design choice, not a Tier requirement. Shorter runtimes (1 to 3 minutes) are increasingly used with reliable generator plants; the decision should be based on generator start, synchronize and transfer times plus a margin for a failed start and retry.

How Keentel Engineering helps here: Keentel sizes UPS and battery systems for real duty cycles, including aging, temperature and generator retry scenarios, and coordinates battery room design with NFPA 855, UL 9540A data, NEC Articles 480 and 706, ventilation, hydrogen or off-gas detection where applicable, and DC short-circuit and arc-flash studies.

10. MV/LV switchgear and protection

Switchgear is where most concurrent maintainability and fault tolerance gaps hide, because buses, breakers, control power and protection are shared by design. US data centers typically take utility power at 13.8 kV, 34.5 kV or, for large campuses, at 69 kV to 345 kV through an on-site substation, then distribute at 13.8 kV or 34.5 kV to unit substations that step down to 480Y/277 V. IT distribution is at 480 V to UPS systems, then 480 V or 415Y/240 V to the racks; 415Y/240 V allows 240 V single-phase to IT power supplies with fewer transformation stages.

Switchgear arrangements by Tier objective
Arrangement Tier fit Notes
Single-ended unit substation, single bus Tier I and II Any bus or main breaker work requires shutdown
Double-ended (main-tie-main) substation Tier III building block Each half can be isolated while the other carries load, if each transformer is sized for both halves or load is transferred to the alternate path
Separate A and B substations feeding A and B paths Tier III and IV Needs independent control power and protection on each side
Ring or loop MV distribution with sectionalizing Tier III at MV level Each section maintainable; protection must remain selective in all configurations
Isolated A and B MV systems in separate rooms Tier IV Compartmentalized; ties, if any, need careful fault-tolerance analysis

Equipment is typically metal-clad MV switchgear to IEEE C37.20.2, metal-enclosed LV power circuit breaker switchgear to IEEE C37.20.1 and UL 1558, or switchboards to UL 891. MV metal-enclosed interrupter switchgear is covered by IEEE C37.20.3.

Worked Example 6: unit substation sizing and fault duty

Given. A 13.8 kV to 480Y/277 V unit substation serving 2,000 kW of mixed UPS input and mechanical load at 0.95 power factor. Transformer impedance 5.75%. Utility short-circuit capacity at 13.8 kV 500 MVA. Motor contribution from about 500 kVA of running motors.

Load. 2,000 ÷ 0.95 ≈ 2,105 kVA. In a double-ended substation each transformer must carry both halves when the other is out, so select 2,500 kVA (with fans or a higher rating if the combined load is larger).

Full-load current at 480 V.

Keentel Engineering data center tiers technical figure or engineering formula 12

I_FL = 2,500,000 ÷ (1.732 × 480) ≈ 3,007 A.

Bolted three-phase fault at the 480 V bus. With an infinite source, I_SC ≈ 3,007 ÷ 0.0575 ≈ 52.3 kA. Including the 500 MVA source (Z_source ≈ 2,500 ÷ 500,000 = 0.005 pu on the transformer base), I_SC ≈ 3,007 ÷ (0.0575 + 0.005) ≈ 48.1 kA. Adding motor contribution of about four times motor full-load current (4 × 601 A ≈ 2.4 kA) gives roughly 50.5 kA.

What this means. The switchgear needs a short-circuit rating of at least 65 kA at 480 V (the next common rating above 50.5 kA), and the arc-flash incident energy at the bus can be high. Note that a main-tie-main substation operated with the tie closed doubles the available fault current, which is one reason most designs operate with the tie open and use interlocks or closed-transition logic with momentary paralleling only.

K-factor. For a UPS input spectrum with 20% fifth, 12% seventh, 6% eleventh and 4% thirteenth harmonic current (per unit of fundamental), the K-factor is:

Keentel Engineering data center tiers technical figure or engineering formula 13

K ≈ 3.2, so a K-4 rated transformer, or a standard transformer derated per IEEE C57.110, is appropriate. Modern active front-end UPS rectifiers have much lower harmonic content, which should be confirmed with manufacturer data and checked against IEEE 519 at the point of common coupling.

Protection, selectivity and arc flash
  • Selective coordination. NEC 700.32, 701.32 and 708.54 require selective coordination for emergency, legally required standby and COPS systems. Optional standby IT systems do not have a code mandate, but Tier objectives make selectivity essential in practice: a breaker that trips upstream of a fault takes out more of the critical load than necessary.
  • Ground-fault protection. NEC 230.95 requires ground-fault protection of equipment on solidly grounded wye services of more than 150 V to ground and 1,000 A or more; NEC 215.10 and 240.13 cover feeders and equipment. Multi-source 480 V systems with neutral connections need careful ground-fault sensing design to avoid nuisance tripping and blind spots, particularly when generators and transformers are paralleled.
  • Arc energy reduction. NEC 240.87 requires an arc energy reduction method (such as zone-selective interlocking, differential relaying, an energy-reducing maintenance switch or active arc-flash mitigation) for circuit breakers rated or adjustable to 1,200 A or more.
  • Arc-flash studies. IEEE 1584-2018 calculation methods and NFPA 70E labeling and work practices apply. Concurrent maintainability helps safety, because work can be done on de-energized equipment while the load runs on the alternate path, but only if isolation points are designed to allow it.
  • Protection in all operating modes. Settings must be coordinated for normal, maintenance, generator-only and transfer configurations, including closed-transition paralleling.

How Keentel Engineering helps here: Keentel designs MV and LV distribution for data centers from 4 kV to 345 kV utility interconnections and performs short-circuit, protective device coordination and IEEE 1584 arc-flash studies for every operating mode. We specify switchgear arrangements, interlocks, maintenance-mode settings and arc energy reduction methods so concurrent maintainability and worker safety reinforce each other.

11. HVAC and precision cooling

Mechanical systems follow the same Tier logic as electrical systems: capacity components (chillers, cooling towers, dry coolers, pumps, CRAH or CRAC units, fan walls, CDUs) and distribution paths (piping, valves, headers, ductwork). Many facilities that pass an electrical review fail on the mechanical side, because a single valve or header section cannot be isolated.

Thermal envelopes

ASHRAE TC 9.9 thermal guidelines define a recommended inlet air range of 18 °C to 27 °C (64.4 °F to 80.6 °F) for most IT equipment classes, with wider allowable ranges by class (A1 to A4) and a newer H1 class for high-density equipment with a narrower range. Designing to the upper part of the recommended range reduces chiller energy and increases economizer hours, but leaves less margin in a cooling interruption, which links directly to continuous cooling in Tier IV. ASHRAE Standard 90.4 sets energy-efficiency requirements specifically for data centers, using mechanical and electrical loss components rather than a single PUE target.

Worked Example 7: cooling load and chiller redundancy

Given. 10 MW IT load, UPS losses at 96% efficiency (about 417 kW), electrical distribution losses about 2% of IT (200 kW), lighting and other heat 150 kW.

Heat load. 10,000 + 417 + 200 + 150 ≈ 10,767 kW.

Keentel Engineering data center tiers technical figure or engineering formula 14

Q ≈ 10,767 ÷ 3.517 ≈ 3,061 tons of refrigeration.

Chiller selection. With 1,000-ton chillers, N = 4 (4,000 tons, about 77% loaded) and N+1 = 5 for Tier II and III. For Tier IV, the arrangement must also survive the loss of a complete piping system or plant section, which typically means two independent chilled-water systems (A and B loops) each able to carry the load, or a looped, valved header with compartmentalized plant sections, plus continuous cooling.

Points beyond the count.

  • Size for design-day ambient conditions at the site, not nominal ratings; air-cooled chillers lose capacity sharply in hot weather.
  • Pumps, cooling towers, heat exchangers and makeup water need the same redundancy and maintainability as the chillers.
  • Valves on headers must allow any section to be isolated while the rest of the system carries the load. Double block valves may be needed for safe maintenance.
  • Controls must fail to a safe state: a failed controller should leave equipment running, not stopped.
Liquid cooling

High-density AI racks of 50 kW to over 100 kW typically use direct-to-chip liquid cooling with CDUs separating the facility water loop from the technology cooling loop. The CDU, its pumps, its controls and its power supply become critical Tier components. Concurrent maintainability needs redundant CDU pumps or redundant CDUs with isolation valves; fault tolerance needs CDUs fed from independent power and water paths with automatic switchover; continuous cooling needs CDU pumps on UPS power, because processor temperatures rise within seconds of losing coolant flow.

How Keentel Engineering helps here: Keentel's MEP team designs cooling plants and liquid-cooling distribution to the Tier objective, with the electrical feeds, controls power and UPS backing for pumps, fans and CDUs engineered together. Our concurrent maintainability reviews include every valve, header section, CDU and controller, because the mechanical side is where many design certifications fail.

12. Energy, efficiency and PUE

Higher Tiers carry more equipment at lower utilization, which affects efficiency. Power usage effectiveness (PUE) is the industry's common efficiency metric:

Keentel Engineering data center tiers technical figure or engineering formula 15
Worked Example 8: PUE for the 10 MW hall

Given. IT 10,000 kW; UPS losses 417 kW; distribution losses 200 kW; cooling at an average of 0.8 kW per ton on about 2,500 tons average load (2,000 kW); lighting and other 150 kW.

Facility load. 10,000 + 417 + 200 + 2,000 + 150 ≈ 12,767 kW.

PUE. 12,767 ÷ 10,000 ≈ 1.28. Annual facility energy ≈ 12.767 MW × 8,760 h ≈ 111,800 MWh.

Sensitivity. If a 2N design runs UPS modules at 25% to 35% load where efficiency is one or two points lower than at 70%, UPS losses may rise by 100 to 200 kW, changing PUE by about 0.01 to 0.02 and costing hundreds of thousands of dollars a year at typical industrial energy rates. High-efficiency operating modes, right-sized modules and distributed redundant architectures reduce the penalty. The correct comparison uses the manufacturer's efficiency curves at the loading each architecture actually experiences.

How Keentel Engineering helps here: Keentel calculates PUE and annual energy for each architecture under expected load growth, using real UPS, transformer and cooling efficiency curves, and supports ASHRAE 90.4 compliance documentation. Owners see the true life-cycle cost of each Tier option, not just the capital cost.

13. Fuel systems

Fuel is part of the Tier topology. A redundant generator plant fed through a single pump, filter or header is no better than that single element.

Storage. The Tier Standard sets a minimum on-site fuel storage at design load (commonly cited as 12 hours) as the baseline for all Tiers. NFPA 110 Class defines the minimum running time without refueling for emergency and standby systems, and local codes, insurers and owner policies often require more. Many US enterprise and colocation facilities store 24 to 72 hours; mission-critical sites in hurricane- or wildfire-prone regions often store 96 hours or more and hold priority fuel delivery contracts.

Worked Example 9: fuel storage for the 10 MW hall

Given. Generator load 14,717 kW (Worked Example 3). Diesel fuel consumption at high load about 0.068 gallons per kWh (about 14.7 kWh per gallon); confirm with the manufacturer's fuel curves.

Keentel Engineering data center tiers technical figure or engineering formula 16
Runtime Fuel required
12 h ≈ 12,000 gal
24 h ≈ 24,000 gal
48 h ≈ 48,000 gal
72 h ≈ 72,000 gal
96 h ≈ 96,000 gal

Add unusable tank volume, allowances for fuel quality and testing, and the fuel burned during monthly and annual tests. Diesel stored for long periods also needs a fuel polishing and quality program to remain usable.

Maintainability and fault tolerance. For Tier III, bulk tanks, transfer pumps, filters, day tanks and piping must be maintainable without stopping fuel to running generators, which usually means multiple tanks, duplex pumps and filters, looped or dual supply headers and isolation valves. For Tier IV, a single failure (a pump, a leak, a level control) must not interrupt fuel to the generators needed to carry the load, and fuel routes should be compartmentalized. NFPA 30 and NFPA 37 govern flammable liquid storage and engine installation; EPA spill prevention (SPCC) rules apply above threshold quantities.

How Keentel Engineering helps here: Keentel designs and reviews fuel systems as part of the Tier topology: storage sizing, duplex pumping and filtration, day tank controls, looped headers, isolation, leak detection and the electrical feeds and controls for every fuel component.

14. Fire detection and suppression

Fire protection interacts with Tier design in two ways: it protects the facility, and its own actions (alarm-initiated shutdowns, EPO, suppression release) can interrupt the critical load if they are not designed with the Tier in mind.

Element Common US practice Tier considerations
Information technology equipment rooms NFPA 75 (protection of IT equipment) and NFPA 76 (telecommunications facilities) Room construction, detection and suppression requirements; where NEC Article 645 is used, its disconnecting means must be designed so it does not become a common-cause trip of both A and B systems
Detection Very early smoke detection apparatus (aspirating smoke detection) plus spot detectors, to NFPA 72 Early warning allows investigation before suppression release
Sprinklers Double-interlock preaction sprinklers to NFPA 13 are common in data halls Reduces accidental water discharge; piping routes must respect compartmentalization
Clean agent NFPA 2001 systems for selected rooms (switchgear, battery, critical IT) Room integrity, hold time and equipment shutdown interlocks must be engineered
Battery and ESS rooms NFPA 855, UL 9540A data, explosion control, gas detection Separation of A and B battery rooms supports compartmentalization
Emergency power off NEC 645.10 when the IT room uses NEC Article 645 A single EPO that de-energizes both A and B systems defeats fault tolerance unless deliberately accepted

NEC Article 645. Article 645 is optional. Where the IT room meets its conditions (separation, dedicated HVAC, a disconnecting means for power and HVAC), it allows relaxed wiring methods under raised floors. In return it requires the disconnecting means. Many large facilities choose not to use Article 645, wiring the data hall to the general rules of the NEC instead, which removes the EPO requirement and a significant common-cause failure risk.

How Keentel Engineering helps here: Keentel coordinates fire alarm, suppression and EPO interfaces with the electrical and mechanical design so that a fire event on one side does not trip the other, and so the code decisions (whether to use NEC Article 645, how to apply NFPA 75 and NFPA 855) are made deliberately and documented.

15. Controls, BMS and EPMS: the hidden single points of failure

Modern data centers rely on programmable controls for generator paralleling, load management, transfer schemes, mechanical sequencing and alarm response. A topology that is maintainable and fault tolerant on the one-line diagram can be defeated by a single controller, network, power supply or sensor.

Typical control gaps found in reviews:

  • One PLC pair that performs generator paralleling, load add and load shed for both A and B plants.
  • One DC control power panel (or one battery charger) supplying protection relays for both sides of a main-tie-main substation.
  • A BMS network ring that carries chiller sequencing for both A and B loops, with a single core switch.
  • A shared synchronization controller for A and B UPS systems with no independent fallback.
  • Critical temperature or flow sensors without redundancy, whose failure drives equipment to a stopped state.
  • Control air or instrument air for valves and dampers from one compressor.

Design principles.

1. Fail safe for the critical load. On loss of control, equipment should keep running in its last safe state, not stop.

2. Independence matches the topology. A-side and B-side controls, networks and control power should be as independent as the power and cooling paths they control.

3. Maintainability. Controllers, network switches and power supplies must be replaceable without stopping the systems they control (redundant processors, hot standby, manual override).

4. Automatic response for Tier IV. Failures must be detected and isolated automatically, but automation must not create common-cause failure modes.

5. Cybersecurity. Network segmentation and access control for operational technology are now part of resilience; a cyber event is a single event that can reach every connected system.

The electrical power monitoring system (EPMS) and BMS are essential for operations, alarms and capacity management, but they should never be in the direct control path for protective functions unless designed and tested to that duty.

How Keentel Engineering helps here: Keentel maps control dependencies across electrical, mechanical and fuel systems: which controller, network, sensor and power supply each automatic function relies on. We identify common-cause paths, specify independent architectures and write the controls sequences that integrated systems testing will prove.

16. US codes and standards map

Tier objectives are voluntary performance targets. US codes are mandatory. A data center must meet both, and conflicts are resolved in favor of code.

Topic Key US codes and standards
Electrical installation NFPA 70 (NEC), especially Articles 110, 215, 225, 230, 240, 250, 645, 700, 701, 702, 706, 708 and 480
Electrical safety in the workplace NFPA 70E; IEEE 1584-2018 arc-flash calculations; OSHA 29 CFR 1910 Subpart S
Utility-side installations and overhead/underground lines NESC (ANSI C2) for utility-owned substations and lines serving the campus
Emergency and standby power NFPA 110 (Type, Class, Level), NFPA 111 (stored energy), NFPA 37 (engines), NFPA 30 (flammable liquids)
IT and telecom spaces NFPA 75, NFPA 76
Fire alarm and suppression NFPA 72, NFPA 13, NFPA 2001
Energy storage NFPA 855, UL 9540, UL 9540A, UL 1973
Equipment listing UL 1778 (UPS), UL 891 (switchboards), UL 1558 (LV switchgear), UL 1008 (transfer switches), UL 2200 (generator sets), UL 67 (panelboards), UL 857 (busway)
Switchgear construction IEEE C37.20.1, C37.20.2, C37.20.3
Transformers IEEE C57.12.00, C57.12.01 (dry type), C57.110 (harmonic derating)
Power quality IEEE 519-2022 (harmonics at the point of common coupling), IEEE 1100 (Emerald Book: powering and grounding electronic equipment), IEEE 1159 (monitoring)
Reliability IEEE 3006.5, 3006.7, 3006.8, 3006.9 (and legacy IEEE 493)
Cooling and energy ASHRAE TC 9.9 thermal guidelines; ASHRAE 90.4; ASHRAE 90.1 for non-data spaces
Data center frameworks Uptime Institute Tier Standards; ANSI/TIA-942-C; ANSI/BICSI 002

How Keentel Engineering helps here: Keentel's engineers work daily with the NEC, NFPA, IEEE, UL and NESC requirements that apply to data centers, and we document how each code requirement and each Tier objective is met in the basis of design, so authorities having jurisdiction, certification reviewers and insurers can follow the logic.

17. Utility interconnection and the large-load era

A Tier objective does not count utility power, but the utility connection still dominates schedule, cost and day-to-day reliability. Campuses of 100 MW to over 1 GW now connect at 138 kV, 230 kV, 345 kV or 500 kV, and their behavior is drawing regulatory attention.

Utility service options.

Arrangement Benefit Caution
Single radial feed Lowest cost Every utility outage is a generator event
Two feeds from one substation Protects against line outages Common substation and common-mode events
Two feeds from different substations Higher independence Often not available; may need transmission upgrades
On-site transmission substation (breaker-and-a-half or ring bus at 138 kV to 345 kV) Maintainable, high-capacity service Customer-owned or utility-owned design, NESC and utility standards, long lead times

For the 10 MW hall in the worked examples, the facility load of about 12.8 MW at 0.95 power factor is about 13.4 MVA, roughly 56 A at 138 kV, so the transmission connection is lightly loaded by one hall; campuses are planned for many halls.

Large-load ride-through and grid reliability. NERC has documented events in which large numbers of data center loads transferred to backup power simultaneously during transmission faults, removing hundreds of megawatts of load in seconds and creating frequency and voltage control challenges. NERC has formed a large-loads task force and issued alerts and guidance, and FERC has opened proceedings on the interconnection of large loads and co-location with generation. Utilities increasingly require voltage ride-through settings for UPS and transfer equipment, load ramp-rate limits, data for dynamic models and coordination studies before energization. These requirements are evolving quickly; designs should be checked against the current NERC and transmission provider requirements at the time of interconnection.

Interaction with Tier design. UPS transfer thresholds, generator start logic and ATS settings are chosen for the critical load's protection. They also determine how the facility behaves as a grid load during faults. Settings that transfer to battery on every momentary sag protect IT equipment but may conflict with emerging utility ride-through expectations. Keentel recommends engineering both objectives together.

How Keentel Engineering helps here: Keentel performs large-load interconnection studies and designs customer substations from 69 kV to 345 kV and above, including load flow, short-circuit, protection, power quality, dynamic modeling and ride-through coordination between UPS, transfer equipment and utility requirements. We help owners secure capacity on schedule while meeting the transmission provider's and NERC's expectations.

18. Choosing the right Tier: cost of downtime versus cost of resilience

The highest Tier is not automatically the right one. Each step adds capital, space, energy and maintenance cost. The right objective balances that cost against the cost of downtime and the business's tolerance for planned outages.

Worked Example 10: comparing expected downtime cost

Given. A business with a blended cost of downtime of $9,000 per minute (lost revenue, contractual penalties, recovery labor and reputational cost; published industry surveys show a very wide range, and each owner should estimate its own). For illustration only, use the legacy downtime figures as a rough proxy for expected annual downtime at each Tier.

Tier (legacy downtime proxy) Expected downtime Expected annual cost
I (≈ 28.8 h) ≈ 1,730 min ≈ $15.6 million
II (≈ 22.7 h) ≈ 1,360 min ≈ $12.3 million
III (≈ 1.6 h) ≈ 95 min ≈ $0.9 million
IV (≈ 26 min) ≈ 26 min ≈ $0.2 million

How to use it. The difference between Tier II and Tier III expected downtime cost (about $11 million a year in this illustration) dwarfs the typical incremental capital of a second distribution path on a 10 MW hall. The difference between Tier III and Tier IV (about $0.7 million a year here) must be weighed against the cost of compartmentalization, continuous cooling and full fault tolerance, which can add significantly to capital cost. That is why Tier III dominates enterprise and colocation, and why Tier IV is chosen where single events carry extreme consequences.

Caveats. Real outage cost is not linear: a single long outage can cost far more than many short ones. Outages also tend to cluster around maintenance and human error, which no Tier eliminates. Use a project-specific availability model (Worked Example 4) and an owner-specific downtime cost to make the decision.

A decision framework

1. Quantify the consequence of downtime per minute and per event, including contractual service-level penalties and safety or public impacts.

2. Decide whether planned shutdowns are acceptable. If not, concurrent maintainability (Tier III or equivalent) is the minimum.

3. Decide whether any single failure or event must be survivable. If yes, fault tolerance (Tier IV or equivalent) is required, at least for the most critical systems.

4. Consider IT-level resilience. Applications replicated across multiple sites or availability zones can justify a lower site Tier at each location, provided failover is real and tested.

5. Consider a mixed approach. Some owners build Tier III electrical with Tier IV features where risk is concentrated (compartmentalized battery rooms, fault-tolerant controls, continuous cooling for liquid-cooled halls).

6. Plan expansion. Choose an architecture that can grow block by block without breaking concurrent maintainability.

7. Decide on certification early. Design-document certification is cheapest to achieve before construction documents are complete.

How Keentel Engineering helps here: Keentel facilitates Tier selection workshops with owners, combining availability modeling, downtime cost estimates, capital and life-cycle cost comparisons and expansion planning. The result is a documented, defensible objective for each system, which then drives the basis of design.

19. Commissioning: where the Tier is proven

A design achieves its Tier only when the built facility performs as intended. Data center commissioning is usually organized into five levels:

Level Activity Tier relevance
Level 1 Factory witness testing of major equipment (UPS, switchgear, generators, chillers, controls) Confirms equipment ratings, transfer performance and control logic before shipment
Level 2 Site delivery inspection and installation verification Confirms the right equipment arrived undamaged and is installed per design
Level 3 Pre-functional and startup testing by installers and manufacturers Insulation resistance, torque, relay testing, protective device settings, initial energization
Level 4 Functional performance testing of each system Each system tested in all operating modes, including maintenance bypasses and transfers
Level 5 Integrated systems testing (IST) The whole facility under load banks: utility failure, generator failure, component failures, maintenance scenarios and recovery

Integrated systems testing for Tier objectives. IST scripts should be built from the concurrent maintainability scenario matrix (Tier III) and the failure scenario matrix (Tier IV). Each scenario is executed under representative load, with temperatures, voltages, frequencies and alarm responses recorded. Typical scenarios:

  • Utility failure with all generators available; utility failure with one generator failing to start.
  • Each switchgear section isolated for maintenance, with load carried on the alternate path.
  • Each UPS system placed in maintenance bypass, then isolated.
  • A chiller, pump and CDU each isolated; a chilled-water header section isolated.
  • For Tier IV, live failure simulations: a UPS system trip, a switchgear bus fault simulation (by tripping the main), a controller failure, loss of a control network, and confirmation that cooling continues through the generator start sequence.
  • Return-to-normal sequences, which are a frequent source of outages if not tested.

Electrical tests follow the ANSI/NETA Acceptance Testing Specifications (ATS) and the manufacturer's procedures; generator system testing follows NFPA 110.

How Keentel Engineering helps here: Keentel writes and reviews commissioning plans and Level 4 and Level 5 test scripts directly from the design's maintenance and failure scenarios, witnesses factory and site testing, and verifies protective device settings, transfer timing and control sequences. Owners receive evidence, not assumptions, that the facility meets its objective.

20. Operations: keeping the Tier for 20 years

Industry outage surveys consistently find that a large share of significant data center outages involve human error, procedures or management decisions rather than equipment alone. Topology creates the potential for resilience; operations decide whether it is realized.

Operational practices that preserve a Tier objective:

  • Methods of procedure (MOPs) for every maintenance activity, derived from the concurrent maintainability scenarios, reviewed by engineering and rehearsed.
  • Emergency operating procedures (EOPs) for credible failures, with clear authority and communication.
  • Capacity management. Redundancy disappears silently as load grows: a 2N system loaded above 50% is no longer 2N. EPMS-based capacity dashboards should track loading against the redundancy limit for every system.
  • Maintenance programs based on NFPA 70B (now an enforceable standard for electrical equipment maintenance), NFPA 110 for generators, NFPA 25 for water-based fire protection and the manufacturer's recommendations.
  • Protective device studies kept current. Short-circuit, coordination and arc-flash studies should be updated with every significant change and reviewed at least every five years as NFPA 70E requires.
  • Change management. Every modification is reviewed against the Tier objective, including controls and software changes.
  • Training and staffing appropriate to the Tier, as covered by the Operational Sustainability criteria.

How Keentel Engineering helps here: Keentel supports operators with MOP and EOP reviews, NFPA 70B and 70E program support, study updates after changes, capacity and redundancy audits, and root-cause analysis after incidents. We help operations teams keep the resilience the facility was designed for.

21. AI and high-density facilities: new pressures on Tier design

AI training and inference clusters are changing data center electrical and mechanical design faster than any previous technology wave.

  • Density. Racks of 50 kW to more than 100 kW, with roadmaps beyond that, move cooling to liquid and make continuous cooling a near-necessity even where the facility is not pursuing Tier IV.
  • Load dynamics. Synchronized training workloads can swing power by large fractions of the hall load within seconds. UPS systems, generators and the utility interconnection must handle those swings; generator governors and voltage regulators in particular need to be checked against measured load profiles.
  • New power architectures. Higher-voltage DC distribution (such as 800 VDC to the rack) and rack-level energy storage change where redundancy sits and how maintainability is achieved. Tier analysis still applies, but component and path definitions change.
  • Scale. Campuses of hundreds of megawatts favor distributed redundant and block redundant architectures over 2N to reduce installed capacity, and they bring the large-load interconnection issues described in Section 17.
  • Application resilience. Some AI training workloads can checkpoint and restart; some inference services are replicated across regions. Owners may choose a lower site objective for some halls and a higher one for others.

How Keentel Engineering helps here: Keentel designs power systems for high-density and AI facilities, including liquid-cooling power and controls, generator and UPS performance under rapid load swings, DC distribution architectures and large-load utility interconnection. Our companion blogs on 800 VDC architectures, rack engineering and 1 GW campuses go deeper on each topic.

22. What this means for MEP engineers: a practical checklist

At concept and schematic design

  • Agree the objective for each system (electrical, mechanical, fuel, controls, fire protection) and record it in the basis of design.
  • Choose the redundancy architecture and module sizes together; check utilization at day-one and full-build load.
  • Confirm the generator rating basis, fuel storage duration and refueling strategy.
  • Reserve space for second paths, compartmentalized rooms, fuel and thermal storage.

During design development

  • Build the concurrent maintainability scenario matrix and, for Tier IV, the failure scenario matrix.
  • Run short-circuit, coordination, arc-flash, load flow and harmonic studies for every operating mode.
  • Map control dependencies and control power sources.
  • Coordinate fire protection, EPO and shutdown interfaces with the redundancy design.

Before construction documents are issued

  • Close every gap found in the scenario reviews.
  • Pursue design-document certification if certification is intended.
  • Specify factory witness tests and integrated systems test requirements in the equipment specifications.

During construction and commissioning

  • Verify compartmentalization in the field: routes, penetrations and shared spaces.
  • Execute Level 1 to Level 5 commissioning with scripts built from the scenario matrices.
  • Hand over MOPs, EOPs, studies and as-built documentation that operations can use.

How Keentel Engineering helps here: Keentel supports every step of this checklist as engineer of record, peer reviewer or owner's engineer. Our coordinated electrical, mechanical and controls approach is designed around a single principle: the facility must keep the critical load running when something fails, not just look redundant on paper.

Part C — Three Anonymous US Case Studies

The following case studies are illustrative composites based on conditions Keentel Engineering commonly encounters. Client names, locations and identifying details are withheld or generalized, and figures are rounded.

Case Study 1 — Southeast colocation facility: Tier III design gaps found before construction documents

Situation. A developer in the Southeast planned a 16 MW colocation building designed to Tier III objectives, served at 13.8 kV from two utility feeders, with N+1 generators (2.5 MW class), N+1 UPS systems on A and B paths, and an N+1 air-cooled chiller plant. The owner intended to pursue design-document certification. Keentel was engaged as peer reviewer at the 90% design development stage.

Analysis. Keentel built a concurrent maintainability scenario matrix covering 412 components across electrical, mechanical, fuel and controls. Most scenarios passed. Seven did not:

  • A single DC control power panel and battery charger supplied the protective relays for both halves of each double-ended 13.8 kV switchgear lineup.
  • One generator paralleling PLC pair controlled both A-side and B-side generator buses.
  • The fuel system had a single supply header from the bulk tanks to all day tanks.
  • A chilled-water header valve between plant sections could not be isolated without draining both loops.
  • About 6% of tenant loads were planned as single-corded, with no rack transfer switches specified.
  • Closed-transition transfer between A and B unit substations would momentarily parallel two transformers and exceed the 65 kA rating of the 480 V switchgear.
  • The fire alarm system's EPO interface was wired to trip both A and B UPS systems for an event in either electrical room.

Engineered solution. Separate A and B DC control systems with independent chargers; redundant, independent paralleling controllers for each generator bus with manual override; a looped fuel header with sectionalizing valves and duplex transfer pumps; a valved bypass around the chilled-water header section; a tenant standard requiring rack transfer switches for single-corded loads; open-transition transfers with UPS ride-through (eliminating the momentary paralleling) and updated arc-flash studies; and an EPO scheme limited to the affected room.

Outcome. All seven gaps were closed before construction documents were issued, at an estimated cost below 1% of the electrical and mechanical budget. The design-document review proceeded without major comments, and the scenario matrix became the basis for the commissioning team's Level 5 scripts.

Case Study 2 — Midwest enterprise data center: from Tier II to concurrent maintainability while live

Situation. A financial services company in the Midwest operated a 2.4 MW data center with N+1 generators and N+1 UPS modules, but a single 480 V switchboard, a single UPS output board and single-path distribution. Annual switchboard maintenance required weekend shutdowns, and the business had decided they were no longer acceptable. A thermographic survey had also found a hot connection on the UPS output board that could not be repaired without an outage.

Analysis. Keentel's survey and studies found enough capacity in the existing generator plant and utility service to support a second path, about 85% dual-corded IT equipment, and space in an adjacent storage room for a new B-side electrical room. Short-circuit and coordination studies showed the existing switchboard would need a revised protection scheme once a tie and second board were added.

Engineered solution. A phased upgrade added a new B-side UPS system and output switchboard in the adjacent room (separated by a rated wall), new B-side PDUs and overhead busway, rack transfer switches for single-corded equipment and a main-tie-main arrangement on the 480 V service. Each of 14 cutover steps had a method of procedure, temporary power provisions, revised protective settings and arc-flash labels, and a rollback plan. Work on the hot UPS output board connection was scheduled after the B path was live, so it was performed with the A path de-energized and the load running on B.

Outcome. The facility moved to concurrently maintainable operation with no unplanned outage during the project. The first annual switchboard maintenance after the upgrade was performed with the IT load running on the alternate path, and arc-flash exposure for that work dropped because the equipment could be de-energized.

Case Study 3 — South-central US AI campus: fault tolerance at hyperscale without 2N everywhere

Situation. A developer planned a multi-building AI campus in the south-central US, with an initial 120 MW phase served from a new 345 kV substation through two 345/34.5 kV transformers, and 34.5 kV distribution to 12 data halls of about 10 MW IT each. The tenant required fault tolerance for liquid-cooled training halls but wanted to minimize installed UPS and generator capacity and footprint.

Analysis. Keentel compared 2N, 3-to-make-2 distributed redundant and block redundant (catcher) architectures for each hall. With 1,250 kW UPS modules, 2N required 16 modules per hall against 12 for 3-to-make-2. A reliability block diagram model with common-cause factors showed similar expected outage frequency for the distributed redundant and 2N options once controls were made independent, while the catcher design depended heavily on transfer switch reliability. Load measurements from the tenant's existing clusters showed synchronized power swings of several megawatts within a few seconds, which drove generator governor and UPS input-current-limit settings.

Engineered solution. A 3-to-make-2 distributed redundant UPS and generator architecture per hall, with three compartmentalized electrical rooms; independent controls and control power for each system; UPS-backed CDU pumps and fans with thermal storage for continuous cooling; fault-tolerant paralleling for the generator plants; and a utility ride-through scheme agreed with the transmission provider so that UPS transfers to battery during transmission faults would not remove the whole campus load at once. Short-circuit, coordination, arc-flash and harmonic studies covered every operating mode, and EMT studies confirmed UPS and transfer equipment behavior during 345 kV faults.

Outcome. The architecture met the tenant's fault tolerance requirement with about 25% less installed UPS capacity per hall than 2N, freeing space for additional IT capacity, and the interconnection studies were accepted by the transmission provider without additional mitigation.


Part D — Technical FAQ: Data Center Tiers for Engineers

1. What exactly does a Tier rate?

The site infrastructure that supports the IT equipment: power, cooling, fuel, controls and the spaces that house them. It does not rate servers, storage, networks, software or the resilience of applications. A Tier IV building can still suffer an application outage, and a well-replicated application can run happily on Tier II sites.

2. Who can call a facility "Tier III"?

Only a facility certified by the Uptime Institute should be called Tier III Certified, and the certification type (design documents, constructed facility, operational sustainability) should be stated. Others should say "designed to Tier III objectives", or use the terminology of the framework they actually meet, such as ANSI/TIA-942-C Rated-3.

3. Is the difference between design and constructed-facility certification important?

Yes. Design certification shows the drawings meet the objective. Constructed-facility certification shows the built facility performs to it during on-site demonstrations. Many facilities hold only the first. Tenants and lenders relying on a Tier should ask which they are getting.

4. How do TIA-942-C and BICSI 002 compare with Uptime Tiers?

They use similar four-level concepts (Rated-1 to Rated-4 in TIA-942-C, Classes F1 to F4 in BICSI 002, with F0 below), but they are separate documents with different requirements and assessment models. TIA-942-C covers telecommunications cabling and site systems and allows self-assessment or third-party conformity assessment; BICSI 002 is a design best-practice standard. Many projects reference all three.

5. Are the availability percentages (99.671% to 99.995%) part of the Tier definitions?

No. They are legacy figures from early observational studies and are not in the current Tier Standard. Tiers are defined by topology and performance objectives. For design decisions, Keentel calculates project-specific availability with IEEE 3006 methods instead.

6. Does Tier III require two active distribution paths?

No. Tier III requires redundant distribution paths so that any path element can be maintained, but only one path needs to be active; the other can be an alternate path used during maintenance. In practice most modern designs make both active with dual-corded IT equipment, which improves resilience to failures as well as maintenance.

7. Is N+1 enough for Tier III?

N+1 capacity components are necessary but not sufficient. Concurrent maintainability also requires redundant distribution paths and the ability to isolate every path element, control system, fuel component and piping section without affecting the critical load.

8. Does Tier IV require 2N?

No. Tier IV requires fault tolerance: after the loss of any one system or path, the remaining capacity must carry the load, with automatic response, compartmentalization and continuous cooling. 2N or 2(N+1) are common ways to achieve it, but distributed redundant designs (for example 3-to-make-2) and some block redundant designs can meet the objective with less installed capacity.

9. What is the practical difference between 2N, 2N+1 and 2(N+1)?

2N is two independent systems each sized for the full load. 2(N+1) adds a redundant module to each system, so the facility can lose a whole system and still have a spare module in the survivor; it is often written 2N+2. "2N+1" is used loosely in industry, sometimes for 2(N+1) and sometimes for 2N with one shared spare; specifications should define the term precisely.

10. Why can a 2N UPS system never be loaded above 50%?

Because each system must be able to carry the whole load alone if the other fails. If the total load exceeds one system's rating, the facility is no longer 2N, even though all the equipment is still installed. Capacity monitoring must track this limit continuously.

11. What are the loading limits for distributed redundant designs?

For n systems where any n − 1 must carry the load, the maximum normal loading of each system is (n − 1) ÷ n of its rating: 66.7% for 3-to-make-2 and 75% for 4-to-make-3. Practical designs keep a margin below these limits for imbalance and growth.

12. How does a block redundant (catcher) system work?

Several primary UPS systems each carry their own block of load. A reserve system, matched to the largest block, sits behind them and picks up any failed primary's load through static or automatic transfer switches. It reduces idle capacity compared with 2N, but the transfer devices and their controls become critical and must be analyzed carefully for common-cause failure.

13. Why does the Tier Standard not count utility power?

Because the utility is outside the owner's control and can be unavailable for extended periods. The on-site generator plant is treated as the reliable source. Utility feeds still matter for cost, generator runtime and day-to-day reliability, but they cannot substitute for generator capacity and redundancy.

14. Which generator ratings are acceptable for Tier III and IV?

Ratings that allow unlimited running hours at the design load: continuous ratings, prime ratings applied within their load-factor limits, or manufacturer data center continuous ratings with no runtime limitation. Emergency standby ratings with annual run-hour limits do not qualify on their own. Confirm the rating basis with the manufacturer in writing.

15. How much fuel should be stored?

The Tier Standard's commonly cited baseline is 12 hours at design load on site, and NFPA 110 Class sets the minimum running time for code purposes. Most US enterprise and colocation sites store 24 to 72 hours, and sites exposed to hurricanes, wildfires or long supply interruptions often store 96 hours or more with priority delivery contracts. For a 14.7 MW generator load, each 24 hours needs roughly 24,000 gallons of diesel.

16. What NEC articles apply to data center generators?

Article 700 for emergency (life-safety) loads, Article 701 for legally required standby loads, and Article 702 for optional standby loads, which usually includes the IT load. Article 708 applies only when a government authority designates the facility as critical operations power. Each article has its own requirements for transfer time, wiring separation and selective coordination.

17. Should a data center use NEC Article 645?

It is optional. Article 645 relaxes some wiring methods in IT rooms but requires a disconnecting means for IT power and HVAC. Many large facilities wire to the general NEC rules instead, avoiding the disconnecting means and the risk of an EPO action tripping both A and B systems. The decision should be made deliberately with the authority having jurisdiction and documented.

18. Does the NEC require selective coordination in data centers?

It requires selective coordination for emergency (700.32), legally required standby (701.32) and COPS (708.54) systems. Optional standby IT systems have no code mandate, but selective coordination is essential to meet Tier objectives, because nonselective tripping takes out more of the critical load than the fault requires.

19. Why is continuous cooling required for Tier IV but not Tier III?

Tier IV requires that no single event, including a utility failure, affects the critical load, and a cooling interruption during generator start and chiller restart can overheat dense IT equipment within minutes. Tier III focuses on planned maintenance. Many Tier III designs add continuous cooling anyway for high-density or liquid-cooled halls, because the physics does not depend on the Tier.

20. How quickly can a data hall overheat without cooling?

Faster than most people expect. In Worked Example 2, the air in a 2 MW, 10,000 sq ft hall would warm at about 42 °F per minute if it absorbed all the heat; building and equipment mass slow that substantially, but inlet temperatures can still exceed ASHRAE allowable limits within a few minutes in high-density rooms. Liquid-cooled processors can reach thermal limits within seconds of losing coolant flow.

21. What is compartmentalization and why does it matter?

It is the physical separation of redundant systems and their paths, in separately fire-rated rooms and routes, so that one fire, arc flash, leak or explosion cannot affect both. Worked Example 4 shows why: a 5% common-cause share raised the expected unavailability of two otherwise independent paths from about 0.08 minutes to about 10 minutes a year.

22. What are the most common hidden single points of failure?

Shared DC control power for both sides of a switchgear lineup; one paralleling PLC for both generator buses; single fuel headers; unisolatable chilled-water valves or header sections; shared cable trays or penetrations; EPO circuits that trip both sides; shared network or synchronization controllers; and single-corded loads without transfer switches.

23. How should UPS battery runtime be chosen?

From the generator plant's start, synchronize and transfer time, plus a margin for a failed start and retry, plus aging and temperature factors. Five minutes is common, but one to three minutes is increasingly used with reliable generator plants. For a 10 MW load, a 5-minute, end-of-life design needs roughly 1,300 kWh of DC energy per UPS side (Worked Example 5).

24. What fire and safety standards apply to lithium-ion UPS batteries?

NFPA 855 for installation of stationary energy storage systems above threshold quantities, UL 9540 and UL 9540A for system listing and fire-propagation testing, UL 1973 for batteries, UL 1778 for the UPS, and NEC Articles 480 and 706. The authority having jurisdiction may require explosion control, gas detection and specific separation distances.

25. How does Tier design affect arc-flash safety?

Positively, if it is designed in. Concurrent maintainability allows equipment to be de-energized for work while the load runs on the alternate path, reducing energized work. But closed-transition transfers and closed ties can raise fault current and incident energy, and maintenance modes need their own protective settings. Arc-flash studies to IEEE 1584 and NFPA 70E must cover every operating configuration, and NEC 240.87 arc energy reduction applies to large breakers.

26. What harmonic and power quality issues affect data centers?

UPS rectifiers, IT power supplies and variable-speed drives draw harmonic currents. Modern active front-end UPS units are much cleaner than older six-pulse designs, but large facilities must still meet IEEE 519 limits at the point of common coupling and avoid resonance with power factor correction or filter capacitors. Transformers may need K-rating or derating per IEEE C57.110.

27. How do large data center loads affect the transmission grid?

Campuses of hundreds of megawatts can trip offline together when UPS systems transfer to battery during transmission faults, removing large blocks of load in seconds. NERC has documented such events and is developing guidance and requirements for large loads, and FERC has opened proceedings on large-load interconnection. Utilities increasingly require ride-through settings, ramp-rate limits and dynamic model data.

28. What is integrated systems testing and why is it essential?

It is Level 5 commissioning: the complete facility is tested together under load banks through utility failures, generator failures, component failures, maintenance scenarios and return to normal. It is the only way to prove that the topology, controls and settings deliver the objective, and it is central to constructed-facility certification.

29. Can an existing facility be upgraded to a higher Tier?

Often, yes, but it is complex. Upgrades typically add a second path, switchgear ties, transfer switches, mechanical valving and controls redundancy while the facility stays live. Each cutover needs a method of procedure, temporary provisions, revised protection and arc-flash review, and a rollback plan (Case Study 2).

30. What is the single most important lesson about data center Tiers?

That redundancy on paper is not resilience. A facility is only as reliable as its weakest component, including controls, fuel, valves, shared spaces and procedures. The right question is not how much redundancy has been installed, but whether the whole infrastructure keeps the critical load running when something fails, and whether that has been proven in testing.


How Keentel Engineering Helps


Keentel Engineering is a US power and MEP engineering firm serving data center owners, developers, colocation operators, hyperscale tenants, EPC contractors and utilities, with power system expertise from 4 kV distribution to 765 kV transmission. For data centers at every Tier objective, our services include:


  • Tier strategy and basis of design: framework selection (Uptime Tier objectives, TIA-942-C, BICSI 002), redundancy architecture comparison, Tier selection workshops and documented basis-of-design narratives.
  • Data center electrical design: MV and LV distribution, double-ended and A/B substations, generator paralleling switchgear, UPS systems and battery rooms, busway and PDU distribution, grounding and bonding to IEEE 1100 and the NEC.
  • Utility interconnection and substations: large-load interconnection studies and customer substations from 69 kV to 345 kV and above, including transmission line taps, ring bus and breaker-and-a-half arrangements.
  • UPS and generator sizing: load calculations, step-load and motor-starting analysis, UPS walk-in and generator interaction, battery runtime sizing and fuel storage sizing.
  • Power system studies: short-circuit, protective device coordination, IEEE 1584 arc-flash and NFPA 70E labeling, load flow, motor starting, harmonic analysis to IEEE 519, transient and EMT studies for UPS and transfer equipment.
  • Reliability and availability analysis: reliability block diagrams, fault trees and Monte Carlo models using IEEE 3006 methods, with common-cause and repair-time sensitivity.
  • Concurrent maintainability and fault tolerance reviews: component-by-component scenario matrices across electrical, mechanical, fuel, controls and fire protection, at every design stage.
  • MEP engineering: cooling plant and liquid-cooling design, fuel systems, fire protection coordination and controls sequences, delivered with the electrical design as one coordinated package.
  • Commissioning support: Level 1 to Level 5 test plans, factory witness testing, protective device settings verification and integrated systems test scripts built from the design's scenario matrices.
  • Owner's engineer services: independent technical oversight from site selection and utility negotiations through design, procurement, construction, commissioning and early operations.
  • Operations support: MOP and EOP reviews, NFPA 70B and 70E program support, capacity and redundancy audits, study updates and incident root-cause analysis.
  • Large-load grid compliance: ride-through coordination, dynamic model data and support for transmission provider and NERC large-load requirements.


Why Clients Choose Keentel


  • Systems thinking: we design and review power, cooling, fuel, controls and fire protection together, because that is where single points of failure hide.
  • Utility-to-rack depth: from 345 kV interconnections to 415Y/240 V rack distribution and 800 VDC architectures, one team understands the whole chain.
  • Quantified decisions: reliability models, life-cycle cost and downtime cost analysis turn Tier choices into defensible business decisions.
  • US code fluency: daily work with the NEC, NFPA 70E, 70B, 110, 75 and 855, IEEE 1584, IEEE 3006, IEEE 519, UL listing requirements, NESC and NERC.
  • Commissioning-ready design: our scenario matrices become the methods of procedure and integrated systems test scripts, so the design is built to be proven.
  • Vendor-independent advice: we are not tied to any equipment manufacturer, so our recommendations follow the engineering and the client's interests.
  • Nationwide reach: offices in Tampa, Austin, Sacramento and Baltimore supporting data center projects across the United States.


Talk to an Engineer


Planning a new data center, expanding a campus, preparing for Tier certification or upgrading a live facility to concurrent maintainability? Keentel Engineering can help you choose the right objective, design it, prove it and keep it.


  • Phone: (813) 389-7871
  • Email: contact@keentelengineering.com
  • Book a 15-minute call: calendly.com/keentel-engineering/15min
  • Web: keentelengineering.com


References


  • Uptime Institute, Tier Standard: Topology, and Tier Standard: Operational Sustainability (current editions).
  • Uptime Institute, Accredited Tier Designer technical papers on engine-generator ratings and fuel systems.
  • ANSI/TIA-942-C, Telecommunications Infrastructure Standard for Data Centers, 2024.
  • ANSI/BICSI 002, Data Center Design and Implementation Best Practices (current edition).
  • NFPA 70, National Electrical Code (current edition adopted by the authority having jurisdiction).
  • NFPA 70E, Standard for Electrical Safety in the Workplace; NFPA 70B, Standard for Electrical Equipment Maintenance.
  • NFPA 110, Standard for Emergency and Standby Power Systems; NFPA 111, Standard on Stored Electrical Energy Emergency and Standby Power Systems.
  • NFPA 75, NFPA 76, NFPA 72, NFPA 13, NFPA 2001, NFPA 855, NFPA 30 and NFPA 37.
  • IEEE Std 1584-2018, IEEE Guide for Performing Arc-Flash Hazard Calculations.
  • IEEE Std 3006.5, 3006.7, 3006.8 and 3006.9 (reliability of industrial and commercial power systems); IEEE Std 493 (Gold Book, legacy).
  • IEEE Std 1100 (Emerald Book), Recommended Practice for Powering and Grounding Electronic Equipment.
  • IEEE Std 519-2022, IEEE Standard for Harmonic Control in Electric Power Systems.
  • IEEE Std C37.20.1, C37.20.2 and C37.20.3 (switchgear); IEEE Std C57.12.00 and C57.110 (transformers).
  • UL 1778, UL 891, UL 1558, UL 1008, UL 2200, UL 9540, UL 9540A and UL 1973.
  • ANSI/NETA ATS, Standard for Acceptance Testing Specifications for Electrical Power Equipment and Systems.
  • ASHRAE TC 9.9, Thermal Guidelines for Data Processing Environments (current edition); ANSI/ASHRAE Standard 90.4, Energy Standard for Data Centers.
  • ISO 8528-1, Reciprocating internal combustion engine driven alternating current generating sets: Application, ratings and performance.
  • NESC (ANSI C2), National Electrical Safety Code.
  • NERC, Large Loads Task Force publications and incident reviews on large-load behavior during transmission faults; FERC proceedings on large-load interconnection.
  • Keentel Engineering companion technical blogs: data center Tiers overview, data center load calculations, 1 GW data center explained, RMU vs. MV switchgear for data centers, data center rack engineering, 800 VDC AI data center UPS and 800 VDC vs. 1,500 VDC.


Disclaimer


This article is provided by Keentel Engineering for general technical information and education. It is not engineering advice for any specific project and does not create a professional relationship. Worked examples, component reliability data, costs and case studies are illustrative; real projects require site-specific data, manufacturer information and analysis by qualified engineers. Tier requirements are summarized for discussion only: the Uptime Institute's published Tier Standards govern, and only the Uptime Institute can certify a facility's Tier. Legacy availability percentages are shown for context and are not part of the current Tier Standard. Code requirements vary by edition and jurisdiction; always confirm the editions adopted by the authority having jurisdiction. Case studies are illustrative composites, not records of specific client projects.


Uptime Institute, Tier, Tier Certification, TIA, BICSI, IEEE, NFPA, NEC, UL, ASHRAE, NETA, NERC and other names and marks are the property of their respective owners. Keentel Engineering is not affiliated with or endorsed by these organizations or by any equipment manufacturer.


© 2026 Keentel Engineering. All rights reserved. Written by Keentel Engineering.



A smiling man with glasses and a beard wearing a blue blazer stands in front of server racks in a data center.

About the Author:

Sandip "Sonny" R. Patel, P.E.

IEEE Senior Member · Founder & CEO, Keentel Engineering

In 1995, Sonny Patel earned his Electrical Engineering degree from the University of Illinois. But degrees don't build legacies — action does.

For three decades, he has worked the power industry from every side of the table: 16 years as a utility engineer at Exelon/Commonwealth Edison; generation leadership across hydroelectric, industrial steam turbine, and a 9 GW renewable fleet; NERC Regional Entity Senior Compliance Engineer and Audit Team Lead, auditing some of the nation's largest utilities; and testing and commissioning lead on equipment up to 765 kV — the very top of the North American grid.Utility. Generator. Regulator. Consultant. Few engineers have seen all four seats. Fewer still have sat in them.

His experience spans nuclear, hydro, conventional generation, renewables, oil and gas, mining — and today's data centers, where he is authoring a three-book series on data center design. He is a Licensed Professional Engineer in six states and a Licensed Electrical Contractor in Florida (Unlimited EC) — he doesn't just design the work; he's qualified to stand behind its execution.Today, as Founder and CEO of Keentel Engineering, Sonny leads a nationwide team of engineers delivering substation design, power system studies, NERC compliance, and commissioning — done right, coast to coast.Three decades. Every side of the table. One standard: accountable engineering.

Four workers in safety vests and helmets stand with arms crossed near wind turbines.

Let's Discuss Your Project

Let's book a call to discuss your electrical engineering project that we can help you with.

Man in a blazer and open shirt, looking at the camera, against a blurred background.

About the Author:

Sandip "Sonny" R. Patel, P.E.

IEEE Senior Member · Founder & CEO, Keentel Engineering

In 1995, Sonny Patel earned his Electrical Engineering degree from the University of Illinois. But degrees don't build legacies — action does.

For three decades, he has worked the power industry from every side of the table: 16 years as a utility engineer at Exelon/Commonwealth Edison; generation leadership across hydroelectric, industrial steam turbine, and a 9 GW renewable fleet; NERC Regional Entity Senior Compliance Engineer and Audit Team Lead, auditing some of the nation's largest utilities; and testing and commissioning lead on equipment up to 765 kV — the very top of the North American grid.

Utility. Generator. Regulator. Consultant. Few engineers have seen all four seats. Fewer still have sat in them.

His experience spans nuclear, hydro, conventional generation, renewables, oil and gas, mining — and today's data centers, where he is authoring a three-book series on data center design. He is a Licensed Professional Engineer in six states and a Licensed Electrical Contractor in Florida (Unlimited EC) — he doesn't just design the work; he's qualified to stand behind its execution.

Today, as Founder and CEO of Keentel Engineering, Sonny leads a nationwide team of engineers delivering substation design, power system studies, NERC compliance, and commissioning — done right, coast to coast.Three decades. Every side of the table. One standard: accountable engineering.

Leave a Comment

Related Posts

NERC compliance engineering standards covering PRC, MOD, TPL, FAC, VAR, EOP and CIP.
By SANDIP R PATEL • November 14, 2026
Explore NERC compliance engineering for generator and transmission owners, IBRs and large loads, covering PRC, MOD, TPL standards, audits and modeling.
LVRT to HVRT transition in weak grids showing post-fault overvoltage and inverter voltage response.
By SANDIP R PATEL • November 14, 2026
Learn why LVRT becomes HVRT in weak grids, how post-fault overvoltage trips solar, wind and BESS plants, and how Keentel Engineering uses EMT studies.
STATCOM V-I characteristic graph showing voltage regulation, current limits and short-term overload.
By SANDIP R PATEL • November 14, 2026
Explore STATCOM V-I characteristics, voltage control slope, current limits, short-term overload ratings, reactive power and low-voltage grid performance.
Ground distance protection R-X diagram showing mho and quadrilateral relay settings for a 230 kV lin
By SANDIP R PATEL • November 12, 2026
Learn ground distance protection settings for transmission lines: k0 compensation, mho and quadrilateral relays, mutual coupling and NERC compliance.
800 VDC vs. 1,500 VDC AI data center power distribution chart comparing rack current and conductor l
By SANDIP R PATEL • November 12, 2026
Compare 800 VDC vs. 1500 VDC for AI data centers. Explore efficiency, copper savings, DC protection, safety and power distribution with Keentel Engineering.
Transformer overcurrent protection 51/50 time-current coordination curves showing relay settings and
By SANDIP R PATEL • November 12, 2026
Learn transformer overcurrent protection using 50/51 relay stages, inverse-time curves, inrush checks, fault coordination and arc-flash mitigation.
SVC vs. STATCOM reactive power comparison showing voltage stability and performance during voltage s
By SANDIP R PATEL • November 12, 2026
Compare SVC vs. STATCOM for dynamic reactive power compensation, voltage stability, weak grids, response time, harmonics and transmission system design.
LVDC data center power distribution chart comparing AC, hybrid, 380 VDC and 800 VDC efficiency.
By SANDIP R PATEL • November 11, 2026
Explore LVDC power distribution for data centers, including 380V vs 800V DC, architecture, grounding, protection, UPS, energy efficiency and safety standards.
Grid reliability vs. resilience comparison showing storm outage and power restoration performance
By SANDIP R PATEL • November 11, 2026
Explore grid reliability vs. resilience, extreme weather planning, outage recovery, IEEE 1366 metrics, microgrids and IBR challenges with Keentel Engineering.