Your GPU cluster is running a hotel. Most engineers are still managing it like a house rental

 

Your GPU cluster is running a hotel. Most engineers are still managing it like a house rental.

Here's a stat that should bother every platform architect: enterprise GPU utilization commonly sits at 20-30%. Not because demand is low — because the allocation model underneath most AI platforms was built for a world where one workload gets one whole GPU, full stop.

Red Hat OpenShift AI just closed that gap architecturally. Dynamic Resource Allocation (DRA) went GA in OpenShift 4.21, and it quietly rewrites how GPUs get shared across a cluster. If you're designing AI infrastructure in 2026, this is the shift to understand.

The old model: renting the whole house

For years, Kubernetes handled GPUs through the Device Plugin framework. A GPU request was just an integer: "give me 1 GPU." No nuance about size, memory, or whether a fraction would do.

To get any sharing at all, teams used static MIG (Multi-Instance GPU) configuration — an admin pre-carves a physical GPU into fixed partitions (say, four 3g.20gb slices) via a ConfigMap, ahead of time, based on a guess about future demand.

That guess is almost always wrong by lunchtime.

If traffic patterns shift — say, a burst of small inference requests hits a GPU pre-sliced for large training jobs — those jobs simply queue. Fixing it means manually reconfiguring the GPU, which often requires draining the node. An external add-on called DAS (Dynamic Accelerator Slicer / InstaSlice) later emerged to patch this with just-in-time slicing, using mutation webhooks to carve partitions on the fly. It worked — but it lived outside the scheduler, a bolted-on helper that could drift out of sync with Kubernetes' own decisions.

The analogy that makes this click

Think of a physical GPU as a big luxury house. The old model could only rent the entire house to one guest — even if they just needed a bedroom. Expensive, and often half-empty.

DRA turns that house into a hotel with rooms of different sizes — some studios, some suites — and puts the front desk (the Kubernetes scheduler) directly in charge of the booking system. No separate concierge running around with a clipboard, guessing who needs what.

How the booking actually works

1. The hotel publishes its room list. The NVIDIA driver publishes a ResourceSlice — live inventory of every available GPU slice and its attributes.
2. The guest states a preference, with a backup. A pod's ResourceClaimTemplate declares what it needs via a CEL expression, plus an ordered fallback: "give me a 1g.5gb slice — if none are free, 3g.20gb will do."
3. The front desk matches it — natively. The Kubernetes scheduler evaluates the claim directly against live inventory. No external controller, no webhook, no second system to keep in sync.
4. The guest gets a specific room. The pod binds to an exact slice — e.g. gpu-0-mig-1g5gb-19-5 — and the platform moves on to the next request.

Static MIG vs. DRA — the short version

Aspect	Static MIG (old)	DRA (new)
Slice configuration	Fixed at setup via ConfigMap; admin pre-decides the split	Declared per-workload, at request time
Changing the split	Manual reconfiguration, often needs a node drain	No reconfiguration — scheduler matches existing slices
Who decides	Admin, ahead of time, guessing future demand	The workload itself
No exact match available	Job fails or sits pending until someone intervenes	Automatic fallback to the next-best slice, declared in YAML
Enforcement point	External controller/webhook sitting outside the scheduler	Native to the Kubernetes scheduler
Flexibility	One fixed shape per GPU, all day	Different pods request different shapes from the same GPU pool

A real production scenario

A platform team runs a single A100 80GB GPU shared across an ML workload mix.

Old world (static MIG): Ops pre-configures the GPU into four 3g.20gb slices at 8 AM, based on yesterday's traffic. At noon, a burst of lightweight inference requests arrives — each only needing 1g.10gb. But the GPU is already carved into big slices. Requests queue. Capacity sits wasted until someone manually reconfigures the GPU, usually after draining running workloads.

New world (DRA): Each inference pod's ResourceClaimTemplate simply states: "give me a 1g.10gb slice, fallback to 2g.20gb." When the burst hits, the scheduler checks the GPU's live ResourceSlice inventory and packs seven small inference pods into slices that actually fit — no human touches the GPU config. Later, when a training job needs a bigger slice, it requests 3g.40gb directly, and the scheduler fits it into whatever capacity is free.

Why this matters beyond the YAML

This isn't a minor scheduling tweak. It's a category shift in how GPU capacity is governed:

• Multi-tenancy gets safer. Different teams can share one physical GPU without one workload starving another.
• Utilization goes up without adding hardware. The same GPU serves more shapes of demand across the day.
• Operational fragility goes down. One less external controller to monitor, patch, and debug at 2 AM.
• It's not just NVIDIA's problem to solve alone anymore — the DRA driver is being donated to the CNCF, moving it toward community, not vendor, governance.

If your platform's GPU story is still "one pod, one whole GPU, first come first served" — or "we pre-sliced it this morning and hope for the best" — that's not a GPU platform. That's a house rental pretending to be a hotel.

---

#OpenShiftAI #Kubernetes #GPU #MLOps #CloudArchitecture #RedHat #PlatformEngineering #AIInfrastructure