Junction Temperature vs Case Temperature: Optimising Cold Plates for AI GPUs

By Chandra Dhannina, Senior Simulations Engineer, Conflux Technology

Key takeaways

• Cold plates are rated on case temperature (Tcase), but GPUs and HBM throttle on junction temperature (Tj), which no production sensor reports at a known location.
• In a 1400 W Blackwell-class model, tuning coolant flow for peak case temperature left the hottest junction about 6.5 K above the achievable optimum. Sharing flow by area left it about 12 K above.
• The junction optimum needs an non-uniform cooling: HBM cases about 11 K colder than the compute-die case, because each HBM stack adds about 15 K between case and junction against about 4.8 K for the die.
• An additively manufactured cold plate with channels graded to the power map lowered the hottest junction by about 12 K at the same flow and inlet temperature.
• Specify cold plates against the package power map and each part’s junction limit. Use case temperature to verify the result, not as the design target.

Ask a cold-plate vendor for a number and you get a thermal resistance from coolant inlet to case. It is the right number to test. Put a heater block under the plate and a sensor behind the contact face, and the result is repeatable and defensible. The accelerator does not care about it. It throttles on the hottest junction inside the silicon, a plane no customer can instrument. On retail Blackwell cards, community tools now read six internal temperature channels that NVIDIA does not document. Even the chip’s own numbers arrive without a location [1].

Most specifications treat case and junction as interchangeable. Cool the case and the junction follows. For a single die with uniform power that is nearly true. For a package with two compute dies and eight memory stacks it is not, and the difference changes what the best plate looks like.

What is the difference between case temperature and junction temperature?

Junction temperature (Tj) is the temperature of the active circuitry inside the silicon, and it is what a GPU throttles on. Case temperature (Tcase) is read on the back of each part, under the thermal interface material. Between the two, a compute die has 0.7 mm of bulk silicon. An HBM stack has twelve bonded joints, which conduct far worse than silicon [2]. The standards commentary is frank about how soft the case number is. On high-performance packages, the junction-to-case resistance of the same part has varied by 300 % with the measurement method [3].

Figure 1. Heat climbs from the junction plane inside each part, through the part and the interface, into the plate. Case temperature is defined at the top of each part. Neither plane carries a sensor on a production part.

The experiment: a 1400 W Blackwell-class cold plate model

We built a Blackwell-class package in our cold-plate model: two 600 W compute dies and eight 12-high HBM3E stacks, 1400 W in total [4]. The coolant enters at 35 °C and 1.5 litres per minute, on a direct-die interface. Then we held the total flow fixed and moved it between the die footprints and the memory footprints, as a graded plate would.

Three separate targets were asked to determine the optimum split: the lowest hottest junction, the lowest average case temperature, and the lowest peak case temperature. The last two are what a bench can measure. The first is what the chip cares about.

Which optimisation target gives the coolest junction?

Figure 2. Total flow fixed, shared between the die and memory footprints. Each target stops at a different split. Temperatures are for a uniform power map, no on-die hotspot.

The junction target stops at about 88 % of the coolant to the dies. There the die and the memory reach the same junction temperature, about 73 °C, and moving flow either way makes one of them worse. The average-case target stops a little earlier, at 86 %, and gives away about half a degree at the junction. Small, and worth knowing.

The peak-case target is the problem. The die stays the hottest spot on the face however hard the memory is starved. So, a target that chases the hottest reading keeps pushing coolant to the dies. It only stops near 94 %, when the memory’s own coolant has passed 50 °C and the HBM case finally overtakes. By then the hottest junction sits about 6.5 K above the optimum. The metric that looks most like a bench result is the one that aims worst.

A plate that shares flow by area, 60 % to the dies, lands about 12 K above the optimum in the model. The dies carry 86 % of the power on 61 % of the area. At 1.5 litres per minute, the coolant they do get warms quickly, and the extra film on the memory side buys little back.

Why equal case temperatures do not mean equal junction temperatures

Figure 3. Each part carries its own case-to-junction offset. A balanced junction needs an unbalanced face.

Each part adds its own offset. At 600 W the compute die’s silicon adds about 4.8 K. A 25 W HBM stack adds about 15 K, because its heat crosses the bonded joints rather than bulk silicon [2]. So the junction optimum wants an uneven face: the memory case about 11 K colder than the die case, 57 °C against 68 °C. A target that asks for a uniform face is asking for the wrong plate.

What no cold plate can fix: on-die hotspots

There is a limit to what any plate can do. Put a tenth of a die’s power into 25 mm² and the junction under it rises by about 18 K while the case barely moves. Heat from a hotspot has to spread through the silicon before the plate can act on it. Two 2 × 2 mm hotspots at 100 W/cm² add about 7 K through the silicon alone, and 70 K at 1000 W/cm². The package material makes no difference to that term [5]. The remedy at that scale is to bring the coolant closer to the source. Microsoft’s embedded microfluidics cut peak silicon temperature rise by up to 65 % in its GPU tests [6].

The plate’s cooling pattern is also blurred before it reaches the junction. Silicon and interface spread heat sideways over roughly √(k·t/h), about 2 mm for this stack. Structure in the plate finer than that is wasted. Structure at the 1 to 10 mm scale, the scale of the power map, is not.

How additively manufactured cold plates lower junction temperature

Figure 4. Same package, same inlet, same total flow. Left: one heat-transfer coefficient everywhere, which is what a fixed fin geometry delivers. Right: the coefficient graded to the junction optimum. The face becomes less uniform and the hottest junction falls by about 12 K.

A skived or milled plate delivers one heat-transfer coefficient across the whole footprint, because its fin geometry is one geometry. An additively manufactured (3D-printed) cold plate can change channel size and fin density from one footprint to the next. The coolant then shares itself out, with no second circuit and no balancing valve. In the model, grading the plate from 23 kW/m²K everywhere to a field of 14 to 30 kW/m²K lowers the hottest junction by about 12 K. Same flow, same inlet. Those are effective coefficients from die back to coolant per unit of footprint, TIM included, so they read lower than a wetted-wall figure would.

The direction surprises people. The hot dies get the coarser channels. The finer channels go on the memory, where they choke the flow and push it toward the dies. Tighter fins over the hottest part would do the opposite of what is needed.

Others are pulling the same lever. Topology-optimised channels matched to a chip’s power map gave 13 % lower temperature rise, or 55 % lower pressure drop, than the best straight channels in simulation [7]. Graded-lattice cold plates for multi-chip modules have been printed and tested [8].

How to specify a cold plate for junction temperature

None of this makes case temperature useless. It is the number you can measure, so measure it. But treat it as an instrument reading, not as the target. Specify the plate against the package’s power map and the junction limits of each part, then check the case temperature it produces. A case target does not aim a cold plate well. A power map does.

Conflux designs, additively manufactures and tests direct-to-chip cold plates, with in-house calorimetric test capability.

If you have a power map and junction limits for your next accelerator, talk to our thermal team about what a graded plate would do for it.

How we modelled it. Conflux in-house cold plate design software with its current bench-anchored fits, which covers about 1 to 20 L/min on uniform heater plates. At the optimum the memory footprint runs below that range, so its film sits at the laminar fully developed floor. Package: two 600 W compute dies and eight 25 W HBM3E stacks, 35 °C inlet, direct die. Case is the back of each part. Junction is case plus power times θJC, with 0.008 °C/W for the die and 0.6 °C/W for a 12-high HBM stack (plausible range 0.35 to 1.10). Uniform power maps, no on-die hotspot. The film coefficient on each footprint scales with the flow it receives, and coolant heating is counted per footprint.