INFERENCE · LAB 04 / 04
One line that sees through a machine.
INTERACTIVE ROOFLINE · perf = min(BW·I, PEAK)
Every architecture debate ends up testifying in front of this chart.
The roofline model has one rule: attainable performance = min(bandwidth × arithmetic intensity, peak compute). The sloped roof belongs to bandwidth-bound workloads, the flat roof to compute-bound ones, and where they meet is the ridge point. Where a workload stands decides which hardware upgrade can help it.
LLM decode's arithmetic intensity is nearly constant — an op or two per byte — so it stands forever at the far left of the slope. That is why piling on compute does nothing for decode: the NPU raises a flat roof it can never touch. The chart below is alive; verify it yourself.
Decode (the blue dot) stands on the slope: multiply bandwidth and it multiplies with you; drag compute to ×10 and it does not move — the flat roof is forever out of its reach. That is the entire mathematics of the AI-PC dilemma. (Relative-value demo.)
Position picks the medicine
Slope-side ailments take bandwidth; flat-side ailments take compute. The wrong prescription upgrades only the keynote.
Decode is pinned left
Autoregressive generation does constant ops per byte; no model size changes its x-coordinate on the roofline.
PIM changes the rules
Processing-in-memory doesn't move the dot right — it lifts the whole slope. This is the case for the Noeton's memory-centric design.