Designing an inference stack for interactive screening

A screen that takes a coffee break to run gets used differently than one that returns while you're still typing. That difference is worth designing for from the start, even before there's a working system to measure — so this is a description of the plan, not a performance report.

The design

The intended shape: batch candidate structures into fixed-size graphs, cache featurization across repeated screens of overlapping composition space, and run a distilled, cheaper model as a first-pass filter before a full ensemble scores whatever survives. Each piece is a standard technique on its own; the combination is what we're building and testing.

Status

The batching and featurization layer is in development. The distilled first-pass filter and the full-ensemble scorer are both planned but not yet built, so there is no throughput or cache-hit number we can honestly report — anything specific here would be invented.

Once there's a working stack, this post gets replaced with real wall-clock numbers, measured the same way for every revision so they stay comparable over time.