Resident Model Lab studies local language-model inference around the existing OLMoE mixture-of-experts model. Nurowerx develops the runtime, not the foundation model. The question is which expert weights should remain in GPU memory without changing the model router’s choices.
Resident Model Lab is an implemented local research platform for studying how an existing mixture-of-experts language model can operate with fewer of its components held in fast GPU memory at once. The software manages component loading and residency, then measures GPU memory, logical expert reads, response time, and output fidelity.
An experimental command-line runtime manages a bounded GPU-resident expert set. Selection v2 keeps the router's expert choices intact while a layer-partitioned cache decides which expert weights stay GPU-resident. Generated tokens were identical across all 128 tested requests.
The existing OLMoE router chooses experts for each layer. Selection v2 computes exactly those experts — no substitution, no compression.
A bounded, layer-partitioned cache decides which experts stay on the GPU, protecting the ones the current request needs and evicting others.
The complete expert weight store (about 12 GiB) stays on local storage; missing experts are loaded on demand. Storage is not active model context.
Selection v2 always executes the experts requested by OLMoE's original router. It is not model compression and does not substitute experts. Nurowerx builds the surrounding local runtime; the foundation model is existing open work.
Expert caches are partitioned by layer, with protection for experts the current request needs, replacing a single global LRU cache. The complete expert store (about 12 GiB) remains on local storage and is loaded on demand.
Results from one small experiment with serial, nonrandomized passes and an empty application cache. Timings are subject to caching, thermal, and CPU effects. Each figure is annotated with what it does and does not show.
New 128-slot vs prior 256-slot default. Caused by halving cache capacity, not by the new algorithm alone — and mean latency was 11.4% higher.
16 engineering tasks × 2 cache conditions × 4 configurations. Not a claim of universal output equivalence.
Also 13.6% fewer logical expert reads (9.9% / 4.8% at 256 slots). Logical reads are not measured SSD I/O.
Includes bitwise-equal full next-token logits for three selected prompts, empty and warm cache.
| Cache strategy | GPU slots | Peak GPU tensors | Mean response |
|---|---|---|---|
| Prior global LRU | 256 | 3.92 GiB | 14.53 s |
| Layer-partitioned (v2) | 256 | 3.92 GiB | 13.10 s |
| Prior global LRU | 128 | 2.42 GiB | 17.28 s |
| Layer-partitioned (v2) | 128 | 2.42 GiB | 16.19 s |
All four configurations solved the same 10 of 16 tasks, including shared failures — fidelity was preserved, correctness was not improved. At equal capacity, layer partitioning reduced mean latency and logical expert reads.
Only eight novel independent tasks; no independent audit or full public code release. No host-memory savings, cheaper hardware, or commercial benefit is claimed.
An MoE expert is a mathematical model component, not necessarily a domain specialist. Cache residency changes which weights are held in fast memory, not which experts the router requests. Logical expert reads count runtime requests; they are not measured physical SSD reads.
Layer partitioning and request protection were tested together, not independently. The public note proposes larger independent task suites, longer contexts, randomized repetitions, isolated mechanism studies and host-memory constraints. Those experiments are not completed findings.
Public research software v0.1.0: TRACE · LOOP · LENS · FRAME connects provenance, evaluation, visualization and reviewed corrections. Includes NRS-Bench, three technical reports and 54 developer/CI-reported tests on controlled synthetic evaluations, not a real-model or human study.
Recorded-video detection, temporal tracking, replay and exports in a local prototype. The synthetic map is not verified geolocation.
Evidence provenance, conversational memory and inspectable uncertainty, with founder-led Epistemic Trust Lab synthetic-policy evaluation kept distinct.
Public notes disclose reported methods and limitations; they are not a substitute for independent validation.
All research themes →