RECURRENT STATE EXPLORER · v0.1
Carry Memory Lab
Inspect a small RNN trained on least-significant-bit-first addition.
One 106-parameter model learned from 1–6 bit inputs. Compare its ordinary recurrence with the same weights reset before every bit, alongside an independent exact carry reference.
A finite synthetic experiment, not a general arithmetic proof. Use exact arithmetic for real calculations. Model probabilities are uncalibrated.
Follow one addition
Exact-schema JSON input
JSON.parse last-key semantics apply; the raw JSON is retained. Editing JSON selects JSON mode. Selecting the hidden-state view does not apply or revive earlier inputs.
Loading the frozen model and study results…
Reference carry and target bits are shown for comparison. They never enter the RNN; the network sees only the current operand bits and its previous hidden state. Reset is an intervention on the same trained weights, not a second fit or an optimized memoryless baseline.
Every bit, including the terminal carry
Position 0 is the least significant bit. The last input pair is 0,0. On narrow screens, focus or scroll a table to view every column.
| Position | a bit | b bit | Exact carry in | Target bit | Recurrent bit | Recurrent p1 | Reset bit | Reset p1 |
|---|
Inspect raw hidden activations
Eight dimensionless tanh coordinates, −1 to +1. Color is a numeric guide; no coordinate is identified as “the carry neuron.” Full precision is retained in the report.
| Position | h0 | h1 | h2 | h3 | h4 | h5 | h6 | h7 |
|---|
The report includes both modes, every bit and hidden activation, full input values, the exact reference and provenance. Inputs stay in page memory until cleared/refreshed; downloaded files remain where saved.
Complete current report JSON
Fixed experiment results
Loading study…
| Suite | Cases per mode | Recurrent | Reset each bit |
|---|
Seven/eight-bit exhaustive probes use new numeric ranges. Wider fixed random suites contain only 500 cases each; they do not exhaust all 64-bit pairs. Padding repeats the same held-out values at longer widths. Carry/zero stress includes known small pairs, explicitly tagged. Suite memberships overlap; counts cannot be summed as independent observations.


Complete result summary · All 1,304 metric groups · All case predictions, gzip · Every integer failure, gzip · Complete bit logits and probabilities, NPZ
Scope and reproducibility
The task is a designed least-significant-bit-first representation of an exact rule. The reset intervention changes the hidden-state distribution as well as removing recurrence. Finite passing probes do not prove arbitrary addition or identify what an individual neuron means. A high bit accuracy can still give a wrong integer; both scores and all failures remain.
The single seed-329 tanh RNN has 106 parameters and ran 80 full-BPTT epochs, 2,720 updates and 307,760 representation visits. Checkpoint 80 was selected on validation bit NLL before current-model testing. No teacher forcing, true carry input, calibration or post-test tuning.
Original synthetic data, code, weights, results and figures: Renchao Wu, Apache-2.0. Method reference: D2L author textbook, BPTT; no textbook code, prose or figures are copied. This page computes locally with native JavaScript, without external model APIs, analytics or storage.
Reproduce and interpret · Pre-fit protocol · Research basis · Apache-2.0 license