RECURRENT STATE EXPLORER · v0.1

Carry Memory Lab

Inspect a small RNN trained on least-significant-bit-first addition.

One 106-parameter model learned from 1–6 bit inputs. Compare its ordinary recurrence with the same weights reset before every bit, alongside an independent exact carry reference.

A finite synthetic experiment, not a general arithmetic proof. Use exact arithmetic for real calculations. Model probabilities are uncalibrated.

Follow one addition

Operands must be unsigned canonical decimal strings, at most 18446744073709551615. No signs, exponents, decimal points or leading zeros. Width must fit both operands; the sum may need 65 bits. Large operands stay BigInt throughout.

Exact-schema JSON input

JSON.parse last-key semantics apply; the raw JSON is retained. Editing JSON selects JSON mode. Selecting the hidden-state view does not apply or revive earlier inputs.

Loading the frozen model and study results…

Fixed experiment results

Loading study…

Whole-integer exact accuracy · fixed dependent probes, not a population guarantee
SuiteCases per modeRecurrentReset each bit

Seven/eight-bit exhaustive probes use new numeric ranges. Wider fixed random suites contain only 500 cases each; they do not exhaust all 64-bit pairs. Padding repeats the same held-out values at longer widths. Carry/zero stress includes known small pairs, explicitly tagged. Suite memberships overlap; counts cannot be summed as independent observations.

All 80 BPTT epochs, all fixed suite integer accuracies, padding control and full carry-chain error probe
One actual fit, two state modes. Exact addition remains the reference. Open full-size experiment figure.
The fixed 1 plus 15 input, model bit-one probabilities, separate exact carry trace and all eight hidden activations in both modes
The default was chosen by exact carry structure before model testing. Hidden activation plots do not establish a causal carry neuron. Open full-size state figure.

Complete result summary · All 1,304 metric groups · All case predictions, gzip · Every integer failure, gzip · Complete bit logits and probabilities, NPZ

Scope and reproducibility

The task is a designed least-significant-bit-first representation of an exact rule. The reset intervention changes the hidden-state distribution as well as removing recurrence. Finite passing probes do not prove arbitrary addition or identify what an individual neuron means. A high bit accuracy can still give a wrong integer; both scores and all failures remain.

The single seed-329 tanh RNN has 106 parameters and ran 80 full-BPTT epochs, 2,720 updates and 307,760 representation visits. Checkpoint 80 was selected on validation bit NLL before current-model testing. No teacher forcing, true carry input, calibration or post-test tuning.

Original synthetic data, code, weights, results and figures: Renchao Wu, Apache-2.0. Method reference: D2L author textbook, BPTT; no textbook code, prose or figures are copied. This page computes locally with native JavaScript, without external model APIs, analytics or storage.

Reproduce and interpret · Pre-fit protocol · Research basis · Apache-2.0 license