> Publication note: experimental results below describe the original frozen source and runtime. The separate input-validation/verification revision is documented at the end.

# Independent solver reliability study

Status: **complete**. Counts: `{'expected': 896, 'attempted': 896, 'successful': 896, 'failed': 0, 'pending': 0, 'comparisons': 28}`.

Protocol: `bda2d2f7df523c492c9959f294d8506bc2d71896e5b98a5f482994c3f8bd3377`. Core `2.2.1-projected-separation` unchanged.

**Primary comparator is the frozen old projected controller, not repulsion.**

Historical replay: `{'classifications': {'feasible_witness': 10}, 'episodes': 9, 'old_exact': 10, 'revised_certified': 10, 'steps': 10}`.

Gates: `{'safety_reliability': False, 'progress_cost': False, 'decision': 'retain_frozen_controller', 'promoted': False}`.

SLSQP success means reported convergence, not a proof of optimality. Feasibility is determined by independent residual and strict swept checks.

## Descriptive totals (not pooled inference)

| Controller | Complete | Collisions | Physical violations | Uncertified | Independent uncertified | Optimizer failures | Recovered |
|---|---:|---:|---:|---:|---:|---:|---:|
| old_projected | 448 | 0 | 0 | 29 | 0 | 29 | 0 |
| revised | 448 | 0 | 3 | 0 | 0 | 113 | 113 |

## Paired revised minus old projected

Each row has its own 16 seeds. Values are mean [95% descriptive paired Student-t interval]. All metrics, missing seeds, and controller means are in `summary.json`.

| Condition | Policy | n | Observed fraction delta | RMSE delta | Energy proxy delta | Safety | Progress/cost |
|---|---|---:|---|---|---|---|---|
| nominal | adaptive_lloyd_bandit | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| nominal | fixed_lloyd | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| nominal | lloyd_no_consensus | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| nominal | random_walk | 16 | 0 [0, 0] | 3.62729e-08 [-1.69487e-08, 8.94945e-08] | 0.0115983 [-0.00750689, 0.0307035] | False | True |
| nominal | lawnmower | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| nominal | static_kmeans | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| nominal | frontier | 16 | 0.000976562 [-0.000657996, 0.00261112] | 0.00047748 [-0.00107989, 0.00203485] | 0.0603644 [-2.53685e-05, 0.120754] | True | True |
| wind | adaptive_lloyd_bandit | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| wind | fixed_lloyd | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| wind | lloyd_no_consensus | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| wind | random_walk | 16 | -6.10352e-05 [-0.000191129, 6.90582e-05] | 2.17535e-08 [-2.4613e-08, 6.812e-08] | 0.00879473 [-0.0099508, 0.0275403] | True | True |
| wind | lawnmower | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| wind | static_kmeans | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| wind | frontier | 16 | 0.00518799 [-0.00305018, 0.0134262] | -0.00488666 [-0.0119241, 0.0021508] | 0.0402861 [-0.0336213, 0.114193] | True | True |
| command_delay | adaptive_lloyd_bandit | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| command_delay | fixed_lloyd | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| command_delay | lloyd_no_consensus | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| command_delay | random_walk | 16 | 0 [0, 0] | 2.06712e-07 [-2.33884e-07, 6.47307e-07] | 0.0030226 [-0.00341992, 0.00946511] | True | True |
| command_delay | lawnmower | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| command_delay | static_kmeans | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| command_delay | frontier | 16 | 0.000366211 [-0.00433398, 0.00506641] | 0.00140775 [-0.00223548, 0.00505098] | 0.142972 [-0.368438, 0.654382] | True | False |
| combined_stress | adaptive_lloyd_bandit | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| combined_stress | fixed_lloyd | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| combined_stress | lloyd_no_consensus | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| combined_stress | random_walk | 16 | 0 [0, 0] | 0 [0, 0] | -2.44894e-10 [-5.8317e-10, 9.33817e-11] | True | True |
| combined_stress | lawnmower | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| combined_stress | static_kmeans | 16 | 0 [0, 0] | 0 [0, 0] | 0 [0, 0] | True | True |
| combined_stress | frontier | 16 | 0.00769043 [-0.00572791, 0.0211088] | -0.00482245 [-0.0157135, 0.00606858] | 0.119078 [-0.0574676, 0.295624] | False | False |

## Interpretation and limitations

Observed fraction is physical grid coverage, not information coverage. Energy is mean squared stored velocity, not joules. Path length, acceleration integral, and preferred-command deviation are retained per episode and paired in the JSON summary.

All executed motions are independently audited in extended-precision swept geometry with endpoint checks, finite-difference integration and wind-adjusted actuator acceleration. No missing records are counted safe. Partial counts are lower bounds.

Recovery witnesses certify local convex feasibility, not global physical viability. Failed optimizer termination is not infeasibility. The historical optimizer candidates are reconstructed, while the original journal motion states are copied verbatim into replay evidence.

The preregistered gates are necessary conditions for research nomination only. No automatic promotion, no real-flight claims, no population safety guarantee, and no tuning on these seeds.

See `../../../docs/SOLVER_RELIABILITY_STUDY.md`, `protocol.json`, `source_snapshot/`, `replay.json`, `episodes/`, and `verification.json` for methods and evidence.

## Findings: why the candidate is not promoted

All 10 historical failures and all 29 new old-projector failures have independently checked feasible witnesses. All ten reconstructed historical candidates and all ten saved braking commands satisfy the frozen numerical certificate. This is optimizer termination failure, not demonstrated local infeasibility.

The revised controller eliminated controller-uncertified steps (29 to 0), but optimizer failures **increased from 29 to 113**. Those 113 commands were recovered as feasible; they are not claimed to be optimal. All 142 captured new-seed failure states are classified feasible by explicit independently checked commands; none are classified locally infeasible or unresolved.

The three physical audit failures are tiny actuator-acceleration excesses, not collisions. They pass the unchanged velocity-space tolerance but exceed the preregistered acceleration-unit tolerance. They remain failures and block the safety gate:

| Condition | Policy | Seed | Step (zero based) | Actuator acceleration (m/s²) |
|---|---|---:|---:|---:|
| combined_stress | frontier | 8101 | 18 | 5.00000001496449 |
| combined_stress | frontier | 8108 | 18 | 5.00000001120175 |
| nominal | random_walk | 8112 | 74 | 5.00000001046132 |

The allowed physical upper bound is 5.00000001 m/s². No post-outcome tolerance adjustment was made.

- **command_delay / frontier**: observed-fraction lower bound -0.0043339836 (required ≥ -0.005); RMSE upper 0.0050509774 (required ≤ 0.005); energy upper 0.65438238 (required ≤ 0.28959816).
- **combined_stress / frontier**: observed-fraction lower bound -0.005727908 (required ≥ -0.005); RMSE upper 0.0060685792 (required ≤ 0.005); energy upper 0.29562397 (required ≤ 0.33760452).

Coverage and RMSE benefits are not established uniformly across strata; descriptive pooled means cannot override these paired gates. Mean fleet path length was 342.0865 → 342.3973 m; acceleration integral 604.8195 → 601.2380; preferred-command deviation cost 4.8889 → 3.8557. These are descriptive totals, not pooled superiority tests.

## Execution and verification

896 unique 100-step episodes, 89,600 independently audited motions, 448 exact initial field/RNG/state pairs, no missing motions, no failed episodes, no replacement seeds. Tests: **116 passed**. All sources and prior inputs remain hash-identical; all complete records were verified against unchanged journal re-audits in eight bounded batches. No episodes were rerun for verification. `verification.json` binds batches, records, journals, and final summaries.

A preparation schema error was corrected before sealing. The first execution reservation failed before attempts because macOS sandboxing blocked semaphores; the second completed all episodes. Both 600-second reservations remain charged (1200/1800 seconds). The serial verification command timed out at 240 seconds; bounded batch verification replaced it, with no changes to experimental source. Errors and validation details are retained in `execution_notes.json`, `setup_error_001.json`, the coordinator-error artifact, and `verification_timeout.json`. The report tree occupies approximately 474 MB, including full compressed motion evidence and failure snapshots.

## Implementation hardening revision (separate from the experiment)

**validation-hardening-v2** rejects invalid/nonfinite positions, previous velocities, preferred commands, wind, landed flags, shapes, limits, derived scales, and residuals before certification or optimization. The frozen baseline and revised experiment remain the original captured implementations; their 896 episodes were not rerun.

All **89,600 saved motions**, **494,812 saved command/candidate/witness occurrences** (**104,619 unique state/command checks**, cache scoped per episode), and **10 historical replay states** were compared against the original hash-verified audit snapshot in 16 bounded batches. There were **zero certificate, residual, or physical-audit differences**. Uncertified preferred commands and optimizer candidates remain uncertified; they are not silently treated as executed commands.

The hardened implementation also passed its regression suite and the frozen separation/core-v2 suites; exact command and output are retained in `hardening_v2/tests.json`.

No original runtime, cost, progress, or safety metric is attributed to the hardened implementation. Additional validation can add overhead; measuring that requires a fresh study, never replacing saved timing metrics. The decision remains **retain the frozen controller**.

The stable site entry points remain `summary.json` and `report.md`. Original publication bytes are in `experiment_publication/` at unchanged hashes. `verification.json` and the original verification batches are not rewritten; their two publication entries resolve explicitly to that archive. `hardening_v2/manifest.json` binds the new implementation and all classification-comparison receipts.

Safe read-only checks (including failures; no publication, recovery, locks or error-file writes):

```sh
venv/bin/python -B -m studies.solver_reliability.run verify
venv/bin/python -B -m studies.solver_reliability.run verify --batch 0 --max-seconds 120
```

Use batch indices 0–15 for a complete bounded comparison. Verification checks historical snapshots, not current-source equality. A source mismatch is expected and reported; it is never hidden by updating experiment hashes. See `../../../docs/SOLVER_RELIABILITY_STUDY.md` for fresh-run commands and validation details.