adversarial_foraging/tools/README.md

Optimal grid search over conf.toml space

This directory contains a small pipeline for mapping the phase diagram of the foraging simulator: which team wins as you vary two conf.toml parameters. It is built for the specific experiment red aggressive vs. blue pacifist, varying view_dist and the ball weapon radius, but generalises to any pair of numeric fields.

The idea

A naive sweep would run a fixed, large number of seeds for every grid point. Most of that work is wasted: a point deep in red territory is decided after a handful of seeds, while only points near the phase boundary need many.

The pipeline therefore combines three things:

1. Sequential stopping. Each grid point is evaluated in parallel batches of seeds. After every batch we test the n-sigma stopping rule and stop as soon as the outcome is certain (or a hard cap is hit). Decisive points cost ~64 seeds; only boundary points pay for the full budget.
2. Adaptive refinement. After a coarse grid, cells whose four corners do not agree on a winner are refined (centre + edge midpoints) so extra samples are spent where the boundary actually is. Repeatable.
3. Common random numbers. Every point is evaluated with the same seed sequence 0, 1, 2, …, which removes seed-to-seed noise from comparisons between neighbouring grid points and sharpens the apparent boundary.

The statistic and the stopping rule

Each seed produces scores (red, blu) summing to at most food_amount. Define the per-seed outcome

d = (red - blu) / food_amount          # in [-1, 1], 0 == even match

d > 0 means red out-foraged blue, d < 0 means blue did. Across N seeds we track the running mean m, sample standard deviation s, and standard error SE = s / sqrt(N). The stopping rule is

|m| / SE >= sigma      (default sigma = 3)

i.e. the RED-BLU food-share difference is at least sigma standard errors from zero. The winning side is the sign of m. A point that reaches --max-samples without satisfying the rule is reported as tie (undecided) — that is exactly the phase-boundary band.

Caveat — optional stopping. Looking at the data repeatedly and stopping at n sigma inflates the true type-I error (the "look-elsewhere" effect), so the nominal confidence is optimistic. For a phase diagram this is fine and standard; if you need calibrated error bars, raise --sigma, or use the per-point n, sd_diff and sigma columns to apply a sequential correction (e.g. a Bonferroni or alpha-spending adjustment) afterwards.

Architecture

Piece	Role
simulator --sequential	New flag on the Rust binary (in src/main.rs). Runs seeds in parallel batches, applies the n-sigma rule, prints one full-precision JSON line and exits. Additive: the GUI and fixed-count headless paths are untouched, and all existing tests still pass.
tools/sweep.py	Grid driver. Renders a conf.toml per point, fans points out over a thread pool (each simulator process pinned to 1 rayon thread), does coarse + adaptive passes, and appends resumable JSONL + CSV.
tools/plot_phase.py	Matplotlib phase diagram: discrete winner regions, the mean_diff = 0 boundary, and the continuous difference field.

Usage

# build once
cargo build --release --manifest-path simulator/Cargo.toml

# coarse grid only (fast)
python3 tools/sweep.py --view-dist 100:550:25 --radius 0:250:12.5

# coarse + one adaptive refinement pass, then plot
python3 tools/sweep.py \
    --view-dist 100:550:25 --radius 0:250:12.5 \
    --sigma 3 --max-samples 2000 \
    --refine 1 --refine-max-samples 2500 --seed-budget 3000000 \
    --workers 16 --rayon-threads 1 \
    --out sweep_out/results.jsonl
python3 tools/plot_phase.py sweep_out/results.jsonl -o sweep_out/phase.png

# preview the grid without running
python3 tools/sweep.py --dry-run

# pin other fields with --set (repeatable)
python3 tools/sweep.py --set kills_user=false --set food_amount=50

Axes accept either a range (start:stop:step) or an explicit list (0,10,20,30).

Key options

Flag	Meaning
--view-dist, --radius	the two swept axes
--sigma	n-sigma stopping threshold for the coarse pass (default 3)
--min-samples / --max-samples	lower bound before testing / hard cap
--batch	seeds per parallel batch between rule checks
--refine	number of adaptive refinement passes
--refine-sigma / --refine-max-samples	stopping settings for refined points
--seed-budget	global safety valve: stop scheduling once this many seeds have run
--workers / --rayon-threads	points in parallel / simulator threads per point
--set KEY=VALUE	pin any other conf.toml field
--out	JSONL path (a .csv is written alongside; both are resumable)

Output columns

view_dist, weapon_radius, n, sigma, winner, stopped, mean_red, mean_blu, mean_diff, sd_diff, se_diff, red_wins, blu_wins, ties, weapon_kills_user, food_amount

• n — seeds actually run for this point
• sigma — |mean_diff| / se_diff achieved
• winner — red / blu (sign of mean_diff)
• stopped — sigma (decided) or max_samples (undecided → tie in the plot)

Result of the bundled experiment

conf.toml (red aggressive, blue pacifist, forwardball, kills_user = true), coarse grid view_dist ∈ [100, 550] × 25, radius ∈ [0, 250] × 12.5, one refinement pass, sigma = 3: 714 points, ~650k seeds (a fixed 2000-seed sweep over the same grid would have been ~1.4M seeds).

phase.png shows a clear transition:

• Low view_dist (≲ 140): blue wins almost everywhere. Red starts far from the food and, with little sensing, forages no better than the pacifist while paying for an unused weapon.
• Higher view_dist: red wins above a curved radius threshold and blue below it. The boundary is non-monotonic: the radius red needs dips to a minimum near view_dist ≈ 250 and rises again towards 550, because the forwardball sits at view_end (view_dist ahead of the robot), so a large view_dist places the blast far ahead and needs a larger radius to connect.
• The boundary band (tie points) is where |mean_diff| / SE < 3 even after the sample cap.

Extending

• Other axes: point --view-dist/--radius at any numeric field, or add a third axis. For a third axis, wrap sweep.py in a loop over the extra value and use separate output files, then plot a slice montage.
• Other statistics: the sequential mode currently tracks the food-share difference. Win-rate (red_wins/blu_wins) is already recorded if you prefer a Bernoulli statistic.
• Finer boundary: more --refine passes and/or a larger --seed-budget.