Tom/eDNA_Stream_E01_custom_models
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Made more compatible with experimental conditions

Browse Source
This commit is contained in:
Tom Kasper
2026-10-03 15:17:03 +01:00
parent 96ae5d78ba
commit 3e8ccdbdad
7 changed files with 188 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+20 -18
View File
@@ -157,25 +157,27 @@ rehydrated, never guessed, when a saved checkpoint is loaded.
| Flag | Default | Role |
|------|---------|------|
| `--store` / `--out-dir` | required (or via base config) | Where the signal store lives and where the run directory is created. |
| `--store` / `--out-dir` | required (or via base config) | Where the signal store lives and the run directory itself (created if missing; artifacts written directly into it). |
| `--config` | — | Base `RunConfig` JSON (a previous run's `config.json`). |
| `--run-id` | derived | If empty: `{arch}_{budget_M}M_s{stride}_g{n_genera}` (control adds `-shuf`). |
| `--stage` | `arch_ladder` | Stage tag: `arch_ladder`, `windows_ablation`, `robustness`, `control`, `extra`. |
| `--stage` | `ladder_point` | Stage tag: `ladder_point`, `windows_ablation`, `robustness`, `control`, `extra`. |
| `--control` | off | Marks the run as a shuffled-label control (overrides `--stage`). |
| `--notes` | — | Free-text note stored verbatim in `config.json`. |
### Run directory layout
`--out-dir` is the run directory; give one per run (e.g. one per
snakemake rule output).
```text
OUT_DIR/
└── RUN_ID/
├── config.json # realised RunConfig (round-trips exactly)
├── ckpt.pt # best-epoch weights (CPU tensors)
├── history.parquet # per-epoch: epoch, lr, train/val loss, val acc, recall, seconds
├── val_report.json # written by `validate`
├── report.json # written by `test`
├── trap_report.json # written by `test` (when --trap-store is given)
└── gate_verdict.json # written by `gate` (common parent of the runs)
├── config.json # realised RunConfig (round-trips exactly)
├── ckpt.pt # best-epoch weights (CPU tensors)
├── history.parquet # per-epoch: epoch, lr, train/val loss, val acc, recall, seconds
├── val_report.json # written by `validate`
├── report.json # written by `test`
├── trap_report.json # written by `test` (when --trap-store is given)
└── gate_verdict.json # written by `gate` (common parent of the run dirs)
```
### Control runs
@@ -239,19 +241,19 @@ python -m custom_models build --arch cnn --budget 1000000 --n-classes 6
# 3. tune the class ladder (train each stage on the same store)
for g in 3 4 5 6; do
python -m custom_models train --store data/store_v1 --out-dir runs \
--arch cnn --budget 1000000 --genera $(head -n $g genera.txt | paste -sd,) \
--stage arch_ladder --notes "information ceiling"
python -m custom_models train --store data/store_v1 \
--out-dir runs/g$g --arch cnn --budget 1000000 \
--genera $(head -n $g genera.txt | paste -sd,) \
--stage ladder_point --notes "information ceiling"
done
# 4. shuffled-label control
python -m custom_models train --store data/store_v1 --out-dir runs \
# 4. shuffled-label control (its own run directory)
python -m custom_models train --store data/store_v1 --out-dir runs/control \
--arch cnn --budget 1000000 --control
# 5. evaluate + gate (candidates need a `report.json` from `test`)
python -m custom_models test --run-dir runs/cnn_1.0M_s4_g6 --trap-store data/trap
python -m custom_models gate --runs runs/cnn_1.0M_s4_g6 \
--control runs/cnn_1.0M_s4_g6-shuf
python -m custom_models test --run-dir runs/g6 --trap-store data/trap
python -m custom_models gate --runs runs/g6 --control runs/control
```
Run a subset via: