# Repeating the Assignments

## Choose Your Comparison

The original study ran five model settings on three assignments twice: 30 sessions. Both runs used the same fixed order. `settings.csv` lists every session in that order.

To repeat an assignment in another application, use the same source files and prompts. Record the application, model, tools, settings, and date. Any changed condition belongs beside the results. A new application or updated model is a new comparison.

## Prepare Each Session

1. Start a fresh session with an empty output folder.
2. Provide `prompts/paste/00-project-instructions.txt` as the shared instructions.
3. Attach the five files in `data/source-pack/august-pipeline/` without changing their contents.
4. Send the selected task prompt unchanged.
5. Record the file, elapsed time, usage, cost, stopping reason, and available action records.
6. Keep failed attempts in the results. Record retries separately.

The prompts name `source-pack/august-pipeline/` and `outputs/`. Preserve those logical folders. The original service mounted uploads under `/mnt/session/uploads/` and collected files from `/mnt/session/outputs/`. Those are standard task locations, not private machine paths. An application with different file handling requires a documented change.

The email prompt requires a Markdown file with fewer than 350 body words. The brief requires one landscape PDF page. The deck requires six to eight landscape PDF pages. Each prompt specifies the content and checks. Do not supply completed outputs to a new model.

## Original Run Settings

The runner used Anthropic Managed Agents with `anthropic==0.125.0`; PDF checks used `pypdf==6.6.2`. The exact configured model identifiers appear in `settings.json` and `rig/run_managed_agents.py`.

Every session had a new cloud environment, no prior conversation, and no attached memory, repository, vault, or external tool server. Network access and package-manager access were blocked. The enabled tools were `bash`, `read`, `write`, `edit`, `glob`, and `grep`. Speed was set to `standard`.

The runner omitted the effort setting. Returned records show `high` for Sonnet, Opus, and both Fable settings; Haiku has no recorded value. No temperature or random seed was set in the agent-creation call. The order seed in the plans did not produce different orders between the two recorded runs.

The retained files do not identify a fixed copy of the service's operating system and installed software. Its hidden instructions and full tool descriptions are also unavailable. The records omit the complete input sent to each model request. These gaps prevent exact recreation of the service environment.

Sources: selected fields from each session's `manifest.json`, `agent.json`, `environment.json`, and `session-created.json` are preserved in `settings.json`. Runner files are retained local source copies; no run-time hash establishes that those exact source bytes were executed.

## Run the Retained Code

The runner requires compatible Anthropic Managed Agents access and access to the listed model identifiers. Availability and behavior may have changed. Check the provider's [sessions](https://platform.claude.com/docs/en/managed-agents/sessions), [files](https://platform.claude.com/docs/en/managed-agents/files), and [event documentation](https://platform.claude.com/docs/en/managed-agents/events-and-streaming) before using it.

From the unzipped package:

```sh
python3 -m venv .venv
.venv/bin/pip install -r rig/managed-agents-requirements.txt
.venv/bin/python rig/run_managed_agents.py --help
```

The runner reads `ANTHROPIC_API_KEY` from the calling environment. No credential is included. For one new email attempt with the original first-session limit:

```sh
.venv/bin/python rig/run_managed_agents.py \
  --model haiku --task b --run-id new-comparison \
  --max-session-usd 3.00 --timeout-seconds 3600 --post-run-account
```

This command starts paid work. The spending limit is a submitted service setting, not a predicted bill. The original second-run Sonnet email recorded $0.36 against a submitted $0.35 limit.

Follow `settings.csv` in order. Start each entry once with its listed model, task, and limit. Use a distinct run name for each repeat. The retained scheduling script follows the plans, which differ from three observed limits. Running it alone does not reproduce those departures.

## Recorded Departures

Run one used different limits by session; its first Haiku email had a $3.00 limit. Run two's plan assigned each new session the remaining study budget. Its first Haiku, Fable 5, and Sonnet emails instead had limits of $0.17, $0.88, and $0.35. `settings.json` preserves every submitted limit.

The repeat Sonnet email reached its spending limit after saving and checking the file. Its recorded $0.36, 198.119 seconds, 22 requests, and 22 tool calls end at that stop. The other 29 sessions stopped normally. The records do not establish other effects of the changed limits.

A separate follow-up asked each model to describe its work. Reported task totals exclude the later interval. For the repeat Sonnet email, raising the budget was followed by a read-only directory check before the account prompt. The check and account share a $0.03 recorded increment that cannot be split exactly. The saved email matches the file written before the budget stop, byte for byte. Preserve both intervals separately; do not assign the full increment to either activity.

`task_stop_reason` and `task_stop_event` in `records/sessions.json` retain the original stopping reason and its source position and time. The [budget-stop record](records/claim-ledger.json) also identifies the final file write and the later check.

Three emails failed the recorded word-count check: Haiku in run one and Fable 5.1 in both runs. No blind review scored output quality. Two fixed-order runs cannot establish typical variation, causes, or expected savings.

A later, separate [numerical review](records/output-headline-audit.md) checked six reported figures across all 30 outputs on September 5. The checks were selected after the runs; model labels were visible. It does not provide an overall quality score.

Sources: the two retained plans, observed settings, session totals, and [claim ledger](records/claim-ledger.json).

## Compare the Results

Compare the recorded task bill, time to the recorded stop, and saved file. Identify any budget stop beside those results. Keep the four token categories separate. `analyze.py` applies the September 3 model prices to each category, then calculates runtime from recorded active seconds.

Across the original sessions, token charges total $76.78834225. The 17,480.326 active seconds add $0.388451688889 at $0.08 per hour. Token and runtime charges total $77.176793938889 before cent rounding. Rounding each session separately reproduces all 30 recorded costs, which sum to $77.17. Rounding the combined study total instead would give $77.18.

The runtime price was verified from current official sources on September 5, 2026; it was not saved with the September 3 token prices. See [runtime pricing](records/runtime-pricing.json) for source links and provenance. All recorded web-search and web-fetch counts are zero. Matching the recorded costs does not identify individual request models or allocate runtime to particular tools.

The published tool records are shortened and cleaned. New runner logs contain local paths and provider account identifiers. The package contains transformed records suitable for public inspection; it does not provide the complete original event archive.