Decision dossier

The next decision

Address the issues found, retain the test inputs containing deliberate defects and rerun the affected checks. Before using real capital, you also need to review costs, risk, the timing of model selection and execution, and identify who is responsible for the decision. Finding no issues in these checks does not constitute approval to deploy capital.

Reports included in this dossier

• Observed account value
• Comparison of decision records
• Assigned work and returned results
• Observations required by the calculation

Each report addresses the question in its title. Read its inputs, findings and limitations in the corresponding section. No report has been supplied for the checks listed as missing.

Checks without a report: When inputs were available, Learning and evaluation data, Comparison of the two rankings

NOT ASSESSED: order execution by the broker · inclusion of all costs · selection history · future returns

Observed account value

Drawdown at observation points (%)

Observations
12

Maximum observed drawdown
13.86%

The program reads account values in the CSV and measures the fall from each previous peak. The observation count tells you how many values it read. It cannot reconstruct lows between samples; daily data may leave an intraday loss invisible.

Declared input file hashes

SHA-256 of the check report (JSON)

{
  "version": "2.0.0-pilot",
  "command": "equity",
  "input_sha256": "bcd623762610ef5fdb589b8ed44abe8db08c0778439ee4578dc7c82d4fcfecea",
  "parameters": {
    "command": "equity",
    "assume_no_flows": false,
    "initial_groups": 3,
    "test_groups": 2,
    "gap_seconds": 0,
    "top_k": 3
  },
  "result": {
    "scope": "observed_samples_only",
    "observations": 12,
    "first": "2026-01-01T00:00:00Z",
    "last": "2026-01-12T00:00:00Z",
    "largest_gap_seconds": 86400.0,
    "cash_flow_basis": "supplied_zero_flow_column",
    "equity": {
      "max_drawdown_pct": 13.861386138613863,
      "peak_row": 3,
      "trough_row": 6,
      "underwater_pct": [
        0.0,
        4.0,
        0.0,
        2.9702970297029703,
        8.91089108910891,
        13.861386138613863,
        0.0,
        4.901960784313726,
        7.8431372549019605,
        0.0,
        2.912621359223301,
        0.0
      ]
    },
    "balance": {
      "max_drawdown_pct": 0.0,
      "peak_row": 1,
      "trough_row": 1,
      "underwater_pct": [
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0
      ]
    },
    "not_assessed": [
      "between_sample_drawdown",
      "overfitting",
      "temporal_leakage",
      "cost_completeness",
      "broker_accuracy",
      "future_profitability"
    ]
  }
}

Comparison of decision records

You are rewriting the function that selects purchases and want to check it before replacing the existing one. The records describe the same decision about invented stocks A and B: the model scores A at 0.7 and B at 0.6, with both eligible. The first program selects A; the second selects B. selected marks a chosen stock (1) or an excluded stock (0); size is the requested quantity, here one share of the selected stock. Two rows therefore change, one for A and one for B. The records retain scores and choices but not the code linking them: the report detects a difference without proving its cause. In the invented Museum case, you can instead follow and reproduce a specific mechanism: reordering stocks without reordering their scores. We have not calculated which purchase would have been more profitable.

Issues found
2

First difference
Selection (selected)

The program compares matching IDs in two records and reads fields in a fixed order: decision time, score, side, eligibility, selection and size. “First difference” means the first discrepancy encountered in that comparison, not an established cause or necessarily the earliest event in time. It does not check steps missing from both records or broker fills.

What the findings mean

• ID: A. Different fields: selected, size.
• ID: B. Different fields: selected, size.

Declared input file hashes

SHA-256 of the check report (JSON)

{
  "version": "3.0.0-pilot",
  "command": "trace",
  "input_sha256": [
    "45eccbc4320b3a0c140d88391cec536720c98e9ae8cadd4855690c6db8f42aab",
    "b0ba1fcbcedfdfe40c98dfd04c2c33791bd923232cbbc4aea9f9b2c973e4cd56"
  ],
  "result": {
    "scope": "declared_trace_only_not_execution_or_cost_parity",
    "issues": [
      {
        "id": "A",
        "stage": "selected",
        "changed_fields": [
          "selected",
          "size"
        ],
        "left": {
          "selected": "1",
          "size": 1.0
        },
        "right": {
          "selected": "0",
          "size": 0.0
        }
      },
      {
        "id": "B",
        "stage": "selected",
        "changed_fields": [
          "selected",
          "size"
        ],
        "left": {
          "selected": "0",
          "size": 0.0
        },
        "right": {
          "selected": "1",
          "size": 1.0
        }
      }
    ],
    "left_rows": 2,
    "right_rows": 2,
    "exact_match": false,
    "first_difference": {
      "id": "A",
      "stage": "selected",
      "changed_fields": [
        "selected",
        "size"
      ],
      "left": {
        "selected": "1",
        "size": 1.0
      },
      "right": {
        "selected": "0",
        "size": 0.0
      }
    },
    "not_assessed": [
      "missing_stages_in_both_traces",
      "broker_fills",
      "source_code_closure",
      "cost_model"
    ]
  },
  "verdict": "ANOMALIES"
}

Assigned work and returned results

Before repeating a backtest, you want to know which sources to repair. Ask two AI assistants to look for inconsistent dates and missing values in four CSVs, A, B, C and D: assign A and B to the first, C and D to the second. In this invented case, each response instead contains a review card for all four files. You receive eight cards about the same four CSVs, not eight new sources: you must compare repeated reports and reconcile their findings before deciding which data to repair. The check below flags four cards outside their assignment and four repetitions, eight findings in total; both criteria can concern the same file. A correct delivery contains one card per assigned file, without cards from the other assignment. This program checks declared identifiers; it does not prove which files were actually read, the quality of the analysis or the independence of the judgements. The Museum addresses a different need: a second opinion on the same originals, with the earlier verdict withheld from the second reviewer.

Issues found
8

Distinct results required
4

A batch is a group of items assigned to the same request. The check compares assigned identifiers with those present in returned results. One result can trigger several findings, for example because it concerns an item outside that request’s assignment and is also duplicated. The finding count is therefore not the number of independent problems and does not measure analysis quality.

What the findings mean

• ID: C. Request: one. The response includes a result that was not assigned to this request.
• ID: D. Request: one. The response includes a result that was not assigned to this request.
• ID: A. Request: two. The response includes a result that was not assigned to this request.
• ID: A. Request: two. This result was returned more than once.
• ID: B. Request: two. The response includes a result that was not assigned to this request.
• ID: B. Request: two. This result was returned more than once.
• ID: C. Request: two. This result was returned more than once.
• ID: D. Request: two. This result was returned more than once.

Declared input file hashes

SHA-256 of the check report (JSON)

{
  "version": "3.0.0-pilot",
  "command": "batches",
  "input_sha256": [
    "789fe77a3db498e27a400055108c93bd08d488cbbce96e95b26148c1477e60e2"
  ],
  "result": {
    "scope": "declared_identity_partition_only",
    "targets": 4,
    "issues": [
      {
        "code": "RESPONSE_OUTSIDE_REQUEST",
        "batch": "one",
        "id": "C"
      },
      {
        "code": "RESPONSE_OUTSIDE_REQUEST",
        "batch": "one",
        "id": "D"
      },
      {
        "code": "RESPONSE_OUTSIDE_REQUEST",
        "batch": "two",
        "id": "A"
      },
      {
        "code": "DUPLICATE_RESPONSE",
        "batch": "two",
        "id": "A"
      },
      {
        "code": "RESPONSE_OUTSIDE_REQUEST",
        "batch": "two",
        "id": "B"
      },
      {
        "code": "DUPLICATE_RESPONSE",
        "batch": "two",
        "id": "B"
      },
      {
        "code": "DUPLICATE_RESPONSE",
        "batch": "two",
        "id": "C"
      },
      {
        "code": "DUPLICATE_RESPONSE",
        "batch": "two",
        "id": "D"
      }
    ],
    "not_assessed": [
      "answer_quality",
      "model_context",
      "economic_benefit",
      "retry_idempotency"
    ]
  },
  "verdict": "ANOMALIES"
}

Observations required by the calculation

You are preparing an indicator for the rule deciding a purchase. Before calculating it, you need every price required by its window. This report’s invented calendar requires 1, 2 and 3 January 2026; the file contains only day 1 with value 11 and day 3 with value 13. The decision on day 3 is therefore missing the 2 January value. The report flags MISSING_OBSERVATION, letting you stop preparation before incomplete input reaches the rule or model. Recover the value from its source; if you cannot, keep that decision blocked for missing input. This report does not calculate the indicator or simulate a purchase. In the moving-average case, you instead see which order can change when an older session hides the gap. The dates in this report belong to a teaching calendar, not a stock exchange’s sessions.

Issues found
1

Decisions examined
1

Before an indicator is calculated, the program looks for every observation required by the declared calendar. It separately flags a missing date, an empty value and an input arriving after the decision. It checks presence and timing, not the formula’s correctness or the values’ origin.

What the findings mean

• Decision time: 2026-01-03T00:00:00Z. The window does not contain every observation required by the declared calendar. Required dates that are missing: 2026-01-02T00:00:00Z.

Declared input file hashes

SHA-256 of the check report (JSON)

{
  "version": "3.0.0-pilot",
  "command": "windows",
  "input_sha256": [
    "5b501d08ef6c4264e3b82465fd3cf01e0f4efd50f6637bb1d6e86869e73d76be"
  ],
  "result": {
    "scope": "single_series_declared_calendar_and_availability",
    "required_observations": 3,
    "windows": [
      {
        "decision_at": "2026-01-03T00:00:00Z",
        "expected": [
          "2026-01-01T00:00:00Z",
          "2026-01-02T00:00:00Z",
          "2026-01-03T00:00:00Z"
        ],
        "missing": [
          "2026-01-02T00:00:00Z"
        ],
        "late": [],
        "nulls": [],
        "codes": [
          "MISSING_OBSERVATION"
        ]
      }
    ],
    "issues": [
      {
        "decision_at": "2026-01-03T00:00:00Z",
        "expected": [
          "2026-01-01T00:00:00Z",
          "2026-01-02T00:00:00Z",
          "2026-01-03T00:00:00Z"
        ],
        "missing": [
          "2026-01-02T00:00:00Z"
        ],
        "late": [],
        "nulls": [],
        "codes": [
          "MISSING_OBSERVATION"
        ]
      }
    ],
    "not_assessed": [
      "exchange_calendar_accuracy",
      "provider_revisions",
      "feature_formula",
      "model_fit_exposure"
    ]
  },
  "verdict": "ANOMALIES"
}