# ASTRA retrospective usability pilot

Status: protocol ready; no participants recruited or results collected.
Purpose: find what busy teams skip or misunderstand. This is not independent scientific validation or a regulatory assessment.

## Select 2–3 real past decisions
Choose one low-cost reversible decision, one consequential decision, and (if available) one decision with an unresolved objection or disappointing outcome. Obtain the team's permission; use redacted material the participants are allowed to share. Do not upload confidential records solely for this test.
Recruit the original decision owner and a colleague who can challenge the reasoning. Record their roles, relevant expertise and conflicts. A facilitator observes without answering form questions immediately.

## Run each session
1. Give participants the information available at the time; keep later outcomes separate. Ask them to define success and identify comparable past cases, including failures. Unknown base rates stay unknown.
2. Ask the owner to complete the lite draft without coaching. Time active work, interruptions and help separately. Record each field skipped and the participant's exact reason.
3. Ask the colleague to reconstruct the decision: what was chosen, why, what would change it, and who must answer an objection? Record errors and missing information.
4. For the consequential case, switch to Full. Observe base-rate reasoning, consequences, reversibility, conflicts, confidentiality, response deadline and escalation owner. Do not turn a name in a field into assumed consent.
5. Introduce an actual historical objection or clearly labeled hypothetical challenge. Ask who should respond and what happens when the deadline passes. Record whether they mistakenly interpret silence, a response, or a lock as agreement.
6. Reveal the historical outcome only after the reconstruction task. Record hindsight contamination and recall limitations. Do not present retrospective accuracy as prospective predictive ability.
7. Ask: What did you skip? Which words confused you? What took longest? What would you remove? What information was missing? Would you use this for another decision, and why?

## Observation sheet — repeat for each case
- Case ID; real / synthetic; permission and redaction status:
- Decision date; information cutoff; reversible / consequential:
- Participant roles, conflicts and relationship to the original decision:
- Format order and prior ASTRA familiarity:
- Setup / completion / review / correction / help minutes:
- Task completed, abandoned or assisted; errors:
- Skipped field; reason; confusing wording (verbatim):
- Reviewer reconstruction errors and unanswerable questions:
- Challenge responsibility understood? Deadline and escalation understood?
- Requested change; severity; proposed repair; retest result:
- Historical outcome (kept separate from at-the-time reasoning):

## Decide what to change
Retain incomplete attempts and unfavorable feedback. Fix any misunderstanding of privacy, approval or evidence status before expanding the pilot. Simplify repeated sources of confusion and retest the same task. With 2–3 decisions, report observations and case limits; do not claim measured population-wide time savings, adoption or scientific effectiveness.