CL-12 / Reproducibility / proposed
The reproducibility relay
Can a new teammate repeat a result from the record alone?
01 / One week
Make the smallest useful proof.
Take another project’s synthetic dataset and instructions. Re-run three key tests independently and document discrepancies.
02 / Four weeks
Stretch the experiment.
Build a reusable evidence checklist, comparison report and release gate across four projects.
03 / Materials
Start with what is available.
- $0–40; existing laptop
- Skills to practise: QA, documentation, skeptical reasoning
Illustrative materials only. Equipment purchases, expenses and access need human review.
04 / Privacy + boundaries
Choose a safe test fixture.
Only use approved synthetic/public data. Do not copy secrets into logs or let agents mark their own work independently verified.
05 / Success
Measure the change.
Three tests reproduce within declared tolerance or yield an explicit, actionable failure report
06 / Deliverable + review
Leave a usable handoff.
Independent replication note, annotated checklist and corrected instructions
Human review: A human mentor defines tolerances and reviews discrepancies. Mike decides release; agents help execute but do not approve themselves.
Before work enters the world.
- Pay, hours, eligibility, supervisor, location, agreements and legal review must be resolved before recruitment or work begins. Budgets are illustrative materials estimates in USD, exclude labor, tax and shipping, and are not spending authorization.
- Use synthetic or licensed public fixtures first. A human supervisor must approve any participant involvement, consent, collection and data lifecycle.
- Agents may draft, code, propose tests and document evidence. They cannot supervise interns, approve their own work or infer authority to send messages, buy materials or publish.
- PointCast publication is the programme’s intended context. Physical venues, partners and installations need separate confirmation; no external venue partnership is claimed.
- Credit every contributor with permission. Publish measured results and limitations only after human review; preserve a record of corrections.