How Braintree Coaching Australia measures results: analytics and marking methodology

Braintree Coaching Australia is an Australian exam-preparation provider (not the PayPal payments service). This page explains the methodology behind the numbers we report: how our analytics and AI-assisted marking are validated, what we measure, and what we deliberately will not claim. It is the companion to our results and outcomes hub, which reports the findings; this page explains how those findings are produced.

We publish it so a parent comparing providers can see the workings behind each claim. If a claim cannot be traced back to a method and a record, we do not make it.

How is Braintree Coaching Australia's scoring analytics validated?

Our analytics are validated by being deterministic and checkable. The clearest example is our selective entry score calculator: it runs a frozen, versioned scoring engine with the cut-off values held server-side, so the same practice inputs always produce the same interpretation, and the engine cannot be quietly changed between sittings.

Those cut-off values are not guessed. They encode the cut-off trends we observe each published season, updated only when a new season's official data is published. That is why the calculator is guidance for interpreting a practice result — never an official score. The NSW Department of Education runs the real selection and reserve-list process; the placement test page is the authoritative source for test format and sitting rules. Where our tool and the Department differ, follow the Department.

How is writing feedback both AI-marked and teacher-marked?

For selective and Opportunity Class writing, Braintree Coaching Australia gives both AI-marked and teacher-marked writing feedback on the same script. The workflow is fixed:

1. AI-marked first pass. AI-assisted marking triages a large set of scripts quickly — flagging likely errors, timing patterns, and recurring gaps. That AI-marked pass never stands as the final judgement a parent sees.
2. Teacher-marked second pass. Every qualitative assessment of writing and reasoning is teacher-marked by a qualified tutor against the relevant exam criteria before it reaches a family.
3. Tutor record is the source of truth. Where the AI-marked and teacher-marked passes disagree, the tutor decides. The mock-review log parents rely on is the teacher-marked outcome, not the automated pass.

In practice this means AI speeds up triage; it does not replace the teacher. For the parent-facing marking SLAs, what AI versus teachers do, and the Calm Data walkthrough of how feedback arrives on the LMS, read our AI and teacher marking guide. This methodology page keeps the validation principle and measured-versus-not-claimed remit; it does not duplicate those parent workflow details.

What does Braintree Coaching Australia measure?

We measure preparation and observed movement, not certainty. Concretely, our tutor team records:

• Mock-review logs — how each student performs across full timed practice papers, and which mistakes recur, section by section. These logs are the evidence behind our published common-mistake banks for OC practice and the NSW Selective format guide.
• Reserve-list tracking — when NSW Selective reserve-list families report offers, logged by band across a full round (published on our reserve-band timing report).
• Enrolment volume — more than 5,000 students enrol with Braintree Coaching Australia each year across free and paid courses, published as an audited lower bound rather than an exact daily count.

These are the inputs. The results and outcomes hub turns them into an inspectable programme definition, anonymised ranges and a consented descriptive report — never a cherry-picked trophy board or an unaudited success rate.

What does Braintree Coaching Australia not claim?

These are the claims we refuse to make:

1. No placement total or success rate. We do not publish a "students placed" number or a percentage success rate, because no audited figure exists. A previous internal estimate was retracted rather than published. Parent-reported review cards on the results and outcomes hub are not converted into a rate.
2. No causal coaching claim without evidence. Improvement on repeated practice items is preparation movement in our logs — not proof of general learning transfer, and not proof that coaching caused an official offer.
3. No individual predictions. Everything we report is aggregate and historical. A practice score, a calculator estimate, and a mock percentile describe preparation — they do not predict how a child will do on the day.
4. AI does not replace the teacher. Automated marking never delivers the final qualitative judgement; a qualified tutor always does.
5. Official process wins. Selection, cut-offs, and reserve-list movement are set by the relevant education authority — see the NSW Department of Education selective high schools overview — not by our analytics.

The hub now publishes the programme fields (enrolment cohort, participation, follow-up window, denominator, missing responses, verified outcome source) and a consented descriptive report limited to records we can audit. Method claims on this page stay lockstep with that hub; we do not invent a second outcomes narrative here.

Where do these figures come from?

The score-calculator bands on this page come from the scoring engine we run for families. The annual enrolment figure is the audited lower bound we publish as "5,000+". We do not estimate a placement total. If we do not have an audited figure, we leave it off the page.

How do we commission an external material review?

Parents comparing programmes should be able to inspect how practice materials are reviewed — including when that review has not yet been commissioned. This section publishes the criteria, sample limitations, relationship rules, and correction-log status for our first representative external material review across NAPLAN, NSW Selective, and ACER Scholarship sample packs. It does not claim that an independent educator has already signed off.

Reviewer credentials

Not yet commissioned. A named qualified independent educator will appear here only after operator authorisation of the reviewer and spend. We will not invent a person, photo, CV, or “externally reviewed” badge to fill this slot.

Relationship and conflicts

When the commission proceeds it will be a paid engagement. We will state the paid relationship, selection method, review date, and any conflicts (prior employment, tutoring of the sampled packs, equity, or marketing roles). A paid review is not an endorsement by ACARA, the NSW Department of Education, or ACER. Coaching is not necessary for NSW Selective or Opportunity Class entry, and those departments do not recommend or endorse coaching.

Review criteria

Every item in the frozen sample is scored on the same axes we use for internal academic review: correctness; age and level fit; skill alignment; explanation quality; distractor quality; transfer or follow-up where present; accessibility and clarity; and honest original-versus-official labelling. Weak, typical, and strong items are included — not only polished examples.

Sample limitations

The first commission covers a small representative sample only: selected Year 3 NAPLAN library items (with one Year 5 cross-check), NSW Selective sample-pack items, and ACER Scholarship Level P / Level 1 teaching examples on our ACER Scholarship practice resources page. It is not a census of every Braintree pack, and it does not cover OC, HAST, EduTest, or SEHS banks in this round. All sampled items are Braintree-authored teaching examples — not official past papers.

Correction log

No material errors from an external material review have been confirmed yet. The product material-error changelog is intentionally empty until a real finding is accepted. When corrections ship, they will be listed here with the set, version, and item citation so a parent can see what changed.

Where the commission packet lives

The frozen sample list, criteria table, and signed-report attachment sit in the Phase 28 E33 evidence pack in this repository. Public claims on this page stay lockstep with that packet: credentials and signed findings appear here only after the report is filled by a real educator.

Related resources

• AI and teacher marking (Calm Data) — parent-facing hybrid marking SLAs and walkthroughs
• Results and outcomes — the anonymised findings this methodology produces
• Selective entry score calculator — the deterministic tool described above
• NSW Selective results interpretation — how to read a Selective result
• OC results — understanding an Opportunity Class outcome
• Reserve-band timing 2024–2025 — school-by-school offer history
• Selective school preparation — the full NSW Selective prep hub