GET /v1/submissions/{id}/report returns a hiring decision, not a leaderboard entry. This
page explains each field well enough to build a UI on, and flags where a number means less
than it looks like it does.
Before grading finishes
The endpoint answers200 with evaluated: false rather than 404, so polling is simple:
status on the
submission — evaluated means the report is ready.
Ship score and verdict
ship_score is a 0–100 composite. verdict is the band it falls in, and
verdict_label is that band’s display text:
confidence (high, moderate, low) says how much evidence was available. A low
confidence verdict is a prompt to look at the work yourself, not a weak candidate.
summary is a short prose rationale suitable for showing a hiring manager verbatim.
scored_as_role names the role lens the submission was graded under — the same work can
be judged differently for a junior versus a senior posting.
The four scored areas
pillars always contains exactly these four, in this order:
Each entry looks like:
score can be null
score can be null
null means there wasn’t enough evidence to report a number — a challenge that
produced no meaningful signal for that area, for instance.Render null as “not enough data”, never as zero. Averaging null to 0 makes a
candidate look bad for something they were never given a chance to demonstrate.low_confidence qualifies the score
low_confidence qualifies the score
The score is reportable but thin. Show it with a caveat rather than hiding it, and
don’t rank candidates on a
low_confidence area.not_assessed explains a gap
not_assessed explains a gap
Which parts of the area the challenge didn’t exercise. Useful as UI subtext — “this
challenge didn’t test X” — so a gap reads as a property of the challenge, not a
weakness in the candidate.
scope_exclusions at the top level is the same idea for the assessment as a whole.
Strengths, flags and interview questions
strengths and flags are short human-readable statements about the work. flags are
things to probe, not disqualifiers.
interview_questions is the field most worth wiring up:
Integrity status
integrity_status is clear or flagged.
Treat flagged as routing: send it to a person. Never auto-reject on it.
Facts
facts is raw context, no judgement attached:
There is no “good” value for any of these. High
ai_authored_pct with a high ship score
is effective delegation; the same figure with a low score is unchecked copying. The
interpretation is already in the pillar scores and the summary — show facts as context,
not as a scorecard.Ranking
pipeline_rank is { rank, total } among your own evaluated submissions for that
assessment — “3rd of 14”. null when there’s nothing to compare against.
percentile is omitted entirely until there’s a large enough comparison population to
make one honest. Don’t build a UI that assumes the key exists; check for it.
Per-task breakdown
For multi-task challenges,tasks reports each task’s key, title, kind, time limit, and
budget used against allotted. Empty for single-task challenges.