> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dotportion.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Understanding reports
> How to read a ship score, verdict, the four scored areas, integrity status and interview questions — and how not to over-read them.
`GET /v1/submissions/{id}/report` returns a hiring decision, not a leaderboard entry. This
page explains each field well enough to build a UI on, and flags where a number means less
than it looks like it does.
## Before grading finishes
The endpoint answers `200` with `evaluated: false` rather than `404`, so polling is simple:
```json theme={null}
{
"submission_id": "00000000-0000-4000-8000-000000000033",
"status": "submitted",
"evaluated": false,
"ship_score": null,
"verdict": null,
"confidence": null,
"pillars": [],
"strengths": [],
"flags": [],
"interview_questions": [],
"integrity_status": null,
"pipeline_rank": null,
"generated_at": null
}
```
Branch on `evaluated`, not on the presence of `ship_score`. When `evaluated` is `false`
the fields `verdict_label`, `summary` and `scored_as_role` are **absent entirely** rather
than null.
Grading typically takes minutes. Poll every 30–60 seconds, or watch `status` on the
submission — `evaluated` means the report is ready.
## Ship score and verdict
`ship_score` is a 0–100 composite. `verdict` is the band it falls in, and
`verdict_label` is that band's display text:
| Verdict | Field value |
| -------------- | ---------------- |
| Strong yes | `strong_yes` |
| Advance | `advance` |
| Borderline | `borderline` |
| Do not advance | `do_not_advance` |
Show the **verdict**, not the number. A score of 71 versus 73 is noise; `advance` versus
`borderline` is a decision. Teams that surface the raw number end up arguing about single
points.
`confidence` (`high`, `moderate`, `low`) says how much evidence was available. A `low`
confidence verdict is a prompt to look at the work yourself, not a weak candidate.
`summary` is a short prose rationale suitable for showing a hiring manager verbatim.
`scored_as_role` names the role lens the submission was graded under — the same work can
be judged differently for a junior versus a senior posting.
## The four scored areas
`pillars` always contains exactly these four, in this order:
| Area | Field value | What it answers |
| ----------- | ------------- | ------------------------------------------------ |
| Correctness | `correctness` | Does it work, and did they build what was asked? |
| Craft | `craft` | Would you merge this PR? |
| AI Fluency | `ai_fluency` | Are they driving the AI, or being driven by it? |
| Rigor | `rigor` | Do they verify their own work? |
Each entry looks like:
```json theme={null}
{
"id": "craft",
"label": "Craft",
"score": 86,
"low_confidence": false,
"summary": "Named the eviction helper for what it guarantees…",
"not_assessed": []
}
```
Two fields do real work and are easy to skip:
`null` means there wasn't enough evidence to report a number — a challenge that
produced no meaningful signal for that area, for instance.
**Render `null` as "not enough data", never as zero.** Averaging `null` to `0` makes a
candidate look bad for something they were never given a chance to demonstrate.
The score is reportable but thin. Show it with a caveat rather than hiding it, and
don't rank candidates on a `low_confidence` area.
Which parts of the area the challenge didn't exercise. Useful as UI subtext — "this
challenge didn't test X" — so a gap reads as a property of the challenge, not a
weakness in the candidate.
`scope_exclusions` at the top level is the same idea for the assessment as a whole.
## Strengths, flags and interview questions
`strengths` and `flags` are short human-readable statements about the work. `flags` are
things to probe, **not** disqualifiers.
`interview_questions` is the field most worth wiring up:
```json theme={null}
[
{
"question": "Walk me through how you decided the eviction order.",
"probes": ["What did you try first?", "How did you confirm it was right?"]
}
]
```
These are generated from what the candidate actually did, so they give an interviewer
specific ground to stand on instead of generic questions. Paste them into your interview
template.
## Integrity status
`integrity_status` is `clear` or `flagged`.
`flagged` means "worth a human look", not "cheated". We deliberately do **not** return
which checks fired or how many — a precise list is exactly what makes a system gameable,
and an accusation you can't fully explain is worse than none.
Treat `flagged` as routing: send it to a person. Never auto-reject on it.
## Facts
`facts` is raw context, no judgement attached:
| Field | Meaning |
| ------------------------------------------------- | ------------------------------------------------- |
| `time_used_minutes` / `time_allotted_minutes` | Working time against the limit. |
| `budget_used_dollars` / `budget_allotted_dollars` | AI spend against the cap. |
| `ai_authored_pct` | Share of final code authored by the assistant. |
| `churn_pct` | Share of written code later rewritten or removed. |
| `ai_exchange_count` | Number of assistant exchanges. |
There is no "good" value for any of these. High `ai_authored_pct` with a high ship score
is effective delegation; the same figure with a low score is unchecked copying. The
interpretation is already in the pillar scores and the summary — show facts as context,
not as a scorecard.
## Ranking
`pipeline_rank` is `{ rank, total }` among your own evaluated submissions for that
assessment — "3rd of 14". `null` when there's nothing to compare against.
`percentile` is **omitted entirely** until there's a large enough comparison population to
make one honest. Don't build a UI that assumes the key exists; check for it.
## Per-task breakdown
For multi-task challenges, `tasks` reports each task's key, title, kind, time limit, and
budget used against allotted. Empty for single-task challenges.