Scoreboard API Reference

The kreview.scoreboard module builds cross-evaluator comparison tables, ranking features by their classification performance across all trained models.

For conceptual explanations, see:

• Models & Metrics
• Dashboard Interpretation Guide

---

kreview.scoreboard

Cross-evaluator scoreboard on an explicit, versioned contract (#108): every field is sourced from a named artifact or explicitly unknown — never a fabricated default.

Docs: https://msk-access.github.io/kreview/scoreboard.html.md

load_selection_qc(directory)

Load per-evaluator selection-QC sidecars (<evaluator>_selection_qc.json).

The select stage writes these next to the selected matrices (matrices/selected/ in the published outdir); the Nextflow scoreboard process stages them flat into its work dir. Both layouts are searched.

Returns:

Type	Description
dict[str, dict]	Dict keyed by evaluator name. Unreadable files are logged and skipped
dict[str, dict]	(the affected evaluator then carries selection_method="unknown" plus a
dict[str, dict]	missing_fields entry — visible, never defaulted).

extract_evaluator_summary(evaluator, data, selection_qc=None)

Extract one scoreboard row from a merged model-results dict + QC sidecar.

THE single place that knows the model-results key shapes for summary fields (#108) — report_data consumes the resulting row rather than re-deriving these.

Contract: every SCOREBOARD_COLUMNS field is either sourced from a named key/artifact or explicitly unknown (NaN / None / "unknown") and listed in missing_fields. No plausible-looking defaults: absence must be distinguishable from a measured value (fail loud, not to 0.0 — invariant #3).

build_scoreboard(output_dir)

Aggregate all evaluator model results into a ranked scoreboard.

Scans output_dir for *_model_results.json (CPU+GPU, merged per evaluator) and *_selection_qc.json sidecars, and returns one row per evaluator with the fixed SCOREBOARD_COLUMNS schema, sorted by best_auc descending.

Parameters:

Name	Type	Description	Default
output_dir	Path	Directory containing the result JSONs (flat, as staged by the Nextflow scoreboard process, or a published outdir — QC sidecars are also found under matrices/selected/).	required

Returns:

Type	Description
DataFrame	DataFrame with the full contract schema. Empty DataFrame (with the
DataFrame	contract columns) if no results are found.