Probe shapes
For each table or table function we run three probe variants:
Each variant is run **3 trials** (median latency reported).
Required filters
Catalog declares `required_filters` per table. We only run probes where those filters are present.
Latency measurement
- Wall-clock from `tools/call` send to response receive.
- Discarded the first trial (warm-up) when computing stats.
- `init` round-trip is logged once per `coral mcp-stdio` session, not per query.
Verdict rules
- **pass** — all 3 variants return data on ≥ 2 of 3 trials, 0 `isError`, response shape matches catalog (≥ 80% of columns).
- **partial** — some pass, some fail; recoverable `isError` (auth, rate-limit).
- **fail** — all variants `isError=true`, response shape wildly off, or persistent failure across fresh sessions.
Edge cases we always probe
1. Empty input on a `WHERE x = ''` filter.
2. Very long input (1k+ char string).
3. Unicode / emoji.
4. Owner with hundreds of repos (`opencode`, `microsoft`).
5. Owner with almost no data (single-commit user).
6. `LIMIT 1` and `LIMIT 1000` — to detect default caps.
7. Filter on a column that exists in the catalog but not in the response.
What this methodology does NOT test
- **Write paths** — Coral ships read-only SQL. Mutating endpoints are out of scope.
- **Auth boundaries** — we use whatever `coral mcp-stdio` is configured with.
- **Cross-source joins** — possible but explicitly out of scope per report (separate page).
Re-running the full sweep
Daily at 04:00 UTC the smoke workflow re-runs **1 trial per surface**. Full sweep (3 trials × 3 variants) runs on demand:
python3 scripts/probe_table.py --source github --table <t> --runs 3
python3 scripts/probe_function.py --source github --function <f> --arg q=opencode --runs 3