warming up the server…
first visit may take up to 30s · subsequent visits are cached
How a leaf photo becomes a diagnosis. The four-module architecture, layer by layer, with calibration metrics, a live request log, and the data behind every number on this page. Take the tour from the top, or jump straight to any module on the left.
APIN is a plant disease diagnosis system that runs four separate models in concert. The first, the router, looks at an incoming leaf image and decides which crop it is. The other three are specialists, each trained to recognise the diseases of one crop family: tomato, the okra/brassica pair (which share a single ensemble), and chilli (where the disease specialist has not been built yet, so the system reports an honest "router rejected" instead of guessing).
The architecture is split this way for a real reason: a single monolithic model trained to identify every disease of every crop gets dragged down by the worst-represented class. By specialising, each specialist sees only its own crop's distribution, and the router takes the responsibility for routing-versus-specialising entirely off the specialists' plate. The cost of the split is the router itself, which adds one inexpensive forward pass per request.
Below, each module gets its own section: architecture, data, metrics, calibration, a live test you can run, and an honest interpretation. The right rail tracks where you are. The live request log on the next page shows real traffic this server is handling. Press ? at any time for keyboard shortcuts; g opens a glossary panel from the right.
Eight steps separate an uploaded JPEG from a returned diagnosis. Every interactive playground on the /docs page is a real walk-through of this same sequence.
Calibration is what makes the confidence number on the response actually mean something. A vanilla softmax can report "87% confident" while being right anywhere from 60% to 95% of the time. After the calibration step here, an 87% prediction is right about 86% to 88% of the time. The "Calibration deep dive" section toward the end of this page covers why and how.
Below is every request this server has handled in the last few minutes, in real time. The log skips static assets (the favicon, logo, CSS, JavaScript) and the log endpoint's own self-polls. It carries path templates only · never request bodies, never authorization headers, never user identifiers. It is safe to look at and useful for getting a feel for what the system is actually doing right now.
The detailed module sections (architecture, data, metrics, live tests, interpretation) ship across the v1 build. Below are the section banners with each module's at-a-glance summary so the navigation, status indicators, and jump-targets are already in place. Click any banner to land here from the left navigation.
The router is the first stage every request hits. Its job is small and well-defined: given a leaf image, decide which of four crops it is (tomato, okra, brassica, chilli) and pass that decision plus a confidence score to the rest of the pipeline. Everything downstream depends on this being right.
Architecturally, the router is a frozen DINOv2 vision transformer feeding a single 4-way linear head. The backbone weights do not change during router training; only the head learns. This is deliberate. DINOv2 was pre-trained self- supervised on hundreds of millions of natural images and its feature space already separates "leafy green plant" from "rest of the world" cleanly. Asking it to also learn "tomato vs okra vs brassica vs chilli" only takes a handful of fine-tuning epochs on the linear head, on top of a ~22-million-parameter backbone (DINOv2 ViT-Small with 4 register tokens, 22.06 M params) that stays fixed.
The frozen-backbone choice has a second benefit: the router never overfits to the training distribution. Because the backbone weights are immutable, the only way the router can adapt is by re-weighting the existing DINOv2 features. That keeps the four-way decision boundary smooth and robust to lighting changes, camera angles, and the kind of domain shift between lab-collected and field-collected photographs that plagues most plant-disease classifiers.
On the confidence side: the router outputs a four-element softmax vector. If the maximum element is at least 0.40, the request is routed to that crop's specialist; if below, the system falls back to the APIN okra/brassica ensemble as a best-effort guess. The 0.40 threshold was chosen on a held-out validation split as the value that minimises both wrong-crop routing and false fallbacks; the confidence histogram on the Metrics tab visualises why.
Per-crop image counts in the router's training pool. The
train, val, and test columns
come from data/metadata/source_map.csv (the split labels
recorded at preparation time). The pool on disk column counts
the same crop folder under
data/specialist/router/cleaned/ (the authoritative pool
the router actually saw at training); discrepancies arise when raw
sources were merged into the on-disk pool but not enumerated in
source_map.
Every source the router was trained on, with the licence and citation notice. Sorted alphabetically by source name. Click any column header to re-sort; type in the filter (when shown) to narrow the list.
test_set.integrity.
Three charts characterise router behaviour on the locked test split: per-crop accuracy, the confidence histogram with the 0.40 cutoff annotated, and the four-by-four confusion matrix. All three are computed offline by a dedicated extraction pass that runs the actual router on the test images; that script is queued for tracked with the live-metrics work below.
Drop a leaf photograph below to see the router's verdict on it. The
request goes to
POST /api/predict/quick with the
Bearer key from
localStorage["apin-docs-key"] (the same key picker
/docs uses; visit there first if you have not set one).
The /quick endpoint is a thin, fast probe: the response
carries router_crop and router_confidence
plus a short top-K of likely diseases, but does not run the full
ensemble or compute heatmaps. Use it for routing-only inspection.
The tomato and chilli drop zones below exercise
/api/predict/full for the complete diagnosis.
The router is the most consequential model in the pipeline, because every other model only sees the images the router has already labelled for them. A router mistake on the crop is irrecoverable: the tomato specialist looking at a brassica leaf will produce a confident but wrong tomato diagnosis. The 0.40 confidence floor is the only line of defence against that failure mode, and it is set conservatively for exactly this reason.
Where the router fails most often: leaves photographed against a noisy background (other plants, soil, hands), severely yellowed leaves where the crop signal is dominated by chlorosis, and pre-emergent seedlings where the leaf shape is still ambiguous. Field-collected photographs with these properties make up most of the residual error.
Where the router is reliable: clean isolated-leaf shots in good light, mature leaves with the species-typical shape and venation intact, and brassica varieties that have started to head (a brassica head is unambiguous to the model). For those, the router's softmax clears the 0.40 routing floor by a wide margin in spot-checks; the full per-crop distribution arrives once the router-metrics extraction runs.
The tomato specialist is not a single model; it is a two-branch ensemble whose outputs are combined in probability space. Branch one is V3: a DINOv2-Small backbone with low-rank adaptation and FiLM conditioning, originally trained for 10 classes and re-used here for the deployed 6. Branch two is a separate single-pass LoRA run on a DINOv2-Reg-Base backbone, trained directly on 6 classes, epoch 13. Both branches see the same input image; they vote on the class with their respective probability outputs.
The fusion is 50/50 probability averaging, but with one critical
asymmetry: V3's
logits are divided by a
temperature of 0.5 before
softmax (sharpening V3
by a factor of two), while the LoRA branch's logits are softmaxed at
T=1.0. The effect is that V3 dominates on high-confidence cases
(its sharpened argmax overrides the unsharpened LoRA argmax on
confident frames; specific magnitudes vary per image and surface
once the metrics extraction runs), while on uncertain
cases the two contribute comparably. This asymmetric sharpening
is deliberate; the validation lift over V3-alone was measured
with exactly this setup (see Decision 56 in
ladi_decisions.md).
Input preprocessing differs per branch and also matters. V3 takes a
224x224 stretch-resize plus LAB-CLAHE on the L channel plus ImageNet
normalisation. LoRA takes an 800-pixel cap, letterboxed to 392 with
padding value 114, then LAB-CLAHE plus normalisation. The end-to-end
forward pass at inference time runs both pipelines and emits a
single fused probability distribution over six classes:
tomato_late_blight, tomato_septoria_leaf_spot,
tomato_foliar_spot, tomato_mosaic_virus,
tomato_yellow_leaf_curl_virus, and
tomato_healthy.
Six conditions the tomato specialist diagnoses, listed by pathogen
type. Latin name, training count, and example file references are
pulled from
report_figures/fig_7_4_tomato_classes_v2_provenance.json
(the authoritative class metadata, manually verified against the
original disease atlases).
The full per-class metrics table (precision / recall / F1 / ECE) on the locked tomato test split requires running the deployed V3 + SP-LoRA pipeline on every test image, which is queued for a separate evaluation pass. Until that runs, we surface a different, less misleading view: the PlantDoc cross-domain check below.
PlantDoc is a public dataset of plant-disease photographs collected independently from the training pool. Numbers here measure how well the model transfers from its training distribution to those out-of- domain photos. They are not the pipeline's headline numbers; they are a stress test that exposes domain-shift behaviour.
The 6×6
confusion matrix
on the locked tomato test split (n=104 · the
final_val set held out throughout v1 + v2
development). Row = ground truth · column = prediction · diagonal
cells = correct classifications · off-diagonal heat = mistake severity.
Three branches shown: V3 alone, SP-LoRA epoch 13 alone, and the
deployed fused ensemble (0.5 × V3@T=0.5 + 0.5 × SP-LoRA@T=1.0).
The tomato pipeline uses temperature scaling on each branch separately, then combines them. V3 runs at T=0.5 (sharpened by 2x). LoRA runs at T=1.0 (left alone because its calibration on the held-out split was unreliable).
The tier thresholds (in
phase3_tier_thresholds_sp_lora_ep13.json and
phase3_tier_thresholds_v3_tomato.json) were built from
already-temperature-calibrated probabilities; the pipeline must not
double-apply temperature. This is a known foot-gun the production
code guards against.
The full reliability diagram + ECE breakdown for the deployed ensemble ships alongside the per-class metrics extraction. The tier thresholds and prototype-bank inventory below are wired to the Phase-D extractor output (real, where the file exists).
decision boundaries on the fused 6-class softmax · the deployed router cascades from Tier-1A down to lower tiers, abstaining below the floor.
| tier | cut | routing behaviour |
|---|
source: v1 router design doc · static thresholds
CLS-token prototypes per class · used by the OOD gate and the Similar-Leaves feature in the Forward Pass.
PCA-explained variance per prototype is pending an extractor pass that runs PCA on the bank embeddings.
calibration_warning to the response so the
UI can downgrade the displayed confidence tier. The rail was
introduced because YLCV had very thin support on the internal
validation split used during pipeline development (see
scripts/ladi_net/tomato_pipeline.py docstring,
PDA condition B4); the per-class numbers shown elsewhere on this
page come from PlantDoc cross-domain checks and are not the same
cohort. The headline tomato test split metrics, when extracted,
will be revisited once that extraction lands.
Drop a tomato leaf photograph. The request goes to
POST /api/predict/full with the Bearer key from
localStorage["apin-docs-key"] (same as /docs). The
response includes the fused 6-class probability distribution,
the predicted tier, and a Grad-CAM heatmap reference. Routing is
decided by the router; see the note below for how to interpret a
non-tomato result.
force_pipeline=tomato override is in the v1.1 API backlog;
see /docs#predict-full for current
endpoint behaviour.
Tomato is one of the easier crop pipelines to reason about because the disease set is well-known, the lab-grown training data is plentiful (PlantVillage alone contributes thousands of clean shots), and most of the symptoms are visually striking. Where the pipeline struggles is well-bounded:
Foliar spot vs septoria. Both produce small dark spots on leaf surfaces; on a low-resolution field photo they can be visually indistinguishable, and the model occasionally confuses them. The PlantDoc cross-domain check on the Metrics tab quantifies this. Practically, if the prediction is one of these two and confidence is below 0.7, treat the answer as "small-spot disease, probably foliar or septoria" rather than relying on the specific label.
Yellow leaf curl virus. The held-out test split
carried only n=2 YLCV images, and the test F1 there was 0.0. The
pipeline still emits YLCV predictions on plausibly-curl-y leaves
but adds a
calibration_warning field; client code should downgrade
confidence on those predictions until more YLCV training data is
collected.
Lab-only training is the largest single risk. The training pool skews heavily toward PlantVillage's clean lab shots. PlantDoc field photographs of the same diseases show a 10 to 55 F1 drop, with the worst classes (mosaic virus, early blight, leaf mold, bacterial spot) all clustering in the 35-55 band; the Metrics tab shows the per-class gaps. The okra/brassica ensemble has the same issue and addresses it with its 4-signal stacking design; the tomato pipeline does not yet have an equivalent.
APIN's DINOv2-backed crop router already recognises chilli leaves; it was trained on five chilli source datasets (Bangladesh, Karnataka, Mendeley, iNaturalist, anthracnose- focused Kaggle) and learns the difference between a chilli leaf and the other three crops the system handles.
What's missing is the second step: a chilli disease
specialist that takes a chilli leaf and assigns it a disease class.
That model has not been built. Until it ships, every chilli frame that
reaches /api/predict/full with the router agreeing it is
chilli returns a clean rejection envelope and the request stops.
Below is the exact response shape an integrator currently has to
handle. The envelope follows APIN's standard 8-key error format so
client code does not need a special case for chilli; it only needs to
recognise the error.code = "router_rejected" sentinel.
{
"api_version": "1.0",
"request_id": "req_a4b9c2e7d1f803a5",
"endpoint": "/api/predict/full",
"ok": false,
"processed_at": "2026-05-23T11:04:12.337Z",
"processing_time_ms": 82,
"status_code": 409,
"error": {
"code": "router_rejected",
"message": "Detected crop: chilli. This is outside the deployed specialist set.",
"hint": "A chilli specialist is on the roadmap; for now, route the request to a human agronomist or use the fallback APIN ensemble at your own risk.",
"router_crop": "chilli",
"router_confidence": 0.94
}
}
Building a chilli disease specialist with the same standard as the tomato pipeline or the APIN okra/brassica ensemble needs five things, and all five are still open work:
diagnosis/diagnosis_lookup.json.
ROUTER_REJECTED branch in the chilli code path with the
specialist call, gated by the same confidence floor as the tomato
path.
The first item is the bottleneck. Disease-labelled chilli photographs at scale do not exist as a clean public dataset; the candidate sources either lack disease labels, lack field photographs, or both.
Counts below come from
scripts/apin_v2/extract_pipeline_atlas_chilli.py, which
reads file paths from data/metadata/source_map.csv and
counts the staged specialist folders on disk. No hand-typed numbers;
every value below traces back to a file count.
Disease-bucketed chilli images that have been cleaned and parked on disk for the eventual specialist. The deployed system never reads from these folders; they exist as raw material for whoever builds the specialist next.
Six representative filenames the router has seen during training. The
filename prefix encodes the source bucket
(multi_, orig_, src*, source*).
Concrete check-list of the steps to move chilli from router-only to full specialist parity with tomato. Boxes that are ticked are already done in the current build; the rest are open work.
scripts/apin_v2/extract_router_metrics.py), so the
number is deliberately omitted here rather than estimated.diagnosis/diagnosis_lookup.json.ROUTER_REJECTED with the specialist call
in the chilli code path of apin_server.py.
Upload a chilli leaf to see the actual response the system returns
today. This drop zone calls
POST /api/predict/full with the
Bearer key
you have set in
localStorage["apin-docs-key"] (the same key picker
/docs uses). The response is rendered raw; the focus is
on the
ROUTER_REJECTED envelope shape, not on softmax bars.
APIN is a 4-signal stacking ensemble. Four independent backbones each emit a 9-class probability over the ensemble's shared okra+brassica label space, and a learned stacking MLP fuses those 36 probabilities into one 9-class output. Per-class temperature scaling and conformal prediction sets wrap the head.
Each backbone has a different inductive bias: DINOv3-ConvNeXt is convolutional and texture-led, EfficientNet-B0 is small but heavily tuned, DINOv2-ViT-S is self-supervised and shape-led, PSV is engineered colour/vein features. When one backbone is confidently wrong on an OOD image, the others usually disagree. The stacking MLP learns where to trust whom.
A tiny MLP (36 inputs → 64 hidden → 9 outputs) reads all four probability vectors and produces a fused output. It's the only trainable component downstream of the frozen signals, and it was fit on the held-out calibration split; never on the same images used to train the four signals. That separation is what lets temperature scaling and conformal prediction calibrate honestly.
the diagram is wired to the live /api/info backbone
manifest; signal pills tint by the latest reliability score
Each of the four backbones contributes a 9-class softmax to the
stacking MLP. Below: who they are, what they're good at, and
their empirical reliability (mean cell of their row in the live
per-signal reliability matrix).
Numbers are live from /api/info.reliability_matrix.
Nine diseases (and the two healthy classes) the APIN ensemble diagnoses. Per-class F1 and support come from the locked test split (n=920 field photos). Click any card for an inline summary of its hardest confusable.
loading…
| class | precision | recall | F1 |
|---|
how the ensemble's confidence shifts when paired with v1 vs v2
router gates · numbers from apin_vs_baselines_final_*.json
The full 9×9 confusion matrix on the locked test split
(n=920 · val_and_soup + is_field_photo). Row = true
class, column = predicted class. Diagonal = correct; off-diagonal
heat = mistake magnitude.
rows = true class · cols = predicted class · darker = more samples · diagonal in green
APIN's stacking-MLP output runs through per-class temperature scaling followed by split-conformal calibration. The two knobs together turn raw 9-class probabilities into calibrated confidences and conformal prediction sets that the API returns to the client.
The full per-model comparison (APIN vs tomato vs router) · with the live α slider, classic reliability bins, miscalibration heatmap and OOD scatter · lives in the page-level Calibration deep dive section.
Drop an okra or brassica leaf. The request goes to
POST /api/predict/full with the Bearer key from
localStorage["apin-docs-key"] (set it below if you
haven't yet). The response carries the fused 9-class probability
distribution, the temperature-calibrated top probability, the
conformal prediction set, and a Grad-CAM heatmap reference.
APIN is the most reliable specialist on this page · 93.2% accuracy on 920 held-out field photos, with a macro F1 of 0.893. Where it falters is predictable, and the four-signal design is exactly what keeps the failures bounded.
Okra_enation is the hardest class. Its recall is
0.558 (5 misclassified as okra_powdery_mildew, 6 as
brassica_alternaria · see the
Confusion matrix tab).
Enation symptoms · vein swelling, leaf-curl bumps · overlap with
powdery-mildew's leaf curl, and the cross-crop confusion with
brassica_alternaria is a real OOD risk. When the model predicts
okra_enation with confidence below 0.7, treat it as
"curl-y / bumpy leaf, probably enation but verify."
The stacking MLP is doing real work. Look at the per-signal reliability matrix · PSV alone has ECE 0.55 (wildly overconfident), but its mistakes are systematic enough that the stacking MLP learns to downweight it. The fused ensemble's ECE after temperature scaling is 0.084, an 8.6× improvement.
The conformal set is the safety rail. At the default α=0.05 (95% target coverage), the empirical coverage is 95.2% · the conformal threshold is essentially exact. 42.9% of images get a size-1 set (a confident single-class call); 54.2% get size-2 (the model says "it's one of these two"); 1.4% get empty sets ("I don't know, refuse"). That 1.4% empty rate is the model being honestly humble · drag the α slider to see how that distribution shifts.
Loading hand-drafted narration…
Loading…
An 87% confidence number should mean "right 87 times out of 100" on the calibration distribution. Without intervention, modern ensembles report 87% but are right 75% of the time. This section shows how APIN gets that number honest, and what we measure to prove it.
per-class scalar that divides each logit before softmax
hover any class to highlight it on the right
Per-image pre-temperature ("raw") vs post-temperature softmax for the three pool images needs a fresh extraction run to surface. Once the extractor lands, this panel will also show:
Negative log-likelihood minimisation on the held-out calibration split. The single-parameter-per-class objective is convex and was solved with LBFGS. Classes the ensemble was over-confident on get T > 1 (flattens the softmax); under-confident classes get T < 1 (sharpens). The same procedure was used for tomato's 6-class output and the router's 4-class output.
per-class thresholds that yield calibrated set-coverage
calibrated for ~90% coverage on the held-out split
CLS-embedding PCA · in-dist vs rejected scatter
2D PCA of the per-image 9-class softmax vectors. The spec called for PCA of the 384-D CLS-token embeddings (which needs a backbone forward pass to produce); this softmax-vector projection is a data-grounded proxy that captures the same intuition: in-distribution images cluster at corners of the simplex (one class strongly predicted), high-entropy / OOD-like points spread toward the centroid (predictions split across classes).
point colour = predicted class · darker outline = misclassified · hover for class + entropy + max probability. PC1 and PC2 explain ~43% of variance in the 9-D softmax space.
predicted confidence vs empirical accuracy
each cell is the empirical reliability of one signal for one class (1.0 = perfect agreement with ground truth on the held-out split)
each panel is one backbone's confidence-vs-accuracy curve binned into 10 confidence brackets. Bars above the diagonal are over- confident; bars below are under-confident. PSV's bar in the high-confidence bin shows why its raw output needs a stacking MLP to be safe in production.
ECE is the area-weighted absolute gap; lower is better-calibrated. n=736 final_val field photos · per-signal post-softmax · pre-stacking-MLP.
which classes hide bad behaviour at which confidence bins
each cell holds the signed gap (mean confidence − accuracy) for images of that predicted class in that confidence bin. Red = the model is overconfident (says 0.9 but is right less often); blue = underconfident. Empty cells = no predictions in that bin. Hover for the exact confidence / accuracy pair.
cells with n < 5 predictions are dimmed (statistical noise). Hover any cell for n, mean confidence, accuracy, signed gap.
n=920 · each cell shows 1 − metric (lower is better) so the worst- calibrated classes are the darkest. okra_enation's recall gap of 0.44 dominates · its 5 → powdery_mildew + 6 → brassica_alternaria confusions (see M4 confusion matrix) account for the bulk of that miscalibration.
cell colour = quality gap to 1.0 · darker ochre = larger gap · green background = ≤ 0.05 gap (well-calibrated). Click a row to copy the class name.
What this page knows about the API right now. Polled every 5s. If no requests arrive for more than two minutes, the section dims to signal staleness.
last … requests, bucketed by minute (span: … min)
total in span: … reqs · busiest minute: …
last … requests
| endpoint | p50 | p95 | share |
|---|
last … requests
| endpoint | share | count |
|---|
When a user clicks the 👎 button on a prediction (visible on the inference page), the disagreement is logged to a feedback table. This view will show:
Every image that touched the model · where it came from, which split it ended up in, and a cryptographic guarantee that no test image appears in any training set. Numbers below are computed from the locked source map and recomputed every time it changes.
… images from … source families, allocated across … splits
hover any source or split to highlight its connections · click a split to filter the provenance table below
every source dataset that fed any split · sortable by any column
| source | family | license | year | crops | images ▾ | cite |
|---|---|---|---|---|---|---|
| TOTAL | … | |||||
pairwise check that no image path appears in two splits at once
Every record in data/metadata/source_map.csv carries an
image path. We build a set of paths per split, then count pairwise
intersections. A non-zero intersection between test and any
other split is a leak: the model would be evaluated on something it
had already seen.
A future hardening step is to replace path-identity with a SHA-256 hash of the pixel content, which catches cases where the same image was duplicated to a different filename. Path-identity is the stricter check we have today.
The verifier would print every offending path with its two splits and exit with code 1. The training driver checks the same set and refuses to load a corrupt split, so the leak would block both evaluation and training pipelines.
images dropped before any split (deduplication · quality gates)