Evaluating SDM with the Continuous Boyce Index

The continuous Boyce index (CBI) is the evaluation metric built for the data species distribution models actually have: presences and background, with no verified absences. It is a specific technique within Model Validation & AUC Metrics, the stage of the Species Distribution Modeling with MaxEnt workflow that judges a fitted model before its suitability map reaches a land manager. AUC-ROC needs a well-defined absence class to be interpretable, which presence-only sampling cannot supply; the Boyce index sidesteps that by asking a purely presence-only question — do more presences fall in high-suitability bins than a random expectation would predict? This page implements CBI from scratch in numpy and scipy so you understand every moving part, and shows how to read its diagnostic curve.

When to Use the Boyce Index

CBI complements rather than replaces AUC. Use the table to see where each metric belongs.

Because CBI is threshold-independent and needs only presences plus the predicted suitability surface, it is the honest default for MaxEnt output. It pairs naturally with the spatially blocked folds in the parent guide: compute CBI on the held-out spatial fold, not on training presences, or it will report memorisation as skill. The presences it consumes should already be debiased and thinned in Presence-Only Data Preparation, and the model it scores comes from MaxEnt Model Training & Tuning.

Concept: The Predicted-to-Expected Ratio

CBI partitions the suitability range into bins (usually via a moving window) and, for each bin $b$, forms the predicted-to-expected ratio

$$F_b=\frac{P_b}{E_b}=\frac{p_b/{\sum }_k{p}_k}{a_b/{\sum }_k{a}_k}$$

where $p_b$ is the number of evaluation presences whose predicted suitability falls in bin $b$, and $a_b$ is the number of background/study-area cells in that same suitability bin. $P_b$ is the observed proportion of presences and $E_b$ the proportion expected if presences were placed at random across the landscape. A well-calibrated model produces $F_b$ that increases monotonically with suitability — presences accumulate faster than random as suitability rises. CBI is the Spearman rank correlation between $F_b$ and the bin suitability, ranging from -1 to +1: values near +1 mean good calibration, near 0 mean the model predicts no better than random, and negative values mean predictions are inverted.

Minimal Reproducible Example

The implementation uses a moving window over the suitability range. For each window it counts presence predictions and study-area (background) predictions, forms the P/E ratio, and finally takes the Spearman correlation of the ratio against the window midpoints.

import numpy as np
from scipy.stats import spearmanr


def continuous_boyce_index(pres_suit, bg_suit, window_width=0.1, n_windows=101):
    """
    Continuous Boyce index from predicted suitability values.

    pres_suit : suitability at evaluation (held-out) presence points, in [0, 1]
    bg_suit   : suitability at background / study-area cells, in [0, 1]
    """
    pres_suit = np.asarray(pres_suit, dtype=float)
    bg_suit = np.asarray(bg_suit, dtype=float)

    lo, hi = 0.0, 1.0
    starts = np.linspace(lo, hi - window_width, n_windows)
    mids, ratios = [], []
    n_pres, n_bg = len(pres_suit), len(bg_suit)

    for s in starts:
        e = s + window_width
        p_in = np.count_nonzero((pres_suit >= s) & (pres_suit < e))
        a_in = np.count_nonzero((bg_suit >= s) & (bg_suit < e))
        if a_in == 0:
            continue                      # skip empty environmental bins
        P = p_in / n_pres
        E = a_in / n_bg
        mids.append(s + window_width / 2.0)
        ratios.append(P / E)

    mids = np.asarray(mids)
    ratios = np.asarray(ratios)

    # CBI is the Spearman correlation between the P/E ratio and suitability.
    cbi, _ = spearmanr(mids, ratios)
    return cbi, mids, ratios


# Example with a well-calibrated synthetic model.
rng = np.random.default_rng(0)
bg = rng.uniform(0, 1, size=10_000)                       # study-area suitability
pres = np.clip(rng.beta(5, 2, size=800), 0, 1)            # presences skew high

cbi, mids, ratios = continuous_boyce_index(pres, bg, window_width=0.1)
print(f"Continuous Boyce index = {cbi:.3f}")

In practice pres_suit comes from sampling the MaxEnt suitability raster at held-out presence coordinates, and bg_suit from sampling it at the background points or a large random sample of study-area cells — both with rasterio.sample.

Parameter Reference

Expected Output and Verification

CBI is a single scalar in [-1, 1]. Verify both the value’s range and the shape of the diagnostic curve — a high CBI with a non-monotonic P/E curve signals a model that is well-ranked overall but miscalibrated in part of the suitability range.

import numpy as np

# 1. CBI is a valid correlation in [-1, 1].
assert -1.0 <= cbi <= 1.0, "CBI outside valid range"

# 2. A well-calibrated model has P/E rising with suitability and crossing 1.
#    Ratio < 1 at low suitability (fewer presences than random),
#    ratio > 1 at high suitability (more presences than random).
low = ratios[mids < 0.3].mean()
high = ratios[mids > 0.7].mean()
assert high > low, "P/E ratio does not increase with suitability"

print(f"CBI={cbi:.3f}  mean P/E low-suit={low:.2f}  high-suit={high:.2f}")

A CBI above roughly 0.8 indicates good calibration; 0.5–0.8 is moderate and worth inspecting; below 0.5, or negative, means the suitability surface is not tracking presence density and the model needs rework upstream in MaxEnt model training & tuning. Always plot the P/E curve alongside the scalar: the ideal is a smooth monotonic rise crossing $F_b=1$ once. A curve that dips at the top end reveals over-prediction in the most suitable habitat — a failure AUC would hide entirely.

Common Pitfalls

• Computing CBI on training presences. Like any performance metric, CBI inflates when evaluated on the data the model saw. Use held-out presences from a spatially blocked fold, consistent with the leakage-aware protocol in the parent Model Validation & AUC Metrics guide.
• Too few presences for the window width. Narrow windows over a small presence set leave many bins with zero presences, making the P/E ratio jumpy and the Spearman correlation unstable. Widen the window or reduce the number of windows when presences are scarce.
• Sparse background at the extremes. If almost no study-area cells fall in the top suitability bins, $E_b$ is estimated from a handful of cells and the ratio explodes. Skip bins with zero background cells (as the code does) and ensure the background sample is large.
• Reading CBI as a discrimination score. CBI measures calibration — whether presence density rises with predicted suitability — not ranking. A model can rank well (high AUC) yet be poorly calibrated (low CBI); report both, since they answer different questions.

Frequently Asked Questions

Related

• Model Validation & AUC Metrics — parent workflow covering spatial cross-validation, AUC, TSS, and threshold selection.
• MaxEnt Model Training & Tuning — where a low CBI sends you to revisit regularisation and feature classes.
• Presence-Only Data Preparation — supplies the debiased, thinned presences CBI evaluates.
• Species Distribution Modeling with MaxEnt — the full modelling workflow this guide belongs to.

Up one level: Model Validation & AUC Metrics · Species Distribution Modeling with MaxEnt