Spatial Thinning of Occurrence Records in Python

Spatial thinning removes occurrence points that sit closer together than a chosen distance, so that no small patch of the study area contributes more records than a remote one. It is one of the first corrections applied during Presence-Only Data Preparation, the stage of the Species Distribution Modeling with MaxEnt workflow that turns opportunistic records into raster-aligned model input. Thinning attacks a specific artifact: dense clusters of presences around roads, trailheads, and research stations inflate the apparent environmental signal of whatever conditions happen to occur near access, and they leak neighbouring points across a random train/test split. This page covers the narrow engineering task of thinning an occurrence set two ways — a distance-based greedy filter and a faster grid-based reduction — and shows how to size the thinning distance defensibly.

When to Use Spatial Thinning

Thinning is the right tool when clustering is geometric — points piled up in space — rather than when survey effort itself must be modelled. It is complementary to, not a replacement for, the bias-surface approach in the sibling guide Handling Sampling Bias in Presence-Only Data. Use the table to pick a thinning strategy.

The distance-based filter is the standard choice for most forestry datasets of a few thousand points. The grid-based method trades exactness for speed on very large citizen-science pulls. The randomized-repeat strategy — the logic behind the widely cited spThin R package — runs the greedy filter many times with shuffled input order and keeps the run that retains the most points, because a single greedy pass depends on processing order.

Minimal Reproducible Example

Both approaches below operate on a GeoDataFrame of presence points. Distances are computed in metres, so the very first step is to enforce a projected CRS — thinning in degrees silently distorts distances by latitude.

import geopandas as gpd
import numpy as np
from scipy.spatial import cKDTree

occ = gpd.read_file("forest_occurrences.gpkg")

# Enforce a projected CRS so the thinning distance is in metres, not degrees.
if occ.crs is None or occ.crs.is_geographic:
    occ = occ.to_crs(occ.estimate_utm_crs())

coords = np.column_stack((occ.geometry.x, occ.geometry.y))


def thin_distance(coords, min_dist, seed=0):
    """Greedy distance-based thinning: retain points >= min_dist apart."""
    rng = np.random.default_rng(seed)
    order = rng.permutation(len(coords))     # randomise processing order
    tree = cKDTree(coords)
    keep = np.ones(len(coords), dtype=bool)

    for idx in order:
        if not keep[idx]:
            continue
        # Neighbours within min_dist (excluding the point itself) are dropped.
        neighbours = tree.query_ball_point(coords[idx], r=min_dist)
        for n in neighbours:
            if n != idx:
                keep[n] = False
    return keep


def thin_grid(coords, cell_size):
    """Grid-based thinning: keep the first point encountered per cell."""
    cells = np.floor(coords / cell_size).astype(np.int64)
    _, first_idx = np.unique(cells, axis=0, return_index=True)
    keep = np.zeros(len(coords), dtype=bool)
    keep[first_idx] = True
    return keep


# Distance-based, 1 km minimum separation
keep_d = thin_distance(coords, min_dist=1000.0)
thinned_d = occ.loc[keep_d].reset_index(drop=True)

# Grid-based, 1 km cells
keep_g = thin_grid(coords, cell_size=1000.0)
thinned_g = occ.loc[keep_g].reset_index(drop=True)

print(f"raw={len(occ)}  distance={len(thinned_d)}  grid={len(thinned_g)}")

To emulate the spThin randomized-repeat strategy, wrap thin_distance in a loop over several seeds and keep the mask that retains the most points:

def thin_repeat(coords, min_dist, iterations=20):
    best_keep, best_n = None, -1
    for seed in range(iterations):
        keep = thin_distance(coords, min_dist, seed=seed)
        if keep.sum() > best_n:
            best_keep, best_n = keep, keep.sum()
    return best_keep

keep_best = thin_repeat(coords, min_dist=1000.0, iterations=20)
thinned_best = occ.loc[keep_best].reset_index(drop=True)
thinned_best.to_file("occurrences_thinned.gpkg", driver="GPKG")

Parameter Reference

Expected Output and Verification

Thinning must produce a strictly smaller set in which no two retained points violate min_dist. Verify the guarantee directly with a nearest-neighbour query rather than trusting the loop — an off-by-one in the radius is easy to miss.

from scipy.spatial import cKDTree

pts = np.column_stack((thinned_best.geometry.x, thinned_best.geometry.y))
tree = cKDTree(pts)

# Query the two nearest neighbours; index 0 is the point itself (distance 0).
dists, _ = tree.query(pts, k=2)
nn = dists[:, 1]

assert len(thinned_best) <= len(occ), "Thinned set is not smaller"
assert nn.min() >= 1000.0 - 1e-6, f"Two points closer than min_dist: {nn.min():.1f} m"
print(f"Retained {len(thinned_best)} of {len(occ)}; min separation {nn.min():.0f} m")

A healthy thin retains somewhere between 30% and 80% of records for moderately clustered forestry data. If it retains under 10%, min_dist is too large relative to the extent and you are discarding real ecological signal; if it retains over 95%, the data were already sparse and thinning is unnecessary. The thinned set then flows on to the environmental sampling step and is evaluated for leakage in Model Validation & AUC Metrics, where spatially blocked folds confirm the thin actually reduced autocorrelation.

Common Pitfalls

• Thinning in geographic degrees. A min_dist of 0.01 degrees is roughly 1.1 km at the equator but far less near the poles, so distances distort with latitude. Always reproject to a projected CRS (estimate_utm_crs() or an equal-area projection) before thinning.
• Ignoring processing order. A single greedy pass keeps whichever point it reaches first, so the retained set depends on row order. Randomise the order and, for reproducibility, run several seeds and keep the largest result — this is exactly what spThin does.
• Confusing grid thinning with a distance guarantee. Grid-based thinning keeps one point per cell, but two points in adjacent cells can sit a few metres apart across the shared edge. If you need a hard minimum separation, use the distance-based filter.
• Setting min_dist from convenience, not grain. A distance untethered from the predictor resolution either over-thins (losing habitat variation the model could learn) or under-thins (leaving duplicate feature vectors). Anchor it to the predictor cell size and the survey’s realistic detection neighbourhood.

Frequently Asked Questions

Related

• Presence-Only Data Preparation — parent workflow covering CRS standardization, uncertainty filtering, thinning, and predictor masking.
• Handling Sampling Bias in Presence-Only Data — the complementary effort-bias correction that pairs with thinning.
• Model Validation & AUC Metrics — spatially blocked cross-validation that confirms thinning removed autocorrelation.
• Species Distribution Modeling with MaxEnt — the full modelling workflow this guide belongs to.

Up one level: Presence-Only Data Preparation · Species Distribution Modeling with MaxEnt