What makes an NBA backtest unrealistic?

The usual causes are features that use information unavailable at bet time, such as box score minutes or ratings computed later, bet prices taken from the close or from the best book at every moment, ignored stake limits, and testing many configurations then reporting the best. Each one inflates results, and several often appear together in the same backtest.

Should a backtest use closing odds or earlier odds?

Use the price that would have been available at the strategy's decision time, from books that could actually be used, and keep the closing price as a separate benchmark. Using the close as the bet price misstates what an earlier strategy would have received and makes closing line value zero by construction, which removes one of the most useful diagnostics.

How should a backtest engine accept outside models?

Through a small interface: a fit call that receives only rows whose outcomes were known before a cutoff, and a predict call that receives a point-in-time feature snapshot and returns probabilities. The engine then owns prices, bet selection, staking, settlement and metrics, so every model is tested under the same rules and harness bugs are fixed once.

How can the engine itself be tested?

Run the market's no-vig probabilities through it as if they were a model, and expect results near the negative of the margin. Shuffle outcomes and confirm performance collapses. Inject a deliberate leak and confirm timestamp checks reject it. If the market-as-model test looks profitable, the problem is in the harness, not in any model being tested.

How to Design a Backtest Engine for NBA Betting Models

A sound NBA backtest engine separates the model from the harness. It builds every decision from point-in-time data, prices bets at odds actually available at decision time, trains walk-forward, applies explicit selection, staking and settlement rules, and reports calibration, closing line value and drawdowns alongside return, so any plugged-in model is tested under identical, honest conditions.

What a backtest engine is for

A backtest engine is the harness around a model, not the model. The model answers one question: given what was known at a moment, what is the probability of each outcome. The engine handles everything else: which data the model is allowed to see, which price a bet would have been struck at, which bets get placed, how much is staked, how each bet settles, and what the results mean.

Keeping those jobs separate is what makes a bring-your-own-model design possible. If the engine owns time, prices, staking and settlement, then any model that respects one interface can be tested under identical conditions, and two models can be compared without either one quietly getting easier rules.

A minimal interface

The cleanest contract is small. The engine calls fit(training_rows) with only rows whose outcomes were known before a cutoff, then calls predict(snapshot) for each decision point, where the snapshot contains features as of that timestamp and nothing later. The model returns probabilities, not picks. The engine decides whether a probability becomes a bet. A model that returns picks cannot be checked for calibration, and calibration is one of the few things a backtest can measure reliably.

Point-in-time data is the core design decision

Every row the engine touches needs two timestamps: when the event it describes happened, and when that information became available. A game result is available after the final buzzer. A player's season average as of a date is available only if it was computed from games before that date. An injury designation is available at the moment the report was published, not the moment the game started.

Store features with an available_at field and build each decision snapshot with an as-of join: for every feature, take the latest value whose available_at is at or before the decision time.

Decide the decision time explicitly. A model meant to bet the morning of a game should be tested on morning snapshots of both features and prices. A model meant to bet shortly before tip-off needs snapshots from then. Mixing the two, for example morning features with near-close prices, produces results that match no strategy anyone could run.

Closing prices versus available prices

The price a backtest assumes for each bet changes the answer more than most modelling choices do.

Closing prices as the bet price are convenient because they are the easiest historical odds to find. They are also the sharpest prices of the day, so a model that shows value against them is being tested against the hardest benchmark. But if the strategy would actually bet earlier, closing prices misstate what it would have received, and using the close as both the bet price and the benchmark makes closing line value zero by construction.

Available prices at decision time are the honest choice. They require timestamped odds snapshots, from books the bettor could actually use, at the moment the strategy would act. Record the closing price separately as a benchmark, so the engine can report both profit at the price taken and the gap between that price and the close.

Best price across many books is the trap. Picking the best available number across every book at every moment assumes access, speed and limits that rarely exist together.

NBA-specific leakage to design against

Basketball has a few leakage paths that a generic engine will not catch on its own.

Availability news. Star availability moves NBA lines sharply, and injury designations change through the day. A feature like "player X played" is known only after tip-off. Use the designation as published at decision time, with its publication timestamp, never the box score.

Minutes and rotations. Projected minutes built from the game's actual minutes leak the outcome. Use projections or rolling averages computed from prior games only.

Ratings updated after the game. Team ratings, adjusted efficiency figures and Elo-style numbers scraped from a site today reflect every game through today. Recompute them historically or snapshot them daily.

Schedule features are safe. Rest days, back-to-backs and travel are known in advance, which makes them clean inputs, as long as rescheduled games are handled with the schedule as it stood at the time.

Walk-forward training

The engine, not the user, should control the training window. A walk-forward loop trains on everything before a cutoff, predicts the next block, advances the cutoff, and repeats. Random train-test splits shuffle future games into training and are not valid for time-ordered data. The open-source sports-betting Python library, for example, uses a time-series split in its documented backtest for this reason, though its published examples use soccer data rather than NBA.

Keep a final holdout season that no configuration ever touches until the design is frozen.

Bet selection and sizing

Separate the rule that turns probabilities into bets from the rule that sizes them, and make both configurable.

Selection. Convert the price to an implied probability, compare it with the model, and bet when the difference clears a threshold. Remove the bookmaker margin from the benchmark before comparing, otherwise every comparison is biased toward no bet.

Sizing. Flat stakes are the clearest way to evaluate a model, because results are not distorted by the bankroll path. Fractional Kelly shows what a growth-oriented strategy would have experienced, including drawdowns. Report both.

Correlation. A spread bet, a moneyline bet and a player prop on the same game are not independent. The engine should either cap exposure per game or report results at the game level as well as the bet level.

Limits and settlement. Apply maximum stakes per market and settle using the book's rules for pushes, overtime and voided markets. Check each book's house rules rather than assuming one convention.

Metrics worth reporting

A single return figure is the least informative output a backtest can produce. A useful report includes:

Log loss or Brier score for the model and for the no-vig market on the same games.
Calibration by probability bucket, so overconfidence shows up before it shows up as losses.
Closing line value, the average gap between the price taken and the closing price.
Return with a confidence interval, plus the number of bets behind it.
Maximum drawdown and longest losing run under each staking rule.
Segment results by market, season phase (regular season, post-deadline, playoffs) and time before tip-off.

Log every configuration that was tried, not just the best one. Testing many strategies and reporting the winner inflates apparent results, and a run log is the only defence.

Sanity tests for the engine itself

Before trusting any model result, test the harness. Feed it the market's own no-vig probabilities as the model: the expected result is roughly the negative of the margin, and anything materially better suggests a pricing or settlement bug. Shuffle outcomes and rerun: performance should collapse toward the margin. Add a deliberate leak, such as the final score as a feature, and confirm the engine's timestamp checks reject it. An engine that passes these three tests is one whose positive results are worth investigating.

This page describes testing methodology and is not betting advice.

Frequently asked questions

What makes an NBA backtest unrealistic?: The usual causes are features that use information unavailable at bet time, such as box score minutes or ratings computed later, bet prices taken from the close or from the best book at every moment, ignored stake limits, and testing many configurations then reporting the best. Each one inflates results, and several often appear together in the same backtest.
Should a backtest use closing odds or earlier odds?: Use the price that would have been available at the strategy's decision time, from books that could actually be used, and keep the closing price as a separate benchmark. Using the close as the bet price misstates what an earlier strategy would have received and makes closing line value zero by construction, which removes one of the most useful diagnostics.
How should a backtest engine accept outside models?: Through a small interface: a fit call that receives only rows whose outcomes were known before a cutoff, and a predict call that receives a point-in-time feature snapshot and returns probabilities. The engine then owns prices, bet selection, staking, settlement and metrics, so every model is tested under the same rules and harness bugs are fixed once.
How can the engine itself be tested?: Run the market's no-vig probabilities through it as if they were a model, and expect results near the negative of the margin. Shuffle outcomes and confirm performance collapses. Inject a deliberate leak and confirm timestamp checks reject it. If the market-as-model test looks profitable, the problem is in the harness, not in any model being tested.