How Analytics Is Applied to Market Questions

Estimate independently. Some process produces a probability without reference to the market price you intend to compare against. A model over historical performance, a simulation, a structural model of the sport. What matters is independence, because an estimate derived from the price cannot say anything about the price.

De-margin the market. Convert posted odds to implied probabilities, then remove the margin using a stated method. The result is the market's estimate of likelihood rather than its price, and it is the only thing your estimate can be meaningfully compared against.

Compare. The difference between your estimate and the de-margined market estimate is the quantity of interest. It is a difference between two estimates, one of which is produced by a large number of participants with money at stake, so the prior should be that the market is right and you are wrong.

Evaluate calibration. Over many observations, did the things you called forty percent occur about forty percent of the time. This is the property that determines whether the estimates mean anything, and it is measurable long before outcomes accumulate enough to say anything about performance.

That is the whole method. The difficulty is entirely in doing step one honestly and step four rigorously.

Circularity. Using market-derived features to produce an estimate and then comparing that estimate against the market. The result reflects the price and the arithmetic connecting them, and any apparent divergence is a modelling artefact. This is extremely common because market data is the most available and cleanest data in the domain.

Comparing against raw implied probabilities. Skipping de-margining means every comparison is offset by the margin, so a model that is exactly as good as the market appears systematically worse. Teams have abandoned sound models over this.

Leakage in backtests. Several specific forms. Using data recorded after the event when estimating before it. Splitting randomly rather than chronologically, so future observations train a model tested on the past. Using closing prices in a model evaluated against closing prices. Rebuilding features with information that was not available at the decision time.

Each is well understood and each is easy to introduce accidentally, which is why the discipline is to reconstruct the exact information state at the moment of the estimate rather than to filter a modern dataset.

Selection during evaluation. Reporting only segments where results were good, or removing observations that look anomalous. If exclusions were not defined before looking, they are results rather than methodology.

Concluding from small samples. Variance dominates over short horizons, so a sequence of good or bad results is uninformative about the method. This is the reason calibration and price movement are used as leading indicators at all.

The market is a strong benchmark

The estimate, the timestamp and the information state. What was known at the moment the estimate was made, so it can be reconstructed later and so leakage can be ruled out rather than assumed away.

The market price used, its source and its time. Comparing your estimate at one moment against a price from another measures the interval as much as anything else.

The de-margining method. Different methods give different answers from identical inputs, and a later change should not silently alter historical comparisons.

The model version. Results produced by different model versions are not comparable, and lumping them together is one of the more common ways a track record becomes meaningless.

Every observation, including the ones you disliked. Exclusions applied after seeing results are the single most effective way to convince yourself of something untrue.

With those recorded, calibration can be measured, leakage can be audited, and a result can be reproduced. Without them the analysis is a story about numbers rather than a measurement, and the difference only becomes apparent when someone tries to rely on it.

This page describes analytical method and is not betting advice.