A profitable sample can come from an edge, favourable conditions or a rule chosen after seeing the result. Validation starts with a fixed hypothesis and separates development data from later checks. This guide shows what to record, which costs to include and how to keep failed cases in the analysis.

What an edge actually is

A trading edge is a hypothesis of positive expected net results under specified rules and conditions. A profitable historical sample is an observation, not the same claim. Evaluation records evidence, uncertainty and what remains unresolved; the sample mean does not reveal a permanent statistical surplus.

Specify instrument, feed, hours, signal, exits and size, including when information becomes available. A closed-candle signal does not automatically permit a fill at that same Close. Record exclusions for missing data or unworkable costs. Discretionary decisions can still be logged and measured; ambiguous rules instead make reproduction harder. Provider loss warnings do not identify causes or compare these methods.

How to tell whether your edge is positive

For a sample without zero outcomes, mean net result equals the positive-result fraction times average win minus the negative-result fraction times positive loss magnitude. Include zero outcomes in the total count when calculating the direct mean. Expectancy is not a complete risk assessment: also inspect losses, drawdowns, dependence and exposure.

Use an invented 100-trade sample: 55 gross wins of +120 EUR and 45 losses of −80 EUR give 55 × 120 − 45 × 80 = 3,000 EUR, or +30 per trade. Assign the same complete-cycle cost to every outcome, covering only charges not already in the gross result. Costs of 10, 25 and 35 EUR give sums +2,000, +500, −500 and means +20, +5, −5; the positive fraction stays 55%. Actual cost assumptions need the account/instrument specification; do not subtract spread or slippage again if already in fills.

Retain a separate gross illustration with 55% wins of +100 EUR and 45% losses of −120: the mean is +1 EUR. Costs greater than 1 EUR per trade, not any positive fee automatically, make it negative. The expected value formula separates size and frequency; none of these assigned outcomes forecasts a year’s income.

Where a retail edge realistically comes from

A plausible explanation is a hypothesis to test, not a result. Specialization, patience, a longer horizon or public-data interpretation do not establish an edge by themselves. Check whether another reader can reproduce the specified signals on the same available data before comparing their outcomes.

At cycle cost 25 EUR, the invented results are 55 × +95 and 45 × −105, mean +5. Sample standard deviation with denominator n − 1 is exactly 100 EUR. NIST’s mean-uncertainty formula uses s/√n; with independent observations and a stable distribution this gives 100/√100 = 10 EUR. That is an estimated standard error, not a maximum loss or a confidence interval by itself. The mean +5 is small relative to it; the observations do not certify positive future expectancy.

For a separate exact probability model, take four independent events, each ±100 EUR with probability 1/2, and copy each event 25 times. The 100-record mean equals (X1 + X2 + X3 + X4)/4. Its distribution standard deviation is 100/√4 = 50 EUR, not 100/√100 = 10. All 16 equally likely four-sign combinations reproduce this: more copies supply no new independent information. This does not estimate effective sample size for every FX strategy. Position weighting also matters, as asymmetric trading bets explains.

Overall ranking can reverse despite improvement in both groups

In this fictitious comparison, B has a higher positive-result fraction in both predefined groups:95% versus 90%,15% versus 10%. Yet its overall fraction is lower:22/40=55% versus A19/30≈63.33%. Group 2 supplies half of B's records but only one-third of A's.

Better in both groups, lower overallFictitious A has 18 positive records of 20 in group 1 and 1 of 10 in group 2. B has 19 of 20 and 3 of 20. B is higher in both groups but 55% overall versus approximately 63.33% for A, because it has a larger share of difficult group 2. Every bar uses the same percentage scale.Also compare group sharesGroup 1A:18/20=90%B:19/20=95%Group 2A:1/10=10%B:3/20=15%All recordsA:19/30≈63.33%B:22/40=55%Equal group weights:A50%,B55%
Every bar uses 0–100%. Numerators and denominators show actual counts in this fictitious comparison; colour identifies the variant.

Check sample composition before comparing overall percentages. With hypothetical equal group weights, A averages(90%+10%)/2=50%; B averages(95%+15%)/2=55%. This differs from the fraction in the actually collected samples. None demonstrates an edge: full net payoffs and uncertainty assessment are also required.

Assumptions: two explicitly fictitious historical samples. Groups 1 and 2 are defined before comparison and remain unchanged after results. Positive means net result above zero. Payoff sizes, observation independence and future-success probabilities are unspecified.

How to validate an edge before risking real money

Record a precise hypothesis, rule version, feeds, costs, information timing and evaluation criteria. Keep all development variants, including rejected ones, and freeze the chosen version before an untouched test. Manual or automated checks can each contain errors; neither a two-year period nor counting by hand proves an edge. See how to backtest a strategy.

Sample requirements depend on outcome variability, dependence between trades, regime coverage and how many strategies were tried. The familiar square-root relationship assumes an appropriate stable sampling model; no fixed count of thirty, one hundred or two hundred guarantees a reliable edge. Preserve unseen test data and compare historical, forward-demo and any small live results after costs, with uncertainty and drawdown limits reported.

Why so many "edges" are illusions

Bailey and co-authors describe how choosing the best result among many configurations can select historical noise. Record the number and results of all trials, not only the winner. A selected favorable period omits relevant conditions. Overfitting is a risk, not proof that every tuned rule fails on its first live account.

If a held-back period is used to revise the rules, it becomes development data. A chronological split preserves information order but does not remove all selection bias, revised-data leakage or future-dependent indicators. Further evaluation needs fresh data or an explicit account of prior reuse.

Test costs and execution explicitly: spread, commission, financing and account-currency conversion can consume a small gross surplus. Slippage is variable and can be favourable or adverse; do not impose a universal one- or two-pip penalty as a factual description. Walk-forward testing evaluates unseen periods but does not prove that an edge is permanent or real merely because each observed segment is positive.

Estimate expectancy from net trade outcomes and report uncertainty. A positive historical average is evidence to investigate, not proof of a permanent edge.

Define the review and the meaning of Forward

MT5’s Forward option divides a selected historical period, using its later part to check parameters; it is still a historical test. Recording signals on data arriving from today is another stage. Demo can check rules and software, but agreement with a backtest does not prove identical live execution. Separate logic errors, data gaps, costs and deviations rather than assigning every difference to overfitting.

Conditions can change, including costs, products and regulation. Historically, ESMA’s 2018 EU retail-CFD intervention applied from 1 August 2018 with 30:1 for regulatory major FX pairs and 20:1 for non-major pairs; this is not a global current-account rule or evidence of a strategy’s expiry date. Review net outcomes and pre-defined conditions rather than win rate alone. A second method need not stay uncorrelated or prevent account loss.

What to do tomorrow

  1. Report sample net mean, variability, large losses, drawdowns and concurrent exposure. A hundred records or a positive mean alone does not confirm a future edge.
  2. Specify signals, rejection rules, information availability and fills reproducibly. Save every tried version and keep untouched evaluation data separate.
  3. Check costs without double deduction, and diagnose historical/demo differences in logic, data and execution separately. Neither agreement nor a gap proves one cause.
  4. Review pre-defined groups and their counts, as the unchanged diagram shows. Selecting only profitable groups afterwards creates a new hypothesis.
  5. Prepare a research card with rule/data versions, trials, baseline/adverse costs, test results and unresolved limits. “Insufficient evidence” is a valid outcome; no mandatory live risk or fixed review quarter establishes safety.

Related reading: the expected value formula separates outcome amounts and frequencies; how to backtest a strategy covers data and execution; the trading journal preserves decisions and deviations; asymmetric trading bets distinguishes planned ratios, actual outcomes and sizing. Each answers a different part of evaluation.