The refusal: not negative on this sample
By Robert Yenokyan,
The AI testing a crypto strategy for us found a good-looking result.
It refused to say the strategy worked.
What it wrote instead: not negative on this sample.
If you decide where client money goes, that sentence is worth more than the result.
The strategy trades pairs of crypto assets that usually move together. The test replayed twenty days of market data to see how its orders would really have been filled.
Then it explained why the result does not mean the strategy works.
A handful of days carried the result. Ten of the twenty days, added together, were negative. Twenty days, it said, was not enough to call the strategy ready to trade.
Next it listed every bias it could not remove from the test, and which way each one would push the result.
Then it named what the test had not modelled at all: how the trades would affect each other inside one portfolio, and how far our own orders would move the price beyond what the order book showed.
The test had also left out funding. On perpetual futures, the crypto contracts the strategy traded, one side pays the other at set times of day. Many trades were still open when a payment came due.
The next step it asked for was not to put the strategy live. It was a longer test, over 90 to 120 days instead of twenty.
Good numbers are easy to come by. Before client money goes near one, you want the list of ways it could be wrong.
Ask that of whoever brings you a result, a person or a machine: what it did on this sample, which days carried it and what the test left out.
Every result from our AI copilot comes with that list: what the test could not account for, and which way each gap would push the result.
If you want research reported to you that way, write to robert_yenokyan@bonton.ai.
What the test did not model at all: funding, how the trades would interact inside one portfolio, market impact beyond the visible order book, a placebo test and a correction for how many variants had been tried. The next step was a run over 90 to 120 days, because a handful of days carried the result.