PULSE

Viral posts promise impossible returns.

On X and Reddit, trading bots show off huge backtests, usually the one version that worked. I wanted to know whether anything I built would survive a test I couldn't bend: data, rules and pass thresholds fixed before running it.

A bot, and a way to check it.

The agent

Deterministic strategies decide; a risk layer checks every order: 5% per position, bracket stops, a daily loss limit and a kill switch. An AI committee votes, but at weight zero until it proves itself on new data.

The hindsight test

Its best backtest used 15 stocks chosen today. On 100 random sets of 15 stocks from the 2005 index, none matched it: a median Sharpe ratio of 0.41 against 0.88.

Outside the US

A 200-day trend filter cut the worst fall in all six markets I tested, but passed the pre-set rules in only three, and cost 1.3 to 4 points a year after Italian taxes.

Long and short, 1 to 5 days

Nine reversal variants, designed with Grok as a second reviewer. The signal existed before 2015, but every variant lost money after costs in 2015–2019.

Write the rules first, then look.

Five steps: an idea, a backtest, a check on years it had never seen, the hindsight test with 100 random sets of stocks, then a pre-registration and a single final run. Each pre-registration fixes data, variants, costs and pass thresholds, signed with a date and a hash of the code. The test command refuses to run if the code has changed, and every variant tried is counted, so the final statistics pay for all of them.

Five decisions against my own interest.

• Paper money only. The bot runs on Alpaca's paper account by default. Real trading needs two explicit settings, and the API keys never leave my machine.
• Point-in-time data. Tests use the S&P 500 members of each day, including companies that later left the index, not today's winners.
• Withdraw my best result. An early test gave the strategy a Deflated Sharpe Ratio of 0.97. When the hindsight test showed it came from the stocks I had picked, I withdrew it in writing.
• Never open the last test. The short-term strategies failed validation, so the 2020–2026 data stayed untouched, as the rules said.
• Keep the failures. One run broke because of a bad price series in the data. I kept the invalid result, wrote a data rule and registered the test again, instead of quietly re-running it.

A clean no is a result.

None of the strategies beat holding the market after costs, so the sensible choice for real money is a low-cost index fund held for years. What I keep is the process, and it carries straight over to growth. To test a new acquisition channel, I would write down before launch the one metric that decides it (say, cost per activated user), the smallest effect worth acting on, the sample size and the stop date. Then I would count every variant tried and read the result once. If the channel misses the bar, it gets cut, not re-sliced until some segment looks good.

Twelve months with frozen code.

The bot keeps running in paper mode with its code frozen, mainly to catch bugs and gaps between live trades and the backtest. One last idea, a weekly version of the strategy, has its rules written and will be tested once on data from 2020 onwards.