I built an AI stock predictor at 12. Here's what I got wrong.

I built an AI stock predictor when I was 12. It took me about six months. It looked like it worked. I don't think it did.

I'd rather say that first.

Why I built it

I kept seeing AI, AI, AI. So I asked myself what I really needed help with. For me, it was investing. I wondered why AI wasn't predicting stocks for me, and I decided to build something that would.

What it was

In the beginning it was all ChatGPT. I'd copy the code, paste it, and run it in Google Colab, which someone recommended to me. Later I moved it over to Google's Antigravity editor.

It pulled about three years of price history for a stock. Then it looked at a bunch of things: a few dozen technical indicators, some numbers about the company (like its P/E ratio), the news headlines, and a list of big market events, like the 2008 crash and COVID. Twelve different models each made a guess, and it blended them into one.

Then it emailed me. A list of stocks, and a report on each one, with a price prediction and a Buy, Hold or Sell.

I used it for a little bit. When it picked something I didn't believe, I figured I was probably the one who was wrong.

Why it looked so good

At one point the report said it was 95% accurate. That was its peak. Then the number started falling as I kept making changes, and I never worked out why. I got annoyed. I was making a lot of tweaks to make it a little bit better, and I couldn't tell what I was breaking.

The accuracy number was also wrong for a while. I didn't catch that either.

Testing was the hardest part. At the start I really didn't know how to test it. I tried old data, pretend money, and real money. Over time I understood it more. I couldn't test it for longer than about three months, because I didn't have enough computing power.

What I found when I looked again

Recently I gave the code to Claude and asked what was wrong. That was the first time it made sense.

My model was predicting today's price. And the numbers it used to do that were built from today's price, like moving averages. That's like guessing someone's test score while looking at the test score. Of course it looks accurate.

So we tested it. We made 30 fake stocks that are pure random noise, where nobody could ever win. We trained a model the way mine was trained. On the test it scored 0.999, which is nearly perfect. Then we asked the real question: was tomorrow up or down? It was right 49% of the time. That's a coin flip.

It got worse. The hourly and monthly predictions were made by adding random numbers to today's guess. There was no hourly data in there at all. The "social media" score wasn't social media, it was a hard-coded list: meme stocks got 75, big companies got 50, everything else got 25. The code also said it used 60+ indicators. I counted about three dozen.

This is the version I looked at, the one from Colab. The one I run now might be different. I'm not sure yet.

What I think now

When I first built it, I thought, "I made this, so it probably doesn't work." I was kind of right, just not for the reason I thought.

I can't tell you whether it ever picked a good stock. Some of its picks went up. But if a system is close to a coin flip, a pick going up doesn't tell you anything. I also don't trust the numbers I had in my head from back then, because some of them were wrong.

Here's what I know now.

• A great score on the data you trained on means nothing. Test on days the model never saw.
• Compare it to doing nothing, like just buying the S&P 500.
• If a result sounds too good, check the test before you celebrate.

I haven't switched it back on.

What I want to do next

I want to get it right this time. Some of this is a plan, and one part is only an idea. I'll say which.

First, the test. Before I add anything new, I want it predicting tomorrow, not today. And I only want to score it on days it has never seen. If it can't beat a coin flip then, nothing else matters.

Second, I want it to write its predictions down before the day happens. It logs what it thinks, with the date. After the market closes, I check it. No going back and changing things.

Third, it has to beat doing nothing. Every result gets compared to just buying the S&P 500. That's the real bar.

Fourth, I'm cutting the fake stuff. The made-up social score goes. So do the hourly predictions. If the data isn't real, it doesn't go in.

Now the big idea. This part is only an idea. It's not a plan yet. I want it to look at every single stock, all the time, and give each one a score out of 100. It would only recommend the ones at the very top. The score would come from at least 100 factors, and each factor would have its own weight. One factor could be fair price: what the stock costs now, next to what it should cost.

I also want it to give me information I can actually use, not a pile of numbers that's really hard to deal with.

It might not work. That's okay. If it can't beat the market, I'll say so, and I'll write about that too.

This isn't financial advice. Don't buy or sell anything because of a program I built. I couldn't prove mine worked.

Drafted and created with the help of AI.