ABC Go Part 3: Puzzle Cards and the Challenge of Smart Recognition

There’s something magical about puzzles. For children, assembling pieces into a complete picture brings joy and a real sense of achievement. Puzzles are used everywhere in early education, and very often they appear in the form of cut cards — simple images divided into two or four parts.

We quickly realized that this idea fits perfectly with ABC Go. Our vision was clear: let’s reward kids for assembling cards while also giving them a task — find and build the correct image first.

Technically, however, things didn’t go so smoothly right away.

In the first version of our product, we encountered a serious issue. Traditional AR and image recognition libraries tend to recognize images even if they are incomplete. For example, even half of a card with a chicken was often enough for the system to say, “That’s a chicken!” This ruined the whole idea of building puzzles.

We knew we had to make it smarter. So, we came up with two key solutions.

First, we designed unique markers for each puzzle piece. This meant adding special visuals — stars, fireworks, and other distinctive background elements — that helped the system tell each part apart.

Second, we made the system track each piece individually, including its position. This allowed the app to know if the pieces were really placed together to form a full card.

But there was one more big challenge.

Children don’t always align puzzle pieces perfectly. Sometimes they are a little bit crooked or not completely connected. For humans, that’s fine — our brains easily fill in the gaps. But the system needed to learn this too.

So, we created a tolerance system that allows for small errors. As long as the puzzle pieces are reasonably aligned, the app considers them complete and rewards the child.

The result? It works beautifully. I tested it myself and loved the experience. It feels natural, playful, and very rewarding for the kids.

With this, ABC Go takes another step forward — not just showing AR objects, but creating meaningful and interactive play that teaches and delights.