Notematic

Beautiful handwritten note recognition

I often like to write notes by hand in a journal -- it gets me away from the screen and distractions, and I find it's less off-putting to pull out a little notebook than your laptop or phone in many places. Of course, the downside of the notebook was that if I ever wanted to refer to it, I needed to transcribe it back into the computer.

Now to be fair, most of my notes, just like thoughts, are not that important. Often the notes are just a way to hold a thought for a moment. I'm also not one for extreme organization or anything, but I found it a bit sad that my digital notes and handwritten notes had to live such separate lives. Now, thanks to LLMs, this no longer has to be the case!

It turns out LLMs like Gemini Flash are really good at deciphering handwriting, and if you give them another pass over the material to polish the content, they can turn scrawls like this:

into polished output like this:

Originally I had the model output a psuedo-HTML to allow the use of margin notes and other formatting, but I found that a Markdown output is convenient for integrating with other note-taking systems, so I changed this to be the default.

My Workflow

I just use the Google Docs scanning functionality to quickly capture a PDF of recent pages and then periodically sync to my notes folder. I use Obsidian casually for notes, so merging with the Obsidian lets me to refer to hand-written notes easily.

Issues

• If you handwriting is as inconsistent as mine, you'll likely find the LLM will struggle with spelling/meaning, but I find it works well enough to understand.

• Gemini 2.0 Flash is the best model for price/performance at the moment, but overloads easily.