search

steveruizok / perfect-freehand

Draw perfect pressure-sensitive freehand lines.
https://perfectfreehand.com
MIT License
4.52k stars 159 forks source link

[feature] Drawing and handwriting recognition #67

Closed mattakamatsu closed 1 year ago

mattakamatsu commented 1 year ago

Do you have any recommendations on what drawing- and handwriting-recognition tool would integrate well with perfect-freehand? E.g. the options in https://github.com/WICG/handwriting-recognition/blob/main/explainer.md . Would love your opinion, thanks!

steveruizok commented 1 year ago

I haven't really looked into this, though I'm excited to see movement on handwriting recognition as a browser API. My hope is that by the time it ships, we'll have an equally effective image recognition via AI. I've experimented with a few models (such as https://huggingface.co/microsoft/trocr-large-handwritten) and they don't really seem to be ready yet.

What would be even better is to train the models on the raw data in this library rather than an image generated from the final strokes. Not sure what that might look like though. In any case, recognition would not be part of this library (which just deals with the "stroke" polygon), so I'm closing the issue. Thanks for the interesting question!