thisisparker / xword-dl

⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛
MIT License
139 stars 30 forks source link

centralize encoding cleanup and add emoji processing step #157

Closed thisisparker closed 6 months ago

thisisparker commented 6 months ago

A little refactor that touches a handful of downloaders, this change moves the "cleanup" step required for various text fields to fit into the constraints of .puz into one central location. That simplifies each downloader a bit and should help avoid bugs where new fields aren't sanitized correctly.

Along the way, we pick up a new dependency, alas. This new emoji library converts :fire: to :fire:. That is better for readability than just dropping the emoji, and also should be losslessly reversible client-side if desired.

Fixes #148 Fixes #155