search

mrichards42 / xword

Cross-platform crossword solving
https://mrichards42.github.io/xword/
GNU General Public License v3.0
42 stars 13 forks source link

Downloader sources #125

Open benfishbus opened 3 years ago

benfishbus commented 3 years ago

Thanks so much for developing xword!

Suggestion: How about adding the NYT "from our archives" puzzles to the downloader? Three or four of them are posted for free download from the puzzle page in .puz format.

Question: Release 0.7.0 is giving me 404 on The New Yorker, but I see source has been updated. Assuming another release is not imminent, can I pull/download default_sources.lua by itself?

mrichards42 commented 3 years ago

Hi! RE: The New Yorker source, yes, you should be able to pull default_sources.lua and replace the file with that. You could also just change the url in the downloader preferences if you don't want to muck around with replacing the script file, although there's a related bug where the preferences don't save (but see #116 for a simple solution).

I'm happy to add more sources if you have a standard url that can be fetched for a given date (i.e. there's year/month/day in some order in the url). It looks like the "from our archives" puzzles change randomly and don't follow a pattern though, so it might be tricky without writing a custom scraper which can be time-consuming.

benfishbus commented 3 years ago

Yes, the NYT archives puzzles change randomly. The links only load in a browser, too, so rudimentary scraping via curl/wget doesn't cut it. The whole DOM has to be loaded, then parsed for .puz links in that section. I'm sure this can be done, and served in an RSS feed for easy fetching, but it's beyond my ken (regardless how many times I poke around in it).