search

thisisparker / xword-dl

⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛
MIT License
147 stars 32 forks source link

Data URL for the WSJ downloader is outdated #32

Closed rjmorris closed 3 years ago

rjmorris commented 3 years ago

Thanks for making this tool available and keeping it maintained! I've been having trouble with the WSJ downloader for a while, and I finally took the time to investigate. It looks like the data URL is no longer valid. I was able to fix it by replacing:

data_url = solver_url.replace('index.html', 'data.json')

with:

data_url = solver_url.replace('?embed=1', 'data.json')

in WSJDownloader.fetch_data().

This worked on the couple of puzzles I tested it on, using the latest release of xword-dl (2021.6.24).

brookehus commented 3 years ago

can verify that xword-dl wsj --latest did not work as expected

thisisparker commented 3 years ago

Excellent catch, and I apologize that I didn't see this issue for a month! I've used precisely your change @rjmorris in a01ab26dfa629920aad083047ff1074699b11721, and shipped that (and a very slight update to the dateparsing logic) in a new version just now. You can update from here or PyPI! Thank you :)