thisisparker / xword-dl

⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛
MIT License
139 stars 30 forks source link

Can't download nyt -d 9/27/2018 #175

Closed clorgie closed 5 months ago

clorgie commented 5 months ago

Me again! I am running code cloned from here this morning, so should still be relevant. Can't download 9/27/18 NYT puzzle:

$ xword-dl nyt -d 9/27/2018                                                                                      [15:48:34]
Traceback (most recent call last):
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/bin/xword-dl", line 33, in <module>
    sys.exit(load_entry_point('xword-dl==2023.12.2', 'console_scripts', 'xword-dl')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/lib/python3.11/site-packages/xword_dl-2023.12.2-py3.11.egg/xword_dl/xword_dl.py", line 257, in main
    puzzle, filename = by_keyword(args.source, **options)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/lib/python3.11/site-packages/xword_dl-2023.12.2-py3.11.egg/xword_dl/xword_dl.py", line 46, in by_keyword
    puzzle = dl.download(puzzle_url)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/lib/python3.11/site-packages/xword_dl-2023.12.2-py3.11.egg/xword_dl/downloader/basedownloader.py", line 98, in download
    puzzle = self.parse_xword(xword_data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/lib/python3.11/site-packages/xword_dl-2023.12.2-py3.11.egg/xword_dl/downloader/newyorktimesdownloader.py", line 178, in parse_xword
    puzzle.clues = [c['text'][0]['plain'] for c in clue_list]
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chris/Library/CloudStorage/Dropbox/crosswords/xword-dl-env/lib/python3.11/site-packages/xword_dl-2023.12.2-py3.11.egg/xword_dl/downloader/newyorktimesdownloader.py", line 178, in <listcomp>
    puzzle.clues = [c['text'][0]['plain'] for c in clue_list]
                    ~~~~~~~~~~~~^^^^^^^^^
KeyError: 'plain'
(xword-dl-env) FAIL

Cheers!

thisisparker commented 5 months ago

Thanks for the report! This one should be fixed too. Does this mean there haven't been issues on puzzles between 2018 and now?

(Incidentally, with these and the other ones you've reported, there's something weird enough going on that the .puz might not be a great solving experience. But it's at least mostly complete!)

clorgie commented 5 months ago

Yep, 2017-now. They were all downloaded using xword-dl at some point! I'm working backward through my archive, only through 2017 so far :)