jpd236 / CrosswordScraper

Browser extension which downloads crosswords from crossword applets for offline solving.
Apache License 2.0
28 stars 1 forks source link

"Scrape error" observed by some users attempting to scrape the New York Times puzzles #12

Closed jpd236 closed 2 years ago

jpd236 commented 2 years ago

I received a report over email that a user is seeing a "Scrape Error" when attempting to scrape the NYT puzzle, as of 3/1/2022. Another user reports the same symptoms on the extension support page in the Chrome store listing.

One of the users was able to reproduce the problem in developer mode, which appears to be that:

My best guess is that the NYT is gradually rolling out a change to the site where the "pluribus" data is moved to "gameData" - that would explain both cases, since pluribus data isn't base64. But the only way I can confirm this is if I start to see the change myself, or if a user can share the HTML page that they're viewing (which I've requested).

jpd236 commented 2 years ago

There is indeed a new applet that appears to be gradually rolling out, but it doesn't include any embedded puzzle data in the HTML itself; instead, it makes an API call to fetch the puzzle data.

While I can't easily verify this locally, I believe the above commit should resolve the issue at least for the current version of the applet that was shared with me.