search

thisisparker / xword-dl

⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛
MIT License
147 stars 32 forks source link

NYT authentication fails with a 403: Forbidden for url #51

Open iperry opened 2 years ago

iperry commented 2 years ago

Might be the same as #38, not sure if NYT has put in place some mitigation measures or not.

Invoked with xword-dl -u <myuser> -p <mypass> nyt

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://myaccount.nytimes.com/svc/ios/v2/login

I was able to authenticate from one location (home), but not from another (office). Manually copying an NYT-S cookie from an already authenticated session and dropping it into the yaml file for the office worked.

Any ideas on what to try?

thisisparker commented 2 years ago

Hm, I am also seeing this error at least intermittently. I think it does reflect some kind of mitigation measures. (I have futzed around a bit with values that I'm pulling directly off mitmproxy from my own app instance, but haven't been able to circumvent it.)

I have to think a bit on how to best solve this. If it is like #38, then maybe it'll just resolve itself, though I kind of doubt it. My hunch is that either we should treat authentication as out-of-scope and provide documentation for manually copying over the NYT-S token or similar, OR switch to a less brittle method. I'm open to suggestions on that front!