search

lheyberger / mtg-parser

Magic: the Gathering decklist parser
MIT License
10 stars 1 forks source link

How to bypass Moxfields scraping detection #7

Closed thebear132 closed 7 months ago

thebear132 commented 7 months ago

I have a similiar program which also downloads decks from Moxfield, here is how to bypass it if you are interested! Any questions, feel free to message me

https://github.com/thebear132/MTG-To-XMage/commit/0272d5914efcdc7c5f1ceaed27b21409c7ac02a8 Check the changes in program.py, just change your user agent when sending the request from (in my case) "python-requests..." to some random valid user agent

lheyberger commented 7 months ago

Thanks for reaching out.

I'm already doing a similar thing in the unit tests of this library (here). The reason it's not in the main library is that it's actually a grey zone and I prefer to let users of the library bypass it however they see fit. The library has an easy way to provide a pre-configured requests.session and this does the trick.

Re-reading the README.ME file, it could be better explained as I'm listing this as a "known issue". Thanks for pointing it out.