vinc3PO / ebayKleinanzeigenAlert

Telegram Alert for ebay kleinanzeigen posts
MIT License
57 stars 15 forks source link

Webpage fetching error for url / ebay banned scrapper? #30

Closed mhumhu closed 1 year ago

mhumhu commented 1 year ago

Screenshot_3

Today at around 3pm Ebay added some weird popup while you are entering the site (see photos). Because of that, when trying to add a link to DB or fetch existing links in db, there is an error, for example:

>> Adding url << webpage fetching error for url: https://www.ebay-kleinanzeigen.de/s-konsolen/k0c279 << Link and post added to the database and script doesn't send messages to telegram anymore.

mhumhu commented 1 year ago

its either that or they banned the scrapper. i tried to debug it a few minutes ago with print(response.text) and this is what i'm getting:

ebAlert links -a https://www.ebay-kleinanzeigen.de/s-xbox/k0 { "title": "IP-Bereich vorrübergehend gesperrt.", "message": "In dem IP-Bereich 89.247.172. kam es vor kurzem mehrfach zu unsicheren Versuchen unsere Plattform zu verwenden. Dies kann auch durch andere Personen versucht worden sein. Daher wurde dieser IP-Bereich zur Vorbeugung von Betrug zeitweilig von der Nutzung von eBay Kleinanzeigen ausgeschlossen. Bitte versuche es einfach nachher erneut.", "more": "Mehr informationen: https://themen.ebay-kleinanzeigen.de/ip-eingeschraenkt/", "ref": "Ref#: 0.26851502.1682708696.2437f0ae IP#: 89.247.172." }

Needless to say that my IP is NOT banned, because i am able to connect without a problem to this url using my PC / mobile phone. so they probably banned something with the scrapper (?) was tested both on ebayAlert-1.0 and ebayAlert-1.3

cyberpete2244 commented 1 year ago

I add searches (links) directly in the database so I do not have this error and the scrapper itself still works, I do get notifications via telegram. So the process of adding links is problematic, I guess.

But this will likely change in a couple of days since with the message you point out they announce a overhaul of the website. So who knows. MaybeLinks and the scrapper and who knows what else will need adjustments.