phillipunlimited / ResidentAdvisorPromoterScrappingTool

A Resident Advisor Scrapping Tool for Promoters
MIT License
2 stars 0 forks source link

Get result for different city #1

Open e-garbage opened 6 months ago

e-garbage commented 6 months ago

Greate script, but I have an issue: I'm stuck on scrapping my own city over and over again. I guess RA detect my IP and that's why it behave like this. I've tried several things:

Maybe there is cash file or cookie I should edit/delete??' I have no idea :/

phillipunlimited commented 6 months ago

Hey Eric,

Oh man, I haven't touched this script in a while. I'm on the go and currently unable to run the script. But I think on line 154 it should be:

soup = await bs ('https://ra.co/promoters/de/berlin')

try that out and let me know.

On Mon, Feb 12, 2024 at 4:28 PM eric nardini @.***> wrote:

Greate script, but I have an issue: I'm stuck on scrapping my own city over and over again. I guess RA detect my IP and that's why it behave like this. I've tried several things:

  • modifying the script on line 154: soup = await bs(' https://ra.co/promoters/[de]}/[berlin}') by directly giving the country code and citylike this for example. I get it to work only one time out of tens of tries
  • routing the trafic through TOR but it never worked

Maybe there is cash file or cookie I should edit/delete??' I have no idea :/

— Reply to this email directly, view it on GitHub https://github.com/phillipunlimited/ResidentAdvisorPromoterScrappingTool/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4MTDCU2Z57MCV6Z6H2NQWDYTKJQTAVCNFSM6AAAAABDFPVEMKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZTCMBZHAYDKNA . You are receiving this because you are subscribed to this thread.Message ID: @.*** .com>

e-garbage commented 6 months ago

Hey phillip

Ok my was probably not fully working last time! You got it right corect syntax is soup = await bs ('https://ra.co/promoters/de/berlin') but I identified a problem with User-agent list: all the "AppleWebKit/537.36" type raised errors on my machine. removing them from the list made the script to work smoothly.

Problem solved