iw4p / proxy-scraper

scrape proxies from more than 5 different sources and check which ones are still alive
MIT License
514 stars 131 forks source link

Sources #31

Closed SwordKing69 closed 6 months ago

SwordKing69 commented 1 year ago

Hii there I used this scraper and it was good but it will be more better if you please provide option to manually add sources for proxies. Like different sources files http,socks4,socks5 so that we can have more proxies from our own desired url/website. Please look into this.

iw4p commented 1 year ago

Hi It's a scraper, which is designed for specific websites. So it can not scrape every proxy website until I implement it because each site has different tags. You can give me websites and I'll add them.

ghost commented 1 year ago

why don't you add a regex for ip:port instead, saves you the trouble.

iw4p commented 1 year ago

why don't you add a regex for ip:port instead, saves you the trouble.

Because I'm not just good at regex, :D kidding, but having a well-structured scraper is better than just using regex in my prospective. Regex is my last solution always. Do you think regex is a better option?

ghost commented 1 year ago

Well, sure why not? it can barely mess with your well structured code as it only consists of 5-6 lines of extra code (for simpler terms), without requiring any fancy structuring to the code.

iw4p commented 1 year ago

It can be a good solution, but I have to test it. If you have time you can do a sample on your fork maybe? And we can check the time for doing benchmarks.

ghost commented 1 year ago

sure thing, I'll let you know once i have done it

iw4p commented 1 year ago

Thank you!