Open cr4kn4x opened 4 months ago
Bull shit need my page back
? What do you mean? I think this version is great and works well.
@cr4kn4x Hi, why close?
@cr4kn4x Hi, why close?
I can reopen again, but I think it won't get merged in near future. Seems like the maintainer is currently inactive
@cr4kn4x Yes the maintainer seems not so active recently. However, your pr is great :heart: and I am very appreciated for your work since the proxy is a must if we don't have large number of accounts and want to scrape in bulk.
Thanks again and I think this proxy function will help lots of users(pls don't get upset/blue just because someone's remark, I back you up :heart: ). :+1:
if we don't have large number of accounts and want to scrape in bulk
unfortunately proxies will not help in this case, as most of Twitter's rate limits are not determined by IP
i appreciate the effort on this PR, it's a great start. however, there are quite a few things to take into consideration. for example, users who want to access the onion site cannot do so, as httpx
doesn't support socks5h. i'll come up with a list of requirements.
Added support for optional usage of proxies and custom user-agents that can be passed in the construction of the Scraper Class. If the options are not provided, the Scraper class will use the random user agent and no proxy.