search

JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.47k stars 708 forks source link

Snscrape works fine locally but not on heroku #198

Closed E-STAT closed 3 years ago

E-STAT commented 3 years ago

Hi.

Please, I used snscrape in a flask app and it did work fine locally using the development version way of installation:

pip install git+https://github.com/JustAnotherArchivist/snscrape.git

I was having issues installing same way on heroku but had to use:

pip install snscrape

I read in a tutorial that some endpoint are not available if snscrape in not installed using the development version way. Therefore, my app is not working when I call some endpoint where snscrape is used.

Please, is there a workaround to get the developement version of snscrape installed on heroku or any other production servers?

Thanks for your responses in advance.

JustAnotherArchivist commented 3 years ago

Duplicate of #79

JustAnotherArchivist commented 3 years ago

@ThePlugJumbo wrote this comment at 08:19 UTC today and then deleted it:

This guy just admits he hasn't solved the issue but still closes it... Dude your entire library doesn't work on AWS and Heroku and you act like its not a big deal or your problem. Terrible developer only got that many upvotes because it was the first. I'm going to remake this junk.

Good luck, mate. You seem to have missed the part where Twitter has banned AWS IPs. There's literally nothing I or you can do about this. snscrape supports proxies via the standard environment variables HTTP_PROXY and HTTPS_PROXY, so use that if you have to run on AWS.