search

juliensalinas / torrengo

Torrengo is a CLI (command line) program written in Go which concurrently searches torrents from various sources.
https://juliensalinas.com/en/golang-concurrent-torrent-scraper-cli-torrengo/
GNU General Public License v3.0
97 stars 15 forks source link

Can't search on yggtorrent anymore #14

Closed pokapow closed 4 years ago

pokapow commented 4 years ago 
An error occured during search on Ygg Torrent
ERRO[0008] The ygg search goroutine broke                _source="torrengo/torrengo.go:498" error="error while fetching url: status code error: 503 503 Service Unavailable" input=pokemon
No result found...
exit status 1

I think this is due to their new browser detector when searching. Is there a way around this?

juliensalinas commented 4 years ago

@pokapow I have the same problem too indeed. I'm working on it!

pokapow commented 4 years ago

In python I use selenium webdriver with Gecko to enable javascript and bypass cloudflare. It's working fine, but I don't know how to manage this in Go.

juliensalinas commented 4 years ago

Yes actually in the past I used to use the Python Clouflare Scrape lib which worked great but as it required Python + Nodejs I ended up using a Go lib which was much better (no external dep and much faster). Now I'm considering going back to the Python solution as I can't seem to find a working Go solution (for the moment).

juliensalinas commented 4 years ago

@pokapow I finally fixed the Ygg issue. I had to completely rewrite the code in order to use Google Chrome under the hood. It makes the program a slightly slower and sadly it requires an external dependency, but that's the best way I found to bypass the Cloudflare protection. Hopefully it will keep working for a long while!

Feel free to let me know if it's not working on your end.

ferdir commented 3 years ago

Hi @juliensalinas and thanks for your work ! Everything was working fine with yggtorrent until now (No result found...) ; it seems to be because of the Cloudflare protection again. Do you have an idea how to fix it ?

juliensalinas commented 3 years ago

IHi @ferdir thanks for the feedback, I'll have a look when I have a moment

juliensalinas commented 3 years ago

@ferdir I don't have any issue with ygg... Maybe it was a temporary problem on their server?

ferdir commented 3 years ago

Indeed there is no more problem