search

JimmyLaurent / torrent-search-api

Yet another node torrent scraper (supports iptorrents, torrentleech, torrent9, torrentz2, 1337x, thepiratebay, Yggtorrent, TorrentProject, Eztv, Yts, LimeTorrents)
MIT License
395 stars 99 forks source link

IPTorrents Authentication Doesn't Work #106

Open sgianelli opened 4 years ago

sgianelli commented 4 years ago

It looks like you cannot authenticate to IpTorrents right now because of the browser identification redirect they do now. If you have a valid cookie then it works but it doesn't seem like torrent-search-api is able to get that cookie on its own now.

JimmyLaurent commented 4 years ago

I can't test since I don't have an account anymore. Can you try replacing /take_login.php with /do-login.php in iptorrents.json please ?

sgianelli commented 4 years ago

It looks like its the same issue -- whatever service they use (something like cloudfare) makes you wait 5 seconds before redirecting you to prevent bots it seems. Not sure if there is going to be an easy way around that. You may be able to repro, or at least see what I'm seeing by navigating to iptorrents in incognito or on a VPN without logging in.

JimmyLaurent commented 4 years ago

It's a cloudfare protection and it's a pain in the ass.. We have the same problem on yggtorrent provider. There's a workaround that uses puppeteer, it's not perfect (heavy and will not work on some dockerized env), but it does the job until we have a better solution. Check this issue if you're interested.

unixfox commented 4 years ago

Sorry for the off-topic. @JimmyLaurent which dockerized environment are you referring to? Is it when docker requires to run it as a non-root user?

JimmyLaurent commented 4 years ago

It's been a while now and It may have changed but I remember having some troubles running chrome inside Heroku or Now services (with free plans).

BTW, good job with the puppeteer solution.

unixfox commented 4 years ago

@JimmyLaurent I was right, Heroku doesn't allow running Docker as root so that's why it doesn't work and for vercel it's planned: https://github.com/unixfox/pupflare/issues/3

JimmyLaurent commented 4 years ago

@unixfox Still off topic: you might be interested by this buildpack, I made some tests and ran puppeteer flawlessly on heroku.

@sgianelli I added a new experimental cloudflare bypass. It's like pupflare, but only use puppetteer to grab cookies. So if it doesn't bother you to use chromium headless, you might be interested. To use it, take the latest version of" torrent-search-api" and also add the "cloudflare-scraper" package to you project dependencies. If you do so, can you tell me if it solve your issue ? Thanks.

97WaterPolo commented 4 years ago

@JimmyLaurent

Hiya! I'm running into the same issue as @sgianelli with IPtorrents "cloudscraper": "^4.6.0", "torrent-search-api": "^2.1.3", with the above versions for the two packages and the default iptorrents.json. Still failing to get through the authentication. Any other tips or suggestions?