search

Cartmanishere / zippyshare-scraper

A module to get direct downloadable links from zippyshare download page.
MIT License
106 stars 28 forks source link

zippyshare changed somehow? #28

Closed ns6000 closed 3 years ago

ns6000 commented 3 years ago

Hey, since today i can't process zippy links, it's testing them before output and all are getting redirected to hell and outputting this error:

if count > 3: logging.error('{} redirected more than {} times'.format(link, count)) return None

i've tried to increase count but no dice...

Cartmanishere commented 3 years ago

@ns6000 It feels like something has changed from Zippyshare side. I will take some time to debug this.

Cartmanishere commented 3 years ago

@ns6000 I have pushed out a fix for this. It is working for me. Please checkout the latest code from this repo and try on your end.

ns6000 commented 3 years ago

it's working now, thank you!

ns6000 commented 3 years ago

I'm not complaining, just FYI. I've got chance to put it through it's paces and it's not 100% successful, there are maybe 1 or 2 instances per 50 links where it's still unable to generate true download link. But it's some timeout issue I presume, because if you try it multiple times, it eventually works.

Cartmanishere commented 3 years ago

@ns6000 I am facing a similar issue as well. This will take some time to debug. Till then, you can try increasing the retry count (I had already increased it to 8, maybe 10-15 should give enough chances that the links are parsed).

ns6000 commented 3 years ago

ok, thanks for the tip

advith231 commented 3 years ago

@ns6000 I am facing a similar issue as well. This will take some time to debug. Till then, you can try increasing the retry count (I had already increased it to 8, maybe 10-15 should give enough chances that the links are parsed).

see the source code of jdownloader2 it works perfectly with all zippyshare files although its wrriten in java

Cartmanishere commented 3 years ago

@advith231 I haven't looked too deep but afaik jdownloader uses an actual browser instance in the java process to run the javascript code and generate the download link.

We have discussed a similar approach for this project here. But that approach is not as fast the current approach where we don't run any JS but extract the link by matching the link generation logic.

As long as this works, I will not move to a browser runtime approach.

Cartmanishere commented 3 years ago

@ns6000 I have taken a new approach for generating download links. In my testing, this has proven more robust than previous approach. Please give it a try by cloning the latest code. (Note: Install the dependencies).

ns6000 commented 3 years ago

@Cartmanishere i've tried 20 files and all without problems, great job, thank you!