0x776b7364 / toggle.sg-download

Python script to automate Toggle.sg video downloads
MIT License
26 stars 18 forks source link

Script gets stuck at certain URLs #29

Closed Kerean closed 7 years ago

Kerean commented 8 years ago

I have been using this awesome script and it works to download both drama serial episodes and Hello Singapore.

However, recently, there seems to be an issue with specific URLs, the script just gets stuck at:

[i] Obtaining URL records from download URL response ...

This is the URL which has this problem. http://video.toggle.sg/en/tv-show/news/aug-2016-hello-singapore/thu-11-aug-2016/438339

The 10 August version also gets stuck. However, if I try to download 2 August, it is okay.

0x776b7364 commented 8 years ago

I haven't examined this in detail, but could it be possible that the video is still being processed? Perhaps try downloading the episodes on 11 and 12 Aug after a week, say, on 20 Aug.

SungMatt commented 8 years ago

It has happened on another URL, the one of Joseph Schooling's race yesterday.

Kerean commented 8 years ago

I tried to download the same 11 August episode again today, and it is still stuck at the same line:

[i] Obtaining media name ... [i] Obtained media name = Aug2016-_HelloSingapore-_Thu_11_Aug_2016 [i] Obtaining URL records from download URL response ...

It doesn't proceed forward after these lines.

I also tried the particular link by SungMatt and encounter the same problem as him. It seems that there is something that is wrong and this is repeatable.

Kerean commented 8 years ago

To add more data points to help you to troubleshoot this, I could download these (just did it like right now):

http://video.toggle.sg/en/series/the-gentlemen/ep1/437634 (9 Aug) http://video.toggle.sg/en/series/the-gentlemen/ep2/437975 (10 Aug) http://video.toggle.sg/en/series/the-gentlemen/ep3/438381 (11 Aug) http://video.toggle.sg/en/series/the-gentlemen/ep4/438706 (12 Aug) http://video.toggle.sg/en/series/the-gentlemen/ep5/439401 (15 Aug) http://video.toggle.sg/en/series/the-gentlemen/ep6/439628 (16 Aug)

It seems that the issue is not the date but something else. Hope you can assist as I really want to download the Hello Singapore episodes before they get removed.

Kerean commented 8 years ago

As an additional data point, specific to Hello Singapore, it seems that the turning point was 8 August. I am able to download up till 5 August but from 8 August onwards, the script gets stuck.

0x776b7364 commented 8 years ago

Thanks for those. I'll take a look as soon as I can.

Kerean commented 8 years ago

Thank you so much! I'm eagerly looking forward to your update!

Kerean commented 8 years ago

Would you have a chance to look at it yet? :)

0x776b7364 commented 8 years ago

Sorry, haven't had the chance to look at it yet. Are you still facing the same issue on the previously-affected URLs?

0x776b7364 commented 8 years ago

For the affected links pasted above, the only available file types were wvm and not m3u8 or mp4. If you've used the default script parameters, as AUTO_DOWNLOAD is set to 1, and no wvm file type is present in FILE_PREFERENCES, the script will enter an unexpected state. Again, just for the affected links above, setting AUTO_DOWNLOAD to 0 will cause the script to list the available files to download; however, only wvm files are present, and I believe it isn't much use to us at the moment. I've pushed 258fe9d which should fix the script hanging issue.

Kerean commented 7 years ago

Thanks so much for helping with this, although I'm not sure if we have yet achieved a solution. I just encountered a similar problem today http://video.toggle.sg/en/tv-show/news/sep-2016-hello-singapore/wed-14-sep-2016/444787. I tried setting Auto_Download to 0 and then downloading all 4 WVM files that are listed after that.

Am I correct to say that Mediacorp only puts in wvm files for certain episodes and m3u7 and mp4 files for other episodes? Is there any way we can use these wvm files or play them back? I tried VLC but it appears that it can only play the first 10 seconds as the rest seems encrypted (based on my quick searches in Google, most of which I don't quite understand).

Would you have some ingenious way of solving this problem that we have, or am I resigned to not being able to download episodes whenever the m3u8 or mp4 is not used by Toggle? I noticed it seems to keep happening to Hello Singapore which I really want to keep.

0x776b7364 commented 7 years ago

I'm not sure on the selection criteria of publishing the videos as m3u8/mp4 vs wvm-only. And yeah, if it's wvm-only, you're out of luck. The script is just a way to download, not to decrypt. You could try playing the file using Google Play Movies on your Android phone or PC, though I've heard that this method doesn't quite work well.

Kerean commented 7 years ago

I see, I guess I'll just download them first and figure out how best to play/decrypt them. You mentioned earlier about "File Preferences". Whilst I am doing the manual downloading now by setting Auto_Download to 0, could you advise how I could set the File Preferences so that it downloads the best file if there is a MP4 or M3u7, and if not, to download the WVM? This will save me a little bit of keystrokes when selecting the files to download.

0x776b7364 commented 7 years ago

If AUTO_DOWNLOAD is set to 0, then the script wouldn't select any file by itself. You'll need to set it to 1, and then change the order of the tuples in the FILE_PREFERENCES variable.

0x776b7364 commented 7 years ago

Closed due to inactivity.