spaam / svtplay-dl

Small command-line program to download videos from some streaming sites.
https://svtplay-dl.se
MIT License
715 stars 119 forks source link

Issue with tv4play.se #925

Open SweDennis opened 6 years ago

SweDennis commented 6 years ago

Hello. I just received this when trying to download an old episode from tv4play.se. Haven't used svtplay-dl in a while so can't say if this is new or old...

Using a fresh install of Linux Mint, 19/Tara

dennis@Mint:~/Downloads$ ./svtplay-dl -v https://www.tv4play.se/program/s%C3%A5-mycket-b%C3%A4ttre/3946904 DEBUG [1531031809.59] ./svtplay-dl/svtplay_dl/init.py/get_media: version: 1.9.11 DEBUG [1531031809.6] ./svtplay-dl/svtplay_dl/utils/init.py/request: HTTP getting 'https://www.tv4play.se/program/s%C3%A5-mycket-b%C3%A4ttre/3946904' DEBUG [1531031809.7] ./svtplay-dl/svtplay_dl/utils/init.py/request: HTTP getting u'http://prima.tv4play.se/api/web/asset/3946904/play' Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "./svtplay-dl/main.py", line 13, in File "./svtplay-dl/svtplay_dl/init.py", line 511, in main File "./svtplay-dl/svtplay_dl/init.py", line 217, in get_media File "./svtplay-dl/svtplay_dl/init.py", line 265, in get_one_media File "./svtplay-dl/svtplay_dl/service/tv4play.py", line 52, in get File "./svtplay-dl/svtplay_dl/utils/init.py", line 68, in request File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 508, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 618, in send r = adapter.send(request, kwargs) File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 508, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='prima.tv4play.se', port=80): Max retries exceeded with url: /api/web/asset/3946904/play (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb07b38b590>: Failed to establish a new connection: [Errno -2] Name or service not known',)) dennis@Mint:~/Downloads$

qnorsten commented 6 years ago

I had the same problem as you earlier today, and now it works again (with latest master). So I assume it was some temporary error on tv4plays side as prima.tv4play.se was unresolved as a host.

Sopor commented 6 years ago

Maybe you should tell us the version of svtplay-dl you are using @SweDennis? Even if i can read 1.9.11 in your debug text i have also the same version number but i'm running latest master. So you can't really rely on that information.

SweDennis commented 6 years ago

@qnorsten Ok, I still have the same problem.

@Sopor- Don't know what to say, the output from svtplay-dl --version gives 1.9.11. Downloaded from https://svtplay-dl.se/download/latest/svtplay-dl

dennis@Mint:~/Downloads$ ls -lrt total 12684 -rwxr-x--- 1 dennis dennis 85978 Jul 8 08:35 svtplay-dl

dennis@Mint:~/Downloads$ md5sum svtplay-dl 0440ef7ce7f550d0b1421090adb24109 svtplay-dl dennis@Mint:~/Downloads$

wahboh commented 6 years ago

I have had problems downloading from tv4play for at least a month now. I have read about an update in the bug tracker, but I have seen no new release after 1.9.11. As the Windows version always seem to lag behind, I installed accordning to instructions on my little Raspberry Pi 3+ w Debian. svtplay works beautifully, but never ever tv4play.se.

Maybe I'm just ignorant, but should not a new release be installed with the standard "sudo apt-get update" command? IS there a new release after 1.9.11?

Anyway, tv4play crashes 100% repeatedly:

DEBUG [1531085916.61] /usr/lib/python2.7/dist-packages/svtplay_dl/init.py/get_media: version: 1.9.11 DEBUG [1531085916.61] /usr/lib/python2.7/dist-packages/svtplay_dl/utils/init.py/request: HTTP getting 'https://www.tv4play.se/program/tareq-taylors-nordiska-matresa/3934584' DEBUG [1531085916.94] /usr/lib/python2.7/dist-packages/svtplay_dl/utils/init.py/request: HTTP getting u'http://prima.tv4play.se/api/web/asset/3934584/play' Traceback (most recent call last): File "/usr/bin/svtplay-dl", line 7, in svtplay_dl.main() File "/usr/lib/python2.7/dist-packages/svtplay_dl/init.py", line 511, in main get_media(urls[0], options) File "/usr/lib/python2.7/dist-packages/svtplay_dl/init.py", line 217, in get_media get_one_media(stream, copy.copy(options)) File "/usr/lib/python2.7/dist-packages/svtplay_dl/init.py", line 265, in get_one_media for i in streams: File "/usr/lib/python2.7/dist-packages/svtplay_dl/service/tv4play.py", line 52, in get data = self.http.request("get", url, cookies=self.cookies) File "/usr/lib/python2.7/dist-packages/svtplay_dl/utils/init.py", line 68, in request res = Session.request(self, method, url, verify=self.verify, proxies=self.proxy, *args, kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 487, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='prima.tv4play.se', port=80): Max retries exceeded with url: /api/web/asset/3934584/play (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x75eb1e30>: Failed to establish a new connection: [Errno -5] No address associated with hostname',))

Sopor commented 6 years ago

The new svtplay-dl will only work with python 3.x and i can see you are using 2.7 so you are not using the latest master. You can see here that master is working:

$ svtplay-dl https://www.tv4play.se/program/tareq-taylors-nordiska-matresa/3934584
INFO: Selected to download hls, bitrate: 4700
INFO: Outfile: tareq.taylors.nordiska.matresa.s04e10.tareq.taylors.nordiska.matresa.del.10-3934584-tv4play.ts
[009/436][==............................................................................................................] ETA: 0:01:45
Sopor commented 6 years ago

If you can't wait for svtplay-dl 2.0 you are free to clone the branch and compile the master. Take it easy with @spaam and appreciate his work instead of complaining. I have seen developers abandon their works because of rude users.

wahboh commented 6 years ago

@spaam @sopor Never ever I intended disrespect for the fine work @spaam and others have done for the svtplay-dl. I'm ever so grateful!! I'm painfully aware of the situation of volountary work and engaged users without patience.

Having stated that with great respect, I think some user frustration could be avoided with just a little bit of better information, and with also the users situation in mind.

You just provided two pieces of info that - to me - was not easy to find.

"The new svtplay-dl will only work with python 3x"... Which version of svtplay-dl are you refering to? Latest in github? Or 2.0?

The (linux) package that is preferred and which which I assumed was "latest of everything" still downloads 2.7?

The case about tv4play was alive for some time, then just closed as solved. But no further info.

I think just two lines on the home page about current status, what happens next, and what to do on my own if I can't wait for an upcoming release (at an undisclosed later date), would be very appreciated by a VERY thankfull and loyal user community.

hamkg commented 6 years ago

no problems with svtplay.se or viafree.se but tv4play still fails although I've installed the latest master. tv4play.se works fine with youtube-dl (not the subs though, but that's another matter). From the error message it may look as if I run Python2 (mentioning "File "/usr/lib64/python2.7/site-packages/" and so on) but: 1) I have python 3.6 installed 2) svtplay-dl installed with command python3 setup.py install 3) svtplay-dl is found in /usr/lib64/python-3.6/site-packages and not in /usr/lib64/python-2.7

bash-4.4$ svtplay-dl --verbose https://www.tv4play.se/program/badhotellet/3961521 DEBUG [1531131484.18] /usr/local/bin/svtplay-dl/svtplay_dl/init.py/get_media: version: -unknown DEBUG [1531131484.18] /usr/local/bin/svtplay-dl/svtplay_dl/utils/init.py/request: HTTP getting 'https://www.tv4play.se/program/badhotellet/3961521' DEBUG [1531131484.53] /usr/local/bin/svtplay-dl/svtplay_dl/utils/init.py/request: HTTP getting u'http://prima.tv4play.se/api/web/asset/3961521/play' Traceback (most recent call last): File "/usr/lib64/python2.7/runpy.py", line 174, in _run_module_as_main "main", fname, loader, pkg_name) File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/bin/svtplay-dl/main.py", line 13, in File "/usr/local/bin/svtplay-dl/svtplay_dl/init.py", line 513, in main File "/usr/local/bin/svtplay-dl/svtplay_dl/init.py", line 219, in get_media File "/usr/local/bin/svtplay-dl/svtplay_dl/init.py", line 267, in get_one_media File "/usr/local/bin/svtplay-dl/svtplay_dl/service/tv4play.py", line 52, in get File "/usr/local/bin/svtplay-dl/svtplay_dl/utils/init.py", line 68, in request File "/usr/lib64/python2.7/site-packages/requests/sessions.py", line 508, in request resp = self.send(prep, send_kwargs) File "/usr/lib64/python2.7/site-packages/requests/sessions.py", line 618, in send r = adapter.send(request, kwargs) File "/usr/lib64/python2.7/site-packages/requests/adapters.py", line 508, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='prima.tv4play.se', port=80): Max retries exceeded with url: /api/web/asset/3961521/play (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb5220438d0>: Failed to establish a new connection: [Errno -5] No address associated with hostname',))

hamkg commented 6 years ago

Firstly: my sincere thanks to Spaam for struggling with the obstacles created by tv4play.

After installing the very latest master (2018-07-12), downloading from tv4play works again - as long as I skip the subtitles. However, when adding the -S parameter I get the following (I guess that the last four lines are the crucial ones):

bash-4.4$ svtplay-dl -S https://www.tv4play.se/program/badhotellet/3961521 Traceback (most recent call last): File "/usr/bin/svtplay-dl", line 4, in import('pkg_resources').run_script('svtplay-dl==1.9.11+176.gc97ec90', 'svtplay-dl') File "/usr/lib64/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib64/python3.6/site-packages/pkg_resources/init.py", line 1438, in run_script exec(code, namespace, namespace) File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/EGG-INFO/scripts/svtplay-dl", line 7, in svtplay_dl.main() File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/init.py", line 63, in main get_media(urls[0], config, version) File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 56, in get_media get_one_media(stream) File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 157, in get_one_media options_subs_dl(subfixes) File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 152, in options_subs_dl subs[0].download() File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/subtitle/init.py", line 58, in download data = self.wrstsegment(subdata) File "/usr/lib64/python3.6/site-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/subtitle/init.py", line 277, in wrstsegment if len(subs) > 1 and itmes[1] == subs[-1][1]: # This will happen when there is two sections in file IndexError: list index out of range

Gurkan911 commented 6 years ago

I have the same problem as hamkg when trying to download the latest episode of Badhotellet (E06) but if I download E05 or older it works:

$ svtplay-dl -S --force-subtitle https://www.tv4play.se/program/badhotellet/3961521 Traceback (most recent call last): File "/usr/local/bin/svtplay-dl", line 4, in import('pkg_resources').run_script('svtplay-dl==1.9.11+176.gc97ec90', 'svtplay-dl') File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1438, in run_script exec(code, namespace, namespace) File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/EGG-INFO/scripts/svtplay-dl", line 7, in svtplay_dl.main() File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/init.py", line 63, in main get_media(urls[0], config, version) File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 56, in get_media get_one_media(stream) File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 157, in get_one_media options_subs_dl(subfixes) File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/utils/getmedia.py", line 152, in options_subs_dl subs[0].download() File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/subtitle/init.py", line 58, in download data = self.wrstsegment(subdata) File "/usr/local/lib/python3.6/dist-packages/svtplay_dl-1.9.11+176.gc97ec90-py3.6.egg/svtplay_dl/subtitle/init.py", line 277, in wrstsegment if len(subs) > 1 and itmes[1] == subs[-1][1]: # This will happen when there is two sections in file IndexError: list index out of range

$ svtplay-dl -S --force-subtitle https://www.tv4play.se/program/badhotellet/3961520 INFO: Outfile: badhotellet.s05e05.badhotellet.del.5-3961520-tv4play.srt

SweDennis commented 6 years ago

I've tried using the latest version from from the repository, as many seem to be running on that, and I can confirm that tv4play now works for me, again.

bpmildh commented 6 years ago

I got exactly the same results as @Gurkan911 with Badhotellet where episode 4-5 worked but 6 threw an error. This program will be gone in a few days though.

hamkg commented 6 years ago

My guess is that episode 5 was the last before tv4play switched to webvtt subfiles embedded in the m3u8 file. Those subs can be extracted with the program m3u8x (Windows only, requires Chrome browser). Hopefully a solution can be built into svtplay-dl in the near future.

Bengt-H commented 6 years ago

Apart from the TV4 subtitle problem i have also found an audio sync problem. If i remove the --subtitles option i can download TV4 programs without the program crashing.

python C:\svtplay-dl\svtplay-dl --q=2500 --flexible-quality=120 --output="badhotellet-s05e06" "https://www.tv4play.se/program/badhotellet/3961521"
INFO: Selected to download hls, bitrate: 2578
INFO: Outfile: badhotellet-s05e06.audio.ts
[881/881][========================================] ETA: 0:00:00
INFO: Outfile: badhotellet-s05e06.ts
[880/880][========================================] ETA: 0:00:00
Merge audio and video into badhotellet-s05e06.ts
INFO: Merge audio and video into badhotellet-s05e06.ts
Merging done, removing old files.
INFO: Merging done, removing old files.
Muxing badhotellet-s05e06.ts into badhotellet-s05e06.mp4
INFO: Muxing badhotellet-s05e06.ts into badhotellet-s05e06.mp4
Muxing done, removing the old file.
INFO: Muxing done, removing the old file.

The resulting video looks ok at the beginning but the audio and video rapidly grows out of sync and become unwatchable. If i download from other sites, like dplay or viafree it works fine.

I'm using Python 3.4.4 and svtplay-dl 2.0 and FFMPEG 3.3 on Windows XP SP3 (Pleas dont give me the usual XP-rant. I have the same problems with svtplay-dl and TV4 on my Windows 10 laptop)

Apart from the problems with TV4 and the nuisance that comes with the new python3-theshold the 2.0 release seem to work fine so far. Thanks for the effort.

Sopor commented 6 years ago

I can confirm @Bengt-H problem. It will be totally out of sync. If i download `https://www.tv4play.se/program/unga-f%C3%B6r%C3%A4ldrar/3965436´ it seems to work, no sync issues.

spaam commented 6 years ago

@Sopor- can you test https://github.com/spaam/svtplay-dl/commit/ea667b76551ef70ea5dc66102cb64787d529a6df if that commit works better ? the commit after i removed -coptyts from remux.

wahboh commented 6 years ago

Dear @spaam

Many thanks for your hard and dedicated work!

I downloaded Windows version2.0 and tried some links to TV4play. Some worked (probably released before the switch) but those after failed when using the --subtitle flag.

Those who succeded had a longer setup time than before, before they started downloadning.

Could this dump be of any assistance?

https://www.tv4play.se/program/agatha-christie-huset-p%C3%A5-udden/3973432

Traceback (most recent call last): File "C:\Python35\lib\site-packages\cx_Freeze\initscripts__startup.py", line 14, in run File "C:\Python35\lib\site-packages\cx_Freeze\initscripts\Console.py", line 26, in run File "bin/svtplay-dl", line 7, in File "c:\projects\svtplay-dl\lib\svtplay_dl__init__.py", line 63, in main File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 56, in get_media File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 157, in get_one_media File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 152, in options_subs_dl File "c:\projects\svtplay-dl\lib\svtplay_dl\subtitle__init__.py", line 58, in download File "c:\projects\svtplay-dl\lib\svtplay_dl\subtitle\init__.py", line 291, in wrstsegment IndexError: list index out of range

https://www.tv4play.se/program/badhotellet/3961521

Traceback (most recent call last): File "C:\Python35\lib\site-packages\cx_Freeze\initscripts__startup.py", line 14, in run File "C:\Python35\lib\site-packages\cx_Freeze\initscripts\Console.py", line 26, in run File "bin/svtplay-dl", line 7, in File "c:\projects\svtplay-dl\lib\svtplay_dl__init__.py", line 63, in main File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 56, in get_media File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 157, in get_one_media File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 152, in options_subs_dl File "c:\projects\svtplay-dl\lib\svtplay_dl\subtitle__init__.py", line 58, in download File "c:\projects\svtplay-dl\lib\svtplay_dl\subtitle\init__.py", line 277, in wrstsegment IndexError: list index out of range

https://www.tv4play.se/program/badhotellet/3961520 = OK Download

https://www.tv4play.se/program/badhotellet/3961519 = OK Download

BTW - are their many foreign users here, or only Swedes..?

spaam commented 6 years ago

@wahboh thanks for the links. i will look at that later today or tomorrow. In this issue it might only be swedes :)

Sopor commented 6 years ago

@spaam

$ git reset --hard ea667b76551ef70ea5dc66102cb64787d529a6df
HEAD is now at ea667b7 Update set a good version when build windows binaries
$ sudo python3 setup.py install

$ svtplay-dl https://www.tv4play.se/program/badhotellet/3961521
INFO: Selected to download hls, bitrate: 4698
INFO: Outfile: badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.audio.ts
[881/881][==============================================================================================================] ETA: 0:00:00
INFO: Outfile: badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts
[880/880][==============================================================================================================] ETA: 0:00:00
Merge audio and video into badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts
INFO: Merge audio and video into badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts
Merging done, removing old files.
INFO: Merging done, removing old files.
Muxing badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts into badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.mp4
INFO: Muxing badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts into badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.mp4
Muxing done, removing the old file.
INFO: Muxing done, removing the old file.

Unfortunately this file is also out of sync :(

spaam commented 6 years ago

@Sopor- what version of ffmpeg do you use? how fast do you detect the out of sync? i dont see any audio / video out of sync issue in mpv. but i only watched 2min. im using using ffmpeg 4.0.

Sopor commented 6 years ago

@spaam If i use ffmpeg 4.0.1 on Windows it seems to be working, no out of sync. I'm using an older ffmpeg on my media player, but it seems to be from 2018. ffmpeg version 3.2.11-1~deb9u1 Copyright (c) 2000-2018 the FFmpeg developers

Sopor commented 6 years ago

Before the introduction it is not out of sync, but at 1:20 it is out of sync. It is hard to know when they only plays music.

Sopor commented 6 years ago

If i take the two files (video and audio) badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.ts and badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.audio.ts and open them in MPC-HC 1.7.13 (64-bit) it is also out of sync.

Sopor commented 6 years ago

If i'm understand this right it has something to do with the ID3v1 at the end of the file. The demuxer sends the id3v1 tag at the end of the file as a data packet to the bitstream filter or decoder. and it seems to have been fixed for about a year ago. I did some testing and you need at least ffmpeg-3.4 to get rid of the out of sync problem @Bengt-H. You will find older versions here but i can't be sure they will run on XP.

Sopor commented 6 years ago

I couldn't find any newer ffmpeg for my Linux but i found compiled ffmpeg here and it seems to work.

@spaam You need to check for ffmpeg 4 or higher (3.4 will still give errors, so 4.x should be used) so this issue will be avoid in the future or else you probably will get more reports about video out of sync

Bengt-H commented 6 years ago

I strongly suspect that they have inserted something that will f*ck up the merging of the streams. Why dooes the audio stream have one more block than the video stream?? On other sites, that dont have this issues the audio and the video streams usually have the same number of blocks.

python C:\svtplay-dl\svtplay-dl --q=2500 --flexible-quality=120 --output="badhotellet-s05e04" "https://www.tv4play.se/program/badhotellet/3961519"
INFO: Selected to download hls, bitrate: 2578
INFO: Outfile: badhotellet-s05e04.audio.ts
[881/881][========================================] ETA: 0:00:00
INFO: Outfile: badhotellet-s05e04.ts
[880/880][========================================] ETA: 0:00:00

python C:\svtplay-dl\svtplay-dl --q=1750 --flexible-quality=150 --subtitle --output="S09E11-varuhuset-fylls-av-karlek" "https://www.dplay.se/videos/ullared/ullared-911"
INFO: Outfile: S09E11-varuhuset-fylls-av-karlek.srt
INFO: Selected to download hls, bitrate: 1749
INFO: Outfile: S09E11-varuhuset-fylls-av-karlek.audio.ts
[253/253][========================================] ETA: 0:00:00
INFO: Outfile: S09E11-varuhuset-fylls-av-karlek.ts
[253/253][========================================] ETA: 0:00:00

By the way i upgraded my FFMPEG to 3.4.1 Open Source Software and Windows 32-bit FFmpeg for Windows XP but it didnt help much. In the resulting files the video got even more fucked up (gray screen about two seconds into the show).

Sopor commented 6 years ago

@Bengt-H Try ffmpeg 4.x, it works. ffmpeg 3.4 showed a gray screen but only if i jumped forward in the video. If i let i play it worked.

Bengt-H commented 6 years ago

I haven't found a FFMPEG ver. 4 that works in XP and i seriously doubt that this is the real reason why it doesn't work properly only for the streams from TV4Play. If it really is the solution it would be very interesting to know more of the underlying details. How about stop using FFMPEG and use AVConv instead?

Sopor commented 6 years ago

And now we can't test it with Badhotellet anymore because TV4 have removed the episodes :( But it seems that you can use svtplay-dl https://www.tv4play.se/program/border-patrol/3966171 instead because i used avconv instead of ffmpeg to download an episode and it is totally out of sync.

Sopor commented 6 years ago

I tried a newer version of avconv (12.3) and it seems to abort the muxing:

[mp4 @ 0x2161d00] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

Non-monotonous DTS in output stream 0:1; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
(this will continue up to 140)

[AVBSFContext @ 0x2160fe0] Error parsing ADTS frame header!
Error applying bitstream filters to an output packet for stream #0:1.
Muxing border.patrol.s09e10.border.patrol.del.10-3966171-tv4play.ts into border.patrol.s09e10.border.patrol.del.10-3966171-tv4play.mp4
INFO: Muxing border.patrol.s09e10.border.patrol.del.10-3966171-tv4play.ts into border.patrol.s09e10.border.patrol.del.10-3966171-tv4play.mp4
ERROR: Something went wrong: avconv version 12.3, Copyright (c) 2000-2018 the Libav developers
  built on Jul 20 2018 02:18:18 with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.4)
[mpegts @ 0x1f720e0] max_analyze_duration 5000000 reached
Input #0, mpegts, from 'border.patrol.s09e10.border.patrol.del.10-3966171-tv4play.ts':
  Duration: 00:22:35.28, start: 10.040000, bitrate: 4413 kb/s
  Program 16727
    Stream #0:0[0x21]: Video: h264 (Main) [[27][0][0][0] / 0x001B]
      yuv420p, tv, 1280x720 [PAR 1:1 DAR 16:9]
      25 fps, 90k tbn
Stream map '0:1' matches no streams.
Sopor commented 6 years ago

@Bengt-H This is why you need to use 4.x of ffmpeg. I found this hot tip when searching: Your ffmpeg is outdated. Update and it will automatically insert some bitstream filters (at least aac_adtstoasc, hevc_mp4toannexb, h264_mp4toannexb).

There is no reason to investigate this any further. If you want to download from TV4 you need to use ffmpeg 4 or higher.

spaam commented 6 years ago

one way to deal with this the out of sync issue with older versions of ffmpeg is to check the version. if the version is older than 4. we will only but the files together into the .ts format ( need to check if that works ), if it works it will hard fail when someone trying to merge subtitles into the file (.mp4).

maybe it will work if its .mkv instead of .mp4 (.ts to .mkv)? it could be a .ts to .mp4 issue only 🤔

re IndexError: list index out of range should be easy thing to fix.

hamkg commented 6 years ago

Tried svtplay-dl https://www.tv4play.se/program/border-patrol/3966171 and it downloaded and merged without errors. However, the audio disappeared completely after just 2 seconds.

Downloading with ffmpeg -i "https://lbs-usp-hls-vod.cmore.se/vod/79c93/c1jchkx00vb(3966171_ISMUSP).ism/c1jchkx00vb(3966171_ISMUSP).m3u8" -acodec copy -vcodec copy -absf aac_adtstoasc "Border patrol del 10.mp4" brought me the program with audio and no sync issues. Linux (Slackware) and ffmpeg-3.4.2

Sopor commented 6 years ago

@spaam This is what i get when trying to merge with ffmpeg 3.4

Input #1, aac, from 'badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.audio.ts':
  Duration: 00:42:57.47, bitrate: 98 kb/s
    Stream #1:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 98 kb/s
Output #0, mp4, to 'output45.mp4':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, progressive), 1280x720 [SAR 1:1 DAR 16:9]
, q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 98 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.audio.ts: Invalid data found when processing input
frame=66000 fps=26750 q=-1.0 Lsize= 1370387kB time=00:43:59.92 bitrate=4252.5kbits/s speed=1.07e+003x
video:1369661kB audio:35kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.050456%

badhotellet.s05e06.badhotellet.del.6-3961521-tv4play.audio.ts: Invalid data found when processing input I get sound 2-3 seconds and then no sound and it seems to be the same if i use mkv, mp4 or ts, but i'm not an expert on ffmpeg so maybe i'm doing something wrong?

tomaforn commented 6 years ago

About the sync problems, I've been fiddling a little with it on another project. It seems that some audio stream chunks cannot be joined/concatenated because if you do then the audio ends up longer than the video. And if you play the audio it contains tiny gaps after each chunk length.

So I think we need to find another way of concatenating those chunks or removing some part from them while doing so.

Edit: The players seems to handle this because they play chunk by chunk, both video and audio, so I guess it's only a problem for us trying to merge them into a single stream.

Bengt-H commented 6 years ago

Since i still prefer doing my dayly hacking on my XP computer i have been experimenting with different FFMPEG builds and versions that works with this OS. I still haven't found a version 4.x that works under XP. I however came upon this one: Making Zeranoe builds work under XP & XP builds by rogerdpack that seem to do the trick. What version it is i'm not really sure of (it reports itself as: ffmpeg version N-90713-g955fa23-Reino), but it works under XP and gives a perfectly synched movie.

python C:\svtplay-dl\svtplay-dl --q=2496 --flexible-quality=200 --output="agatha-christie-huset-pa-udden-s01e01" "https://www.tv4play.se/program/agatha-christie-huset-pA5-udden/3973432"
INFO: Selected to download hls, bitrate: 2578
INFO: Outfile: agatha-christie-huset-pa-udden-s01e01.audio.ts
[873/873][========================================] ETA: 0:00:00
INFO: Outfile: agatha-christie-huset-pa-udden-s01e01.ts
[872/872][========================================] ETA: 0:00:00
Merge audio and video into agatha-christie-huset-pa-udden-s01e01.ts
INFO: Merge audio and video into agatha-christie-huset-pa-udden-s01e01.ts
Merging done, removing old files.
INFO: Merging done, removing old files.
Muxing agatha-christie-huset-pa-udden-s01e01.ts into agatha-christie-huset-pa-udden-s01e01.mp4
INFO: Muxing agatha-christie-huset-pa-udden-s01e01.ts into agatha-christie-huset-pa-udden-s01e01.mp4
Muxing done, removing the old file.
INFO: Muxing done, removing the old file.

What's really missing now is the subtitles...

JohnnyJorgensen commented 6 years ago

I can confirm that the ffmpeg version Bengt-H mentions above solves the synchronisation problem. Hope that the subtitle problem can be fixed soon. :-)

CurveOS commented 6 years ago

I get this trying to download the clip for the music show Diggiloo:

[kristoffer@localhost Video]$ svtplay-dl --verbose https://www.tv4play.se/program/diggiloo/3973323 DEBUG [1532739439.6] /usr/lib/python2.7/site-packages/svtplay_dl/init.py/get_media: version: 1.9.11 DEBUG [1532739439.6] /usr/lib/python2.7/site-packages/svtplay_dl/utils/init.py/request: HTTP getting 'https://www.tv4play.se/program/diggiloo/3973323' DEBUG [1532739439.68] /usr/lib/python2.7/site-packages/svtplay_dl/utils/init.py/request: HTTP getting u'http://prima.tv4play.se/api/web/asset/3973323/play' Traceback (most recent call last): File "/bin/svtplay-dl", line 7, in svtplay_dl.main() File "/usr/lib/python2.7/site-packages/svtplay_dl/init.py", line 511, in main get_media(urls[0], options) File "/usr/lib/python2.7/site-packages/svtplay_dl/init.py", line 217, in get_media get_one_media(stream, copy.copy(options)) File "/usr/lib/python2.7/site-packages/svtplay_dl/init.py", line 265, in get_one_media for i in streams: File "/usr/lib/python2.7/site-packages/svtplay_dl/service/tv4play.py", line 52, in get data = self.http.request("get", url, cookies=self.cookies) File "/usr/lib/python2.7/site-packages/svtplay_dl/utils/init.py", line 68, in request res = Session.request(self, method, url, verify=self.verify, proxies=self.proxy, *args, kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 475, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 596, in send r = adapter.send(request, **kwargs) File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 487, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='prima.tv4play.se', port=80): Max retries exceeded with url: /api/web/asset/3973323/play (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdaf332bf10>: Failed to establish a new connection: [Errno -2] Name or service not known',)) [kristoffer@localhost Video]$ svtplay-dl --version 1.9.11

I use Mageia 6 and it has Python 3 in the repositories, but 2.7 is installed as standard. There are no other issues with Python so I'l wait before Python 3 becomes standard.

No chance of a patch?

Sopor commented 6 years ago

@CurveOS 1.9.11 is old. Use svtplay-dl 2.0 and to install python 3.x can't be that hard?

hamkg commented 6 years ago

It' true what Sopor writes that CurveOS's issues will be solved with svtplaydl 2 and python3. The problem with audio out of sync when downloading from tv4play will remain, though. It's great that Bengt-H has found a solution, but that seems to be windows-only. I'm running slackware linux, and in that environment there's still no tested build script for ffmpeg-4. I could try to compile it myself, but since I'm dependent on a working ffmepg for other applications I'm a bit cautious. The more so since it is possible to get perfect sync with ffmpeg-3.4.2 if I download using the command generated by Stefan Sundin's browser plugin "privatkopiera" (as mentioned in an earlier post). The subtitle issue is another matter. Obviously it is technically possible to catch and merge the webvtt fragments into a .srt file since the program m3u8x can do it. But again, it's windows-only and might be impossible to integrate into svtplay-dl.

wahboh commented 6 years ago

@hamkg Please excuse my ignorance, but how did you get the link to the m3u8 file you refer to in your earlier post? Is there a general way of finding that url?

__"Tried svtplay-dl https://www.tv4play.se/program/border-patrol/3966171 and it downloaded and merged without errors. However, the audio disappeared completely after just 2 seconds.

Downloading with ffmpeg -i "https://lbs-usp-hls-vod.cmore.se/vod/79c93/c1jchkx00vb(3966171_ISMUSP).ism/c1jchkx00vb(3966171_ISMUSP).m3u8" -acodec copy -vcodec copy -absf aac_adtstoasc "Border patrol del 10.mp4"__

hamkg commented 6 years ago

@wahboh: The plugin "privatkopiera" (for Firefox and/or Chrome) finds the m3u8 link and automatically adds the rest of the parameters, giving you a command that you can copy and paste in your terminal: https://stefansundin.github.io/privatkopiera/ Information/instructions in Swedish only AFAIK.

OldSchool297 commented 5 years ago

Tv4 had webvtt subtitles before and now they are streaming, maybe that's the problem? Does privatkopiera works with Tv4?

OldSchool297 commented 5 years ago

How to download subtitle from Tv4 with M3u8x?

OldSchool297 commented 5 years ago

Anyone can download from Tv4?

spaam commented 5 years ago

what error do get?

Sopor commented 5 years ago

It is working perfect for me running latest master

svtplay-dl.exe -S -P hls https://www.tv4play.se/program/helt-sjukt/11978410
INFO: Outfile: helt-sjukt.s03e02.helt.sjukt.del.2-11978410-tv4play.srt
INFO: Selected to download hls, bitrate: 3680
INFO: Outfile: helt-sjukt.s03e02.helt.sjukt.del.2-11978410-tv4play.audio.ts
[237/666][============================....................................................] ETA: 0:00:13
OldSchool297 commented 5 years ago

Använder svtplaydl 2.4

C:>C:\svtplay-dl\svtplay-dl.exe -o C:\svtplay-dl -f https://www.tv4play.se/prog ram/hem-till-g%C3%A5rden/11974145 Traceback (most recent call last): File "C:\Python35\lib\site-packages\cx_Freeze\initscripts__startup.py", lin e 40, in run File "C:\Python35\lib\site-packages\cx_Freeze\initscripts\Console.py", line 23 , in run File "bin/svtplay-dl", line 4, in File "c:\projects\svtplay-dl\lib\svtplay_dl\init__.py", line 10, in

File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\getmedia.py", line 10, in <m odule> File "c:\projects\svtplay-dl\lib\svtplay_dl\service__init__.py", line 10, in

File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\http.py", line 8, in File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\output.py", line 11, in File "c:\projects\svtplay-dl\lib\svtplay_dl\utils\text.py", line 2, in File "C:\Python35\lib\html\parser.py", line 13, in ImportError: No module named '_markupbase' C:\>
spaam commented 5 years ago

@OldSchool297 is that all the output? feels like some lines is missing