Issues with UStream's streaming protocol

chrippa / livestreamer

Command-line utility that extracts streams from various services and pipes them into a video player of choice. No longer maintained, use streamlink or youtube-dl instead.

http://livestreamer.io/

BSD 2-Clause "Simplified" License

3.88k stars 579 forks source link

Issues with UStream's streaming protocol #265

Open k-ross opened 10 years ago

k-ross commented 10 years ago

When trying to save a stream to a file, using a command line like the following:

livestreamer -o test.flv http://www.ustream.tv/creativelive best

The resulting test.flv file will often play audio, but no video. I try to play the file with vlc, mplayer, and mpv. They all give lots of errors on the command line about not being able to decode the video. I can give the exact output if needed.

If I omit the "-o test.flv" argument, and just let it try to play using VLC, it will sometimes work, but it will often have the same problem. Audio only, no video. If I stop and restart it, eventually I will get audio and video.

I can send some sample files that were saved, if needed.

BTW, I tried the "fix_incomplete_tags.py" script on one of my saved files, but it didn't help.

Thanks!

chrippa commented 10 years ago

I'm not able to reproduce this. I also get a few errors, but they don't cause any problems. Could you paste a log with the errors?

k-ross commented 10 years ago

Here's a simple run of "livestreamer -v http://www.ustream.tv/creativeLIVE-rebroadcast best"

http://pastebin.com/paxa7gY7

I had to use a different ustream channel, since they weren't currently broadcasting on the one in the original bug report. But I get the same results.

The first time I ran it, it worked. The second time, I got audio but no video. I let it play for about a minute before I killed it.

k-ross commented 10 years ago

This time I ran "livestreamer -v -o out.flv http://www.ustream.tv/creativeLIVE-rebroadcast best"

The file is available here (17 MB download): https://mail.familyross.net/out.flv

The output on the console was:

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creativeLIVE-rebroadcast [cli][info] Opening stream: live Written 17938978 bytes^C

I let it run for a minute or so, then killed it with Ctrl+C

I am not able to get video in vlc, mplayer, or mpv when I try to play the out.flv file.

k-ross commented 10 years ago

I'm starting to suspect this might be a bug in ffmpeg/libav. I haven't been able to remux the files using ffmeg or avconv, nor play them using vlc, mplayer, or mpv, but those are all based on ffmpeg or libav. Sometimes gstreamer-based totem player will work where vlc or mplayer won't.

I've found a way to remux the files using something not based on ffmpeg or libav. FLVExtract running on Windows will remux the files into an mkv file without issue, and then the files are playable in VLC.

So at least I can repair the files after the fact, if necessary. I just have to use a Windows VM when I'm running on Linux. I may take this up with the ffmpeg or libav people, unless you think there's something in livestreamer that can be tweaked.

Thanks!

chrippa commented 10 years ago

Well, the fact that is works sometimes but not always may mean my implementation of their streaming protocol is not correct. I was able to reproduce the issue on the second channel you posted. This channel works consistently for me though, is it the same for you?

k-ross commented 10 years ago

Yes, I tried about 10 times in a row, and it worked each time.

ankitgokani commented 10 years ago

Hey Guys.. i have a similar problem

I have the stream i used below.. it writes file for sometime properly but later it starts giving this error.. And my windows explorer also crashes sometimes doing it.. It would be great if someone can help

c:\Users~\Desktop>livestreamer -v -o Test.flv http://www.ustream.tv/channel/4 307895 360p [cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/channel/ 4307895 [cli][info] Opening stream: 360p Written 3242683 bytes[stream.uhs][error] Disconnected, attempting to reconnect Written 4289523 bytes[stream.uhs][error] [1390439862] Failed to open chunk: Unable to open URL: http://uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVi deo/4307895/streams/live_5_1390439862_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439862] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439862_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439862] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439862_351060299.flv (404 Client Error: Not Found) Written 4802767 bytes[stream.uhs][error] [1390439864] Failed to open chunk: Unable to open URL: http://uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVi deo/4307895/streams/live_5_1390439864_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439864] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439864_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439864] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439864_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439865] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439865_351060299.flv (404 Client Error: Not Found) [stream.uhs][error] [1390439865] Failed to open chunk: Unable to open URL: http:// uhs-akamai.ustream.tv/sjc/sjc-uhs09/streams/httpflv/ustreamVideo/4307895/streams/l ive_5_1390439865_351060299.flv (404 Client Error: Not Found)

chrippa commented 10 years ago

I've pushed some changes to the ustream streaming which I believe should fix most issues, please try the latest git version.

(Windows builds can be found here: http://livestreamer-builds.s3.amazonaws.com/builds.html)

k-ross commented 10 years ago

Great, I'll give it a try over the next couple days.

Thanks!

RuKeBo commented 10 years ago

I tried this with the build "livestreamer-v1.7.5-65-g752eaaa-win32.zip" and it appeared to work ok for awhile. But after maybe 45 minutes, it gave me the following errors (same error several times):

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creati veLIVE-rebroadcast [cli][info] Opening stream: 720p Written 287375499 bytes[stream.uhs][error] Failed to open chunk 1395191177: Unab le to open URL: http://uhs-live-default.ustream.tv/sjc-uhs23/streams/httpflv/ust reamVideo/952665/streams/live_1_1395191177_1155237609.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395191177: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs23/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395191177_1155237609.flv (404 Client Error: Not Found) etc....this occurred around 12 times.

The URL it lists is not a valid URL, not sure where it got it from.

I would then get the following error when I tried to play it with VLC: No suitable decoder module: VLC does not support the audio or video format "undf". Unfortunately there is no way for you to fix this.

The file will play audio, but no video.

I need to do some more testing to see if this is a time dependent thing, I know it happened as soon as I started livestreamer one time, but not usually. I was outputting to a file, not to VLC directly. Funny thing is, I was using RTMPDump and RTMPSuck for awhile, but Usteam must have changed something recently since RMTPSuck won't work on it anymore but livestreamer does, to a point. I can't find the command line string for RTMPDump.

Also, not sure if this is normal, but the "Written" size increases fast for a short time, then stops for some time. It appears that when this stops, the file gets written to. I believe this may cause the end of the output file to get truncated when the stream ends since it exits before the file is fully written. I still need to test this more.

Thanks

RuKeBo commented 10 years ago

Here is the output from another stream on ustream.tv:

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/ciscol ive [plugin.ustreamtv][error] Unable to fetch desktop streams: Failed to connect to RTMP server [cli][info] Opening stream: mobile_480p Written 93779664 bytes[stream.hls][warning] Failed to reload playlist: Unable to open URL: http://iphone-streaming.ustream.tv/watch/playlist.m3u8?cid=3358636&st ream=live_5&appType=103&appVersion=3&conn=wifi&group=iphone (HTTPConnectionPool( host='iphone-streaming.ustream.tv', port=80): Max retries exceeded with url: /wa tch/playlist.m3u8?cid=3358636&stream=live_5&appType=103&appVersion=3&conn=wifi&g roup=iphone (Caused by <class 'httplib.BadStatusLine'>: '')) Written 100365492 bytes

Also not sure why the quality is being detected so low. At home (slower connection), I get 720P on most streams but at work they are only "mobile_240p" or "mobile_480p".

chrippa commented 10 years ago

I would then get the following error when I tried to play it with VLC: No suitable decoder module: VLC does not support the audio or video format "undf". Unfortunately there is no way for you to fix this.

The file will play audio, but no video.

I believe what is happening is that sometimes a segment will only contain audio, and even if the next segment contains video it will end up confusing at least ffmpeg based players since it seems to rely on scanning the first few FLV tags to see what it contains.

We might be able to work around this by analyzing the stream and not outputting segments until we found one which contains both audio and video.

I need to do some more testing to see if this is a time dependent thing, I know it happened as soon as I started livestreamer one time, but not usually. I was outputting to a file, not to VLC directly. Funny thing is, I was using RTMPDump and RTMPSuck for awhile, but Usteam must have changed something recently since RMTPSuck won't work on it anymore but livestreamer does, to a point. I can't find the command line string for RTMPDump.

UStream has pretty much dropped the usual RTMP streams and are now using a special RTMP + segmented HTTP stream type which Livestreamer attempts to implement. But the implementation is based on reverse engineering the protocol so it's not perfect yet.

Also, not sure if this is normal, but the "Written" size increases fast for a short time, then stops for some time. It appears that when this stops, the file gets written to. I believe this may cause the end of the output file to get truncated when the stream ends since it exits before the file is fully written. I still need to test this more.

The start and stop is happening because the UStream server gives us a list of FLV segments in bursts every few seconds, so it is not because of disk I/O and no data should be lost.

Also not sure why the quality is being detected so low. At home (slower connection), I get 720P on most streams but at work they are only "mobile_240p" or "mobile_480p".

This error

[plugin.ustreamtv][error] Unable to fetch desktop streams: Failed to connect to RTMP server

seem to indicate that your work firewall is probably blocking the connection to UStream's RTMP server which is required to access the streams normally found in the web player.

RuKeBo commented 10 years ago

Let me know if you need me to test something

chrippa commented 10 years ago

I believe the audio only issue should be fixed with 1816c69de0224e6a1d0a8688e6008f9aaea49d15. What happened was that sometimes the H264 header came a few seconds after the AAC header in a segment, this caused a desync which confused player software. We now wait for both headers to make it all synced up.

RuKeBo commented 10 years ago

Great. I will give it a try in a short while.

RuKeBo commented 10 years ago

I don't get the same error anymore, but it is locking up. Here are the error (Read timeout):

This is when saving to a file: [cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creativelive-3 [cli][info] Opening stream: 720p+ Written 119748705 bytes[cli][error] Error when reading from stream: Read timeout

This is when playing to VLC: livestreamer -v http://www.ustream.tv/creativelive-3 best [cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creativelive-3 [cli][info] Opening stream: 720p+ [cli][info] Starting player: 'C:\Program Files (x86)\VideoLAN\VLC\vlc.exe' [cli][error] Error when reading from stream: Read timeout

It doesn't get too far before this happens, 3 files went about 1:45 and another about 1:20.

I am running this in a batch file (when it saves to a file) and when it gets this error, it just stops, it does not exit livestreamer, which creates a problem for me since it can't continue in the batch file. I'm not sure if this error works differently since I think the other errors exited the program, but I could be wrong.

Thanks

chrippa commented 10 years ago

Please run with --loglevel debug in the future, it contains some extra information. I haven't been able to reproduce this myself, watched about 45 min of that stream.

RuKeBo commented 10 years ago

This time it ran a lot longer, about 28 minutes. I don't think you want the whole output so I just posted part of the end. The only thing I see is that when it ended, it seems to have been Adding more chunks to the queue than it did at any other time, 10 times before the error. The most it appears to have done previously was 6.

Written 408595599 bytes[stream.uhs][debug] Download of chunk 1395446356 complete

Written 408633523 bytes[stream.uhs][debug] Adding chunk 1395446357 to queue [stream.uhs][debug] Adding chunk 1395446358 to queue [stream.uhs][debug] Adding chunk 1395446359 to queue [stream.uhs][debug] Adding chunk 1395446360 to queue Written 409990696 bytes[stream.uhs][debug] Download of chunk 1395446357 complete

Written 411421884 bytes[stream.uhs][debug] Download of chunk 1395446358 complete

Written 412798713 bytes[stream.uhs][debug] Download of chunk 1395446359 complete

Written 414198233 bytes[stream.uhs][debug] Download of chunk 1395446360 complete

Written 414205803 bytes[stream.uhs][debug] Adding chunk 1395446361 to queue [stream.uhs][debug] Adding chunk 1395446362 to queue [stream.uhs][debug] Adding chunk 1395446363 to queue [stream.uhs][debug] Adding chunk 1395446364 to queue [stream.uhs][debug] Adding chunk 1395446365 to queue [stream.uhs][debug] Adding chunk 1395446366 to queue Written 415458971 bytes[stream.uhs][debug] Download of chunk 1395446361 complete

Written 416687319 bytes[stream.uhs][debug] Download of chunk 1395446362 complete

Written 418116440 bytes[stream.uhs][debug] Download of chunk 1395446363 complete

Written 419709465 bytes[stream.uhs][debug] Download of chunk 1395446364 complete

Written 421137882 bytes[stream.uhs][debug] Download of chunk 1395446365 complete

Written 422502083 bytes[stream.uhs][debug] Download of chunk 1395446366 complete

Written 422578353 bytes[stream.uhs][debug] Adding chunk 1395446367 to queue [stream.uhs][debug] Adding chunk 1395446368 to queue [stream.uhs][debug] Adding chunk 1395446369 to queue [stream.uhs][debug] Adding chunk 1395446370 to queue Written 423974756 bytes[stream.uhs][debug] Download of chunk 1395446367 complete

Written 425174095 bytes[stream.uhs][debug] Download of chunk 1395446368 complete

Written 426494705 bytes[stream.uhs][debug] Download of chunk 1395446369 complete

Written 427889143 bytes[stream.uhs][debug] Download of chunk 1395446370 complete

Written 427898913 bytes[stream.uhs][debug] Adding chunk 1395446371 to queue [stream.uhs][debug] Adding chunk 1395446372 to queue [stream.uhs][debug] Adding chunk 1395446373 to queue [stream.uhs][debug] Adding chunk 1395446374 to queue [stream.uhs][debug] Adding chunk 1395446375 to queue [stream.uhs][debug] Adding chunk 1395446376 to queue Written 429254291 bytes[stream.uhs][debug] Download of chunk 1395446371 complete

Written 430628123 bytes[stream.uhs][debug] Download of chunk 1395446372 complete

Written 432122751 bytes[stream.uhs][debug] Download of chunk 1395446373 complete

Written 433705815 bytes[stream.uhs][debug] Download of chunk 1395446374 complete

Written 434094352 bytes[stream.uhs][debug] Adding chunk 1395446377 to queue [stream.uhs][debug] Adding chunk 1395446378 to queue [stream.uhs][debug] Adding chunk 1395446379 to queue [stream.uhs][debug] Adding chunk 1395446380 to queue [stream.uhs][debug] Adding chunk 1395446381 to queue [stream.uhs][debug] Adding chunk 1395446382 to queue [stream.uhs][debug] Adding chunk 1395446383 to queue [stream.uhs][debug] Adding chunk 1395446384 to queue [stream.uhs][debug] Adding chunk 1395446385 to queue [stream.uhs][debug] Adding chunk 1395446386 to queue [cli][error] Error when reading from stream: Read timeout

RuKeBo commented 10 years ago

Ran this again, stopped rather quickly, but Added two more chunks after the "Read Timeout" error. Not sure this means anything, but the total of the Add chunks before and after the error was 10 again.

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creativelive-3 [plugin.ustreamtv][debug] Waiting for moduleInfo invoke [cli][info] Opening stream: 720p+ [cli][debug] Pre-buffering 8192 bytes [stream.uhs][debug] Adding chunk 1395446458 to queue [stream.uhs][debug] Adding chunk 1395446459 to queue [stream.uhs][debug] Adding chunk 1395446460 to queue [stream.uhs][debug] Adding chunk 1395446461 to queue [cli][debug] Checking file output [cli][debug] Writing stream to output Written 1328440 bytes[stream.uhs][debug] Download of chunk 1395446458 complete Written 2753940 bytes[stream.uhs][debug] Download of chunk 1395446459 complete Written 4158844 bytes[stream.uhs][debug] Download of chunk 1395446460 complete Written 5586190 bytes[stream.uhs][debug] Download of chunk 1395446461 complete Written 5602029 bytes[stream.uhs][debug] Adding chunk 1395446462 to queue [stream.uhs][debug] Adding chunk 1395446463 to queue [stream.uhs][debug] Adding chunk 1395446464 to queue [stream.uhs][debug] Adding chunk 1395446465 to queue [stream.uhs][debug] Adding chunk 1395446466 to queue Written 7019040 bytes[stream.uhs][debug] Download of chunk 1395446462 complete Written 8411942 bytes[stream.uhs][debug] Download of chunk 1395446463 complete Written 9688838 bytes[stream.uhs][debug] Download of chunk 1395446464 complete Written 11075123 bytes[stream.uhs][debug] Download of chunk 1395446465 complete Written 12314113 bytes[stream.uhs][debug] Download of chunk 1395446466 complete Written 12338020 bytes[stream.uhs][debug] Adding chunk 1395446467 to queue [stream.uhs][debug] Adding chunk 1395446468 to queue [stream.uhs][debug] Adding chunk 1395446469 to queue [stream.uhs][debug] Adding chunk 1395446470 to queue Written 12821730 bytes[stream.uhs][debug] Adding chunk 1395446471 to queue [stream.uhs][debug] Adding chunk 1395446472 to queue [stream.uhs][debug] Adding chunk 1395446473 to queue [stream.uhs][debug] Adding chunk 1395446474 to queue [stream.uhs][debug] Adding chunk 1395446475 to queue [stream.uhs][debug] Adding chunk 1395446476 to queue [stream.uhs][debug] Adding chunk 1395446477 to queue [stream.uhs][debug] Adding chunk 1395446478 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][debug] Adding chunk 1395446479 to queue [stream.uhs][debug] Adding chunk 1395446480 to queue

RuKeBo commented 10 years ago

Always seems to be 10:

Written 203570976 bytes[stream.uhs][debug] Adding chunk 1395446747 to queue [stream.uhs][debug] Adding chunk 1395446748 to queue [stream.uhs][debug] Adding chunk 1395446749 to queue [stream.uhs][debug] Adding chunk 1395446750 to queue [stream.uhs][debug] Adding chunk 1395446751 to queue [stream.uhs][debug] Adding chunk 1395446752 to queue [stream.uhs][debug] Adding chunk 1395446753 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][debug] Adding chunk 1395446754 to queue [stream.uhs][debug] Adding chunk 1395446755 to queue [stream.uhs][debug] Adding chunk 1395446756 to queue

RuKeBo commented 10 years ago

I am not sure any of this is helping, but I ran three instances of livestreamer. Was curious if they would throw an error at the same time. The second instance was started within seconds of the first. The third was started 20-30 seconds after that. Test1 & Test2 added the same chunks and wrote the stream at the same time (close to it), they basically outputted the same thing. The third one also added the same chunks to queue but obviously the file written was different. They all outputted adding the same chunks at the same time minus some lag between them. I assumed the chunks would be different depending on when they were started, but I don't know anything about how this works, just an interesting observation. I left them to run and the end of the output is below. You can see how they all ended differently, I only posted the end of the output. I assume this means that it is sort of random as to when it gets the "Read timeout" error", they did not all error at the same time as I expected. All end with 10 "Adding chunk to queue" again.

Test1 - Started first Written 1458799740 bytes[stream.uhs][debug] Adding chunk 1395448207 to queue [stream.uhs][debug] Adding chunk 1395448208 to queue [stream.uhs][debug] Adding chunk 1395448209 to queue [stream.uhs][debug] Adding chunk 1395448210 to queue [stream.uhs][debug] Adding chunk 1395448211 to queue [stream.uhs][debug] Adding chunk 1395448212 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][debug] Adding chunk 1395448213 to queue [stream.uhs][debug] Adding chunk 1395448214 to queue [stream.uhs][debug] Adding chunk 1395448215 to queue [stream.uhs][debug] Adding chunk 1395448216 to queue

Test2 - Start a few seconds after Test1 Written 1063221731 bytes[stream.uhs][debug] Adding chunk 1395447921 to queue [stream.uhs][debug] Adding chunk 1395447922 to queue [stream.uhs][debug] Adding chunk 1395447923 to queue [stream.uhs][debug] Adding chunk 1395447924 to queue [stream.uhs][debug] Adding chunk 1395447925 to queue [stream.uhs][debug] Adding chunk 1395447926 to queue [stream.uhs][debug] Adding chunk 1395447927 to queue [stream.uhs][debug] Adding chunk 1395447928 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][debug] Adding chunk 1395447929 to queue [stream.uhs][debug] Adding chunk 1395447930 to queue

Test 3 - Started 20-30 seconds after Test1 Written 2366752782 bytes[stream.uhs][debug] Adding chunk 1395448867 to queue [stream.uhs][debug] Adding chunk 1395448868 to queue [stream.uhs][debug] Adding chunk 1395448869 to queue [stream.uhs][debug] Adding chunk 1395448870 to queue [stream.uhs][debug] Adding chunk 1395448871 to queue [stream.uhs][debug] Adding chunk 1395448872 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][debug] Adding chunk 1395448873 to queue [stream.uhs][debug] Adding chunk 1395448874 to queue [stream.uhs][debug] Adding chunk 1395448875 to queue [stream.uhs][debug] Adding chunk 1395448876 to queue

RuKeBo commented 10 years ago

Here I got an additional error. I ran two instances, this one died fairly quickly, the other is still going. The one that is till going did appear to pause for some time at the same time the other died.

I also started another livestreamer instance about 10 minutes after the first and they both ended in error at the same time...so sometimes they can end together. I ran one instance, then another, the second one died, then I closed the 2nd, started a third, and they ended the same time.

Written 44369774 bytes[stream.uhs][debug] Adding chunk 1395455337 to queue [stream.uhs][debug] Adding chunk 1395455338 to queue [stream.uhs][debug] Adding chunk 1395455339 to queue [stream.uhs][debug] Adding chunk 1395455340 to queue [stream.uhs][debug] Adding chunk 1395455341 to queue [stream.uhs][debug] Adding chunk 1395455342 to queue [stream.uhs][debug] Adding chunk 1395455343 to queue [stream.uhs][debug] Adding chunk 1395455344 to queue [stream.uhs][debug] Adding chunk 1395455345 to queue [stream.uhs][debug] Adding chunk 1395455346 to queue [cli][error] Error when reading from stream: Read timeout

[stream.uhs][error] Failed to get module info: Failed to read packet

chrippa commented 10 years ago

It looks like the thread that is responsible for fetching the segments gets stuck. Most likely what happens is that the HTTP request never times out, which is probably caused by a bug currently existing in the HTTP library we use (requests) where the timeout is not applied to streaming responses (it will be fixed in the next version though).

I've pushed a workaround that will attempt to downlaod the whole segment in one go which should make the timeout work correctly. Downside with this approach is that we have to wait for each segment to be fully downloaded before it can be sent to the player.

RuKeBo commented 10 years ago

I see how this works differently from the output. This does work better, I do still get an error at times, it does not lockup though, it keeps going. The video file that is saved stops at points then continues by jumping several seconds (9-15) ahead. I assume this is because it does not download that segment, or segments, because of the error.

Written 233762976 bytes[stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395504961 to queue Written 234428929 bytes[stream.uhs][debug] Download of chunk 1395504961 complete

Written 234441060 bytes[stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395504962 to queue [stream.uhs][error] Failed to open chunk 1395504962: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395504962_432384009.flv (HTTPConnectionPool(host='uhs-live-default.ust ream.tv', port=80): Max retries exceeded with url: /sjc-uhs16/streams/httpflv/us treamVideo/952665/streams/live_1_1395504962_432384009.flv (Caused by <class 'htt plib.BadStatusLine'>: '')) Written 234842468 bytes[stream.uhs][debug] Download of chunk 1395504962 complete

Written 235166189 bytes[stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395504963 to queue Written 235875675 bytes[stream.uhs][debug] Download of chunk 1395504963 complete

chrippa commented 10 years ago

You might want to try using an alternative CDN if you're having trouble with many missing segments, e.g. 720p+_alt_akamai. It's suprising you would end up with missing segments in the first place though, as it attempts to download each segment 3 times. I have never seen a segment download fail more than once.

RuKeBo commented 10 years ago

I will try that. Here is some more output without debug option if it helps:

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creati veLIVE-rebroadcast [cli][info] Opening stream: 720p+ Written 4350089 bytes[stream.uhs][error] Failed to open chunk 1395509151: Unable to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustre amVideo/952665/streams/live_1_1395509151_145051358.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509151_145051358.flv ( Caused by <class 'httplib.BadStatusLine'>: '')) Written 4843604 bytes[stream.uhs][error] Failed to open chunk 1395509153: Unable to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustre amVideo/952665/streams/live_1_1395509153_145051358.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509153_145051358.flv ( Caused by <class 'httplib.BadStatusLine'>: '')) Written 5014265 bytes[stream.uhs][error] Failed to open chunk 1395509154: Unable to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustre amVideo/952665/streams/live_1_1395509154_623067244.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509154_623067244.flv ( Caused by <class 'httplib.BadStatusLine'>: '')) Written 5386652 bytes[stream.uhs][error] Failed to open chunk 1395509157: Unable to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustre amVideo/952665/streams/live_1_1395509157_623067244.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509157_623067244.flv ( Caused by <class 'httplib.BadStatusLine'>: '')) Written 5530098 bytes[stream.uhs][error] Failed to open chunk 1395509158: Unable to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustre amVideo/952665/streams/live_1_1395509158_623067244.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509158_623067244.flv ( Caused by <class 'httplib.BadStatusLine'>: '')) Written 12538390 bytes[stream.uhs][error] Failed to open chunk 1395509171: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509171_25117249.flv (HTTPConnectionPool(host= 'uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uhs 16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509171_25117249.flv (C aused by <class 'httplib.BadStatusLine'>: '')) Written 14110416 bytes[stream.uhs][error] Failed to open chunk 1395509176: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509176_821387706.flv (HTTPConnectionPool(host ='uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-uh s16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509176_821387706.flv (Caused by <class 'httplib.BadStatusLine'>: '')) Written 30403200 bytes[stream.uhs][error] Failed to open chunk 1395509197: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509197_1578006269.flv (HTTPConnectionPool(hos t='uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-u hs16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509197_1578006269.fl v (Caused by <class 'httplib.BadStatusLine'>: '')) Written 34622883 bytes[stream.uhs][error] Failed to open chunk 1395509203: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509203_1578006269.flv (HTTPConnectionPool(hos t='uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-u hs16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509203_1578006269.fl v (Caused by <class 'httplib.BadStatusLine'>: '')) Written 42820703 bytes[stream.uhs][error] Failed to open chunk 1395509214: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509214_1678008696.flv (HTTPConnectionPool(hos t='uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-u hs16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509214_1678008696.fl v (Caused by <class 'httplib.BadStatusLine'>: '')) Written 43557005 bytes[stream.uhs][error] Failed to open chunk 1395509215: Unabl e to open URL: http://uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustr eamVideo/952665/streams/live_1_1395509215_1678008696.flv (HTTPConnectionPool(hos t='uhs-live-default.ustream.tv', port=80): Max retries exceeded with url: /sjc-u hs16/streams/httpflv/ustreamVideo/952665/streams/live_1_1395509215_1678008696.fl v (Caused by <class 'httplib.BadStatusLine'>: '')) Written 47827388 bytes

RuKeBo commented 10 years ago

Using that option seems to be causing more issues, I show some output below. The way it was running before seems to be doing ok. It gives an error about not being able to open a chunk but the next time it tries it is able to download it and writes it. Using "720+_alt_akami" failed to open a chunk 3 times, for 3 different chunks. This is a new error using the "720+_alt_akami":

[cli][info] Found matching plugin ustreamtv for URL http://www.ustream.tv/creativeLIVE-rebroadcast [cli][info] Opening stream: 720p+_alt_akamai Written 342166872 bytesException in thread Thread-1: Traceback (most recent call last): File "threading.py", line 551, in __bootstrap_inner File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\stream\segmented.py", line 104, in run self.write(segment) File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\plugins\ustreamtv.py", line 95, in write skip_header=True): File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\stream\flvconcat.py", line 186, in iter_chunks for tag in self.iter_tags(fd=fd, buf=buf, skip_header=skip_header): File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\stream\flvconcat.py", line 178, in iter_tags for tag in chain(tags_iterator, flv_iterator): File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\stream\flvconcat.py", line 40, in iter_flv_tags tag, offset = Tag.deserialize_from(buf, offset, strict=strict) File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\packages\flashmedia\packet.py", line 18, in deserialize_from return cls._deserialize_from(buf, offset, **kw) File "c:\program files (x86)\livestreamer\livestreamer-1.7.5-py2.7.egg\livestreamer\packages\flashmedia\tag.py", line 289, in _deserialize_from tag_size = U32BE.unpack_from(buf, offset)[0] error: unpack_from requires a buffer of at least 4 bytes

[cli][error] Error when reading from stream: Read timeout

[cli][info] Stream ended

RuKeBo commented 10 years ago

This might be interesting. I had two instances of livestreamer running at the same time, one with "best" and the other with "720+_alt_akami". Both gave errors for the same chunks and it tried to open them 3 times but failed, it did this for several chunks. The errors for each were different though, maybe this is expected. The "best" option seems to work well for some time. It can go hours with just an error now and then for failing to open a chunk once, but it gets it the next try. Then all of the sudden it does what is shown below.

"720+_alt_akami" [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511570_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511570_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511570_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511574_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511574_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http: uhs-akamai.ustream.tv/sjc/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream live_1_1395511574_1610699976.flv (404 Client Error: Not Found) Written 4062461 bytes

"best" [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511570_1610699976.flv (HTTPConnectionPool(host='uhs-live-default.us tream.tv', port=80): Max retries exceeded with url: /sjc-uhs16/streams/httpflv/u streamVideo/952665/streams/live_1_1395511570_1610699976.flv (Caused by <class 'h ttplib.BadStatusLine'>: '')) [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511570_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511570: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511570_1610699976.flv (404 Client Error: Not Found) [stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395511571 to queue [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511571: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511571_1610699976.flv (404 Client Error: Not Found) [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395511572 to queue [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511572: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511572_1610699976.flv (404 Client Error: Not Found) [stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395511573 to queue [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511573: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511573_1610699976.flv (404 Client Error: Not Found) [stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395511574 to queue [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511574_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511574_1610699976.flv (404 Client Error: Not Found) [stream.uhs][error] Failed to open chunk 1395511574: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511574_1610699976.flv (404 Client Error: Not Found) [stream.uhs][debug] Fetching module info [stream.uhs][debug] Fetching module info [stream.uhs][debug] Adding chunk 1395511575 to queue [stream.uhs][error] Failed to open chunk 1395511575: Unable to open URL: http:// uhs-live-default.ustream.tv/sjc-uhs16/streams/httpflv/ustreamVideo/952665/stream s/live_1_1395511575_499902584.flv (HTTPConnectionPool(host='uhs-live-default.ust ream.tv', port=80): Max retries exceeded with url: /sjc-uhs16/streams/httpflv/us treamVideo/952665/streams/live_1_1395511575_499902584.flv (Caused by <class 'htt plib.BadStatusLine'>: '')) Written 844621 byte[stream.uhs][debug] Download of chunk 1395511575 complete

RuKeBo commented 10 years ago

Is there a way for me to implement the requests latest build into livestreamer to test this. I know nothing about Python but I tried it by downloaded the latest requests which appeared to have the "timeout" feature implemented in "adapters.py". I compiled the files in the requests folder and copied them to the livestreamer requests folder. I didn't notice anything different. Or does this need to be implemented somehow in livestreamer in a later release?

Using the 720p+ stream gives me a a lot of lost chunks. Using 720p+_alt_akamai works better, but it seems to always cut off the last 10 seconds or so of the stream if I save to a file. It also seems to loose the first few seconds (not sure how many, maybe 10) of the beginning. I restart livestreamer (in a loop) when the stream ends and use --retry-streams 1 until it picks it up again. This works pretty well, but sometimes, not always, it looses the end and/or beginning. I have also tried it without --retry-streams and just had it wait for 10 seconds before restarting. I have changed this time but it sometimes still misses the beginning. It is possible it is the stream. When using 720p+_alt_akamai, sometimes it will find a mobile stream but not the 720p stream for several tries.

I should try the above while viewing a second instance with VLC and see if that gets cut off as well.

chrippa commented 10 years ago

Are you referring to 5ebf5af0933f899ce7b7c3e3f73378ce9a50348e? The timeout stuff referred to there only affects streaming responses, which the ustream plugin does not currently use.

About your issues, it is likely my implementation is slightly wrong since it's based purely on analyzing the network traffic. I have to decompile the flash player to get a better look how this works, but haven't gotten around to it yet.

RuKeBo commented 10 years ago

I assumed that since you mentioned what is below, that the timeout issue would be solved. maybe it was a different time out issue.

It looks like the thread that is responsible for fetching the segments gets stuck. Most likely what happens is that the HTTP request never times out, which is probably caused by a bug currently existing in the HTTP library we use (requests) where the timeout is not applied to streaming responses (it will be fixed in the next version though).

I was referring to this: https://github.com/kennethreitz/requests/issues/1803, but it could be unrelated. It was just a timeout issue they were dealing with so I assumed it was the one you mentioned.

chrippa commented 10 years ago

Ah, right, I changed the plugin to use non-streaming responses with 3c3e81deff6d8dd934002bc08a596dbb6276f26e though, so that is not an issue anymore.

RuKeBo commented 10 years ago

Just curious, is the change to use non-streaming the best way to do it? Will it be reverted back the way it was before at some point in the future or remain this way?

chrippa commented 10 years ago

Streaming the segments is better in theory since it allows data to flow faster to the output. But in case of errors it can cause incomplete segments to be written, with non-streaming segments we instead get either the full segment or skip it completly (it gives up after 3 attempts to download it). Since we live in a world were internet connections are not always perfect I think going with non-streaming segments is the safest, and I'll probably change other streaming types (HLS) to do the same aswell.

RuKeBo commented 10 years ago

Let me know if this is something that might be an issue in how things are handled in the plugin, or occurs for some other reason. I open livestreamer and have the ustream stream playing in VLC. I also have the same stream open in a browser window on Ustream.tv. The livestreamer stream in VLC is about 10 seconds behind the browser stream, which is not really an issue, that I know of. However, the browser stream will end and then the VLC stream will, a few seconds later, which means the livestreamer stream does not play to the end of the actual stream. This happens the same in an output file, it cuts off the end.

Does the stream end because it in fact ended and it stops livestreamer immediately, but since it is about 10 seconds behind, it cuts it off? Did the stream end and it didn't have the time to write or show the rest of the stream? I am assuming from what you said here:

I've pushed a workaround that will attempt to downlaod the whole segment in one go which should make the timeout work correctly. Downside with this approach is that we have to wait for each segment to be fully downloaded before it can be sent to the player.

...that maybe, and this is pure speculation, that it might be trying to download the next segment, the stream ends, then it stops. If outputting to a file, it gets truncated. If in a player, it ends before finishing. Just curious.

chrippa commented 10 years ago

No, that's not what's happening, there are precautions in place to prevent issues like that. I accidently made the stream think it had one less segment available than it really has in d3a843bc9a83b5a625a60a02b428f4581ec4aa0d, but I've pushed a fix now.

RuKeBo commented 10 years ago

This appears to work better now. It appears to be capturing to the end of the stream.

Thanks

RuKeBo commented 10 years ago

I am getting some peculiar results with this now. Some things seem to be worked out. But for the most part, the issues come at the start or end of a stream. I am using the --retry-streams option and at times it won't capture the beginning of cuts off the end. I am going to assume that this is either an issue with the stream, as sometimes it works perfectly, or possibly the implementation per this:

About your issues, it is likely my implementation is slightly wrong since it's based purely on analyzing the network traffic. I have to decompile the flash player to get a better look how this works, but haven't gotten around to it yet.

The other issue that is more peculiar, is that it, at times, will not sync the audio and video. I was trying to do some testing on this but think it only occurs when outputed to a file, not to VLC. At some point in the video, the video jumps and it appears the audio does as well, and then they are out of sync after that. This does not happen all the time and I don't get an error. I tried it with the debug option and I don't see it doing anything peculiar, maybe it is "fetching module info" more in a row than it normally does, but not sure. The stream it was doing it on is not online anymore so I can't test it on the stream it was occurring on.

Anything I can keep a look out for or test to narrow this down?

chrippa commented 10 years ago

I am using the --retry-streams option and at times it won't capture the beginning of cuts off the end.

I did some testing by sending a stream myself to ustream and it seems there is a delay from where I start the stream on my end and when ustream will let me access it, it's usually between 15 and 30 seconds, is that what you're seeing?

I actually noticed a big bug where writer thread could be ended prematurely by the worker thread and I've pushed a fix, this was likely the cause for missing data at the end.

The other issue that is more peculiar, is that it, at times, will not sync the audio and video. I was trying to do some testing on this but think it only occurs when outputed to a file, not to VLC.

This is really odd, we have code that will make sure any gaps from missing chunks will be fixed by adjusting the timestamps. I've tested this by telling the stream to ignore some chunks, and it always works correctly both on file and in player.

RuKeBo commented 10 years ago

I did some testing by sending a stream myself to ustream and it seems there is a delay from where I start the stream on my end and when ustream will let me access it, it's usually between 15 and 30 seconds, is that what you're seeing?

Yes, but I have noticed it does better on some streams than others. It is fairly consistent on a particular stream. If will either cut off many or only one or two. I guess that's not completely consistent, but it does work better on some than others and some have a lot of issues.

It also appears to have something to do with the segment. It has recorded the same segment more than once and will cut off it off all times. Again ,this is not completely consistent, if it only cuts off a short amount, one time it may catch it all, but there are times it will do the same thing several times.

I will try the latest version and see how it goes.

Thanks

k-ross commented 10 years ago

I get that same audio sync issue. I am also using the --retry-streams option. Maybe it happens when it retries the stream, maybe that's when the audio sync problem occurs.

I will try again with the latest from git, without using --retry-streams, and see if that makes a difference.

Thanks!

k-ross commented 10 years ago

Trying without --retry-stream has the same effect. The de-sync always occurs NEAR the beginning of the file, but not the exact beginning. Usually within a minute or two. If it makes it past this part, it will stay in sync for the rest of the file, even if the file is 10 hours long.

RuKeBo commented 10 years ago

I am not having much luck with the newest version. Maybe it is the stream I am testing (http://www.ustream.tv/creativeLIVE-rebroadcast-2). I was using the "720p+_alt_akamai" option since "720p+" gives me a bunch or errors and I find that at times, only lower quality, like mobile_240p were available at the time. One time there were many other qualities available, not sure if 720p+ was. My audio for most of these got out of sync on this stream and most were cut off. As I watched it on Ustream, it would end a stream, then start it over near the end of what just ended, so it appears that stream is a bit peculiar.

FredThompsonII commented 8 years ago

I actually noticed a big bug where writer thread could be ended prematurely by the worker thread and I've pushed a fix, this was likely the cause for missing data at the end.

The other issue that is more peculiar, is that it, at times, will not sync the audio and video. I was trying to do some testing on this but think it only occurs when outputed to a file, not to VLC.

I'm having these exact issues. Frankly, I'd be happy with a way to keep recording and ignore the end-of-stream signal. Any progress?