search

snobu / destreamer

Save Microsoft Stream videos for offline enjoyment.
MIT License
2.37k stars 439 forks source link

ffmpeg error – Invalid data found #110

Closed gy-mate closed 8 months ago

gy-mate commented 4 years ago

Hi! I've tried downloading an exact Microsoft Stream video three times, but I got this error for each try. I got this error previously for another Stream video, but I succeeded downloading that for the second try. What strange is, that – apart from these two Stream videos – I could download everything else flawlessly.

I'm not sure if this is an ffmpeg or destreamer error.

Below is the pasted code. I've redacted my username, the video's link and its title. If you need those, I can send them privately to you.

MacBook:destreamer gymate1$ ./destreamer.sh -o /Users/gymate1/Downloads/ -u <username> -i <video link> Using ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers Username: <username> Access token still good for 48 minutes. Fetching metadata... Downloading Video: <video title>

Spawning ffmpeg with access token and HLS URL. This may take a few seconds... progress [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 0% 0 0s

ffmpeg returned an error: https://euwe-1.api.microsoftstream.com/api/videos/manifest?api-version=1.0&playbackurl=https://cvcq2ehpzmewuhceldfzfzxrzh.streaming.mediaservices.windows.net/4ba149e0-7841-4c45-9478-7623ffe4704b/2-qjlxf7tmclcb4diahg6bz2o64e.ism/manifest(format=m3u8-aapl): Invalid data found when processing input

Unknown FFmpeg error

MacBook:destreamer gymate1$

lukaarma commented 4 years ago

I'm not really sure were the problem is in this one... because I highly doubt that Destreamer is at fault here and I downloaded the file that ffmpeg is choking on and it seems all good to me so..

Can you please try to run it with the verbose flag and post back here the result?

gy-mate commented 4 years ago

Oh, sorry, I should've done that. Ran it with the verbose flag, but it worked now with this video. Here is the code anyway:

Original Issue verbose (now working) ``` MacBook:destreamer gymate1$ ./destreamer.sh -o /Users/gymate1/Downloads/ -i

But! I've tried it with another Stream video. First I got this error (not the one that I opened the issue for):

New urelated issue (503 on refreshToken) ``` MacBook:destreamer gymate1$ ./destreamer.sh -o /Users/gymate1/Downloads/ -i

Then I tried it again, and got the error I opened the issue for:

Original Issue but on new video ``` MacBook:destreamer gymate1$ ./destreamer.sh -o /Users/gymate1/Downloads/ -i
lukaarma commented 4 years ago

New urelated issue (503 on refreshToken)

the 503 error you got is from our refreshToken function that has been giving us some problems and we are aware that it unstable/not behaving how it should and I think @snobu is already looking into it

Original Issue but on new video

The Issue that is the focus of this problem is one that I couldn't replicate and that I'm sure it's either ffmpeg crashing while waiting/parsing the response from the link we gave it or it's choking parsing the link itself, not sure.

I'm looking into a more stable way to call ffmpeg giving it an already downloaded playlist so we can be sure that's not a problem

Stay tuned for a possible update in the next days. For now all I can suggest you is to brute force it and keep trying, sorry

gy-mate commented 4 years ago

Great, thank you so much guys! You're awesome!

I've actually tried downloading the same video again and it worked this time. So I just have to try a few times.

snobu commented 4 years ago

The 503 is very random and i could not yet find a correlation with something we may be doing wrong. If you look closer it's a Service Unavailable response coming from some frontend (i assume, since it's HTML not JSON)

      statusCode: 503,
      statusMessage: 'Service Unavailable',
...
 data: '<!DOCTYPE HTML PUBLIC "-...>\r\n' +
      '<HTML><HEAD><TITLE>Service Unavailable</TITLE>\r\n' +
      '<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>\r\n' +
      '<BODY><h2>Service Unavailable</h2>\r\n' +
      '<hr><p>HTTP Error 503. The service is unavailable.</p>\r\n' +
      '</BODY></HTML>\r\n'

We could catch it and retry a few times, i'll look into it.

When the token refresh fails (which is still an open bug) you get a 401 Unauthorized.

snobu commented 4 years ago

This should now be fixed via https://github.com/snobu/destreamer/pull/130. git pull and rebuild to get the latest code.

ali4heydari commented 4 years ago

I have destreamer with latest commit 89a942eb24bb3a5774e69791dfb86c97c4c2740e and get the Invalid data found when processing input error.

node: 13.10.1 os: windows 10 x64

mintty-command-line$ ./destreamer.sh -f list.txt --format mp4 -u heydari_ali@comp.iust.ac.ir -v
Using ffmpeg version git-2020-05-15-b18fd2b Copyright (c) 2000-2020 the FFmpeg developers

Username: heydari_ali@comp.iust.ac.ir
Video URLs:
[ 'list.txt' ]

Access token still good for 13 minutes.

Video GUIDs:
[
  'c0811d7f-32f3-462a-973b-75aa1f4e3bcc',
  '90b1d34a-a3b7-4701-8840-942d9887987e',
  'eab49afb-ac64-42c7-bd08-a8562849177a',
  'c8de0d10-466a-4095-a9b6-14141b67b695',
  '2cb63234-6d4b-4cb3-ac38-237484acd81d',
  'b6e7fce9-5d75-4280-8023-3f45a441c40e',
  'ef29dd76-445c-4f7f-8e21-ce15e004444a',
  'a44477ed-a8b7-4c1b-8ad6-0e6496aad1ea',
  '96011beb-b4eb-442b-b2ed-391bebeba70e',
  '775e6c8a-567b-41c8-92f3-e67d01b66694',
  'eddc170c-4ff8-450d-ae5e-ad876c8b9d79',
  'e7ef52f7-e1db-4310-9858-3abe7d039740',
  '7b752f99-1a96-4eee-aab0-28ee52f0af5c',
  'ef92165c-dae4-4f3f-9372-df5a9a151640',
  '99e4b7c6-829d-4744-858f-7827f31f5ca0',
  '6280c023-b8e3-42cd-b4a0-35bee63dd1a2'
]
Fetching metadata...
[ 'videos' ]

Downloading Video: ComputerNetworks1399-3-13

Spawning ffmpeg with access token and HLS URL. This may take a few seconds...
Unable to get number of columns from terminal.
This happens sometimes in Cygwin/MSYS.
No progress bar can be rendered, however the download process should not be affected.

Please use PowerShell or cmd.exe to run destreamer on Windows.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄

ffmpeg returned an error: https://aase-1.api.microsoftstream.com/api/videos/manifest?api-version=1.0&playbackurl=https://g6dn2dch6ae6egwlnztsbpdtoe-aase.streaming.media.azure.net/5e7aa503-3104-46a5-88f8-7e8576d994e8/1-5uzexnjuqyxqrzmo2qxxv7ngoc.ism/manifest(format=m3u8-aapl): Invalid data found when processing input
salvmontagna commented 4 years ago

I fixed "Unhandled error! Timeout or fatal error, please check your downloads directory and try again" by just restarting my computer. Anyway, there is someway to increment token time? I'd like to download links inside a text file, but in 1 hour I just can get about 3 of them.

snobu commented 4 years ago

Yes. Wait for version 2.1, will get pushed next week. Pass -k to the CLI and tick Remember Me during login.

salvmontagna commented 4 years ago

Yes. Wait for version 2.1, will get pushed next week. Pass -k to the CLI and tick Remember Me during login.

That's wonderful, thanks!

snobu commented 3 years ago

First make sure you're using the latest code by going a git clone in a new folder. Follow the steps in the README carefully and it will work for you too.

gy-mate commented 8 months ago

I guess this was fixed in v2.1. Thank you very much, @snobu!