Error downloading video from Twitter message with additional text

Kilmatar commented 6 years ago

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2017.12.14. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.12.14

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

youtube-dl.exe -v https://twitter.com/WZorNET/status/942638289928048640 [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-v', 'https://twitter.com/WZorNET/status/942638289928048640'] [debug] Encodings: locale cp1251, fs mbcs, out cp1251, pref cp1251 [debug] youtube-dl version 2017.12.14 [debug] Python version 3.4.4 - Windows-10-10.0.17046 [debug] exe versions: ffmpeg N-86383-g4705edb, ffprobe N-86383-g4705edb [debug] Proxy map: {} ffmpeg version N-86383-g4705edb Copyright (c) 2000-2017 the FFmpeg developers built with gcc 7.1.0 (GCC) configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib libavutil 55. 63.100 / 55. 63.100 libavcodec 57. 96.101 / 57. 96.101 libavformat 57. 72.101 / 57. 72.101 libavdevice 57. 7.100 / 57. 7.100 libavfilter 6. 91.100 / 6. 91.100 libswscale 4. 7.101 / 4. 7.101 libswresample 2. 8.100 / 2. 8.100 libpostproc 54. 6.100 / 54. 6.100 [tls @ 0000000002643ee0] Server closed the connection [hls,applehttp @ 0000000002642ca0] HLS request for url 'https://video.twimg.com/ext_tw_video/942637672249593857/pu/vid/0/3000/1280x720/w9j2y_rZbE59gak9.ts', offset 0, playlist 0 [hls,applehttp @ 0000000002642ca0] Opening 'https://video.twimg.com/ext_tw_video/942637672249593857/pu/vid/0/3000/1280x720/w9j2y_rZbE59gak9.ts' for reading [h264 @ 00000000026e6d80] Reinit context to 1280x720, pix_fmt: yuv420p Input #0, hls,applehttp, from 'https://video.twimg.com/ext_tw_video/942637672249593857/pu/pl/1280x720/m_EZ56Axc3i2sFTY.m3u8': Duration: 00:00:38.37, start: 0.000000, bitrate: 0 kb/s Program 0 Metadata: variant_bitrate : 0 Stream #0:0: Video: h264 (High), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(left), 1280x720, 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc Metadata: variant_bitrate : 0 Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp Metadata: variant_bitrate : 0 file:WZor - тП░#GoodMorning #NinjaCat's and #WindowsInsiders ЁЯУМThis week 51 we expect the #lastflightЁЯЫйя╕Пthis year to get new #WIP build!ЁЯд╢ЁЯСб ЁЯЪйon #Tuesday we are waiting new ISO's Build 17061-> #DAC, #ADK, ...-942638289928048640.mp4.part: No such file or directory ERROR: ffmpeg exited with code 1 File "main.py", line 19, in File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl__init__.py", line 465, in main File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl__init__.py", line 455, in _real_main File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 1986, in download File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 795, in extract_info File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 887, in process_ie_result File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 849, in process_ie_result File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 1620, in process_video_result File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 1893, in process_info File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 1832, in dl File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\downloader\common.py", line 361, in download File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\downloader\external.py", line 57, in real_download File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\downloader\common.py", line 163, in report_error File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 612, in report_error File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpekokknty\build\youtube_dl\YoutubeDL.py", line 574, in trouble

Description of your issue, suggested solution and other information

https://twitter.com/WZorNET/status/942638289928048640 - tweet with excessive text/formatting/tagging Looks like error parsing such a page

hpistor commented 6 years ago

I just cloned master and it worked on my mac, it doesn't look like it's a parsing error. Maybe a problem with the emojis/other characters on Windows?

SpEcHiDe commented 6 years ago

I got this issue on a system running Debian 9, with YouTubeDL version 2018.05.18!

omgitsraven commented 6 years ago

I'm having an issue that was just closed as a duplicate without saying what it's a duplicate of, but I suppose they're suggesting that it's a duplicate of this?

I made some test cases that demonstrate this happening as a factor of tweet length rather than containing formatting/tagging or not, so I'm not sure if this belongs here, or if this report was mistaken about the cause... I'll put my examples in this thread for now, but if someone can point me towards a more appropriate thread I'll put it there instead.

When using youtube-dl to save a video attached to a tweet, the body of the tweet is used as the filename -- however, this can produce invalid filenames, at least on Windows. Using -o myfilename.mp4 is enough to work around this, but it would be nice if that weren't necessary :)

Note that this problem doesn't occur with very very small videos—the bug only happens when it tries to create .part files, not the actual video files themselves.

In short: youtube-dl.exe https://twitter.com/RavenTest/status/1053357651101470729 WORKS (short video on long tweet)

youtube-dl.exe https://twitter.com/RavenTest/status/1053359277916721154 WORKS (long video on short tweet)

youtube-dl.exe https://twitter.com/RavenTest/status/1053354601607561217 does NOT work (long video on long tweet with default filename)

youtube-dl.exe -o test.mp4 https://twitter.com/RavenTest/status/1053354601607561217 WORKS (long video on long tweet with manual filename)

ajj8 commented 4 years ago

This output format fixes it:

-o %(title).220s-%(id)s.%(ext)s

Still should be implemented into youtube-dl, though.

omgitsraven commented 4 years ago

I had to turn the value 220 down further before it would work for me, but that's a great workaround, thanks!

mwattsun commented 4 years ago

I fixed this bug and submitted a pull request. Twitter videos now download without problem. See: https://github.com/ytdl-org/youtube-dl/pull/24604 https://gist.github.com/mwattsun/a5eeb63087c8281066242321f0511185 https://github.com/mwattsun/youtube-dl/blob/master/youtube_dl/extractor/twitter.py

ytdl-org / youtube-dl