filecxx / FileCentipede

Cross-platform internet upload/download manager for HTTP(S), FTP(S), SSH, magnet-link, BitTorrent, m3u8, ed2k, and online videos. WebDAV client, FTP client, SSH client.
http://filecxx.com
6.99k stars 452 forks source link

download segment create video ts corruption #463

Open candrapersada opened 1 year ago

candrapersada commented 1 year ago

why download segment make video ts corruption? and is there any option to retries the broken segment download without re-downloading everything for example if segment 201 is damaged from 500, will the download be repeated for 201 when using connection 16?

ffmpeg version n5.1.2-7-ga6e26053c2-20221125 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.2.0 (crosstool-NG 1.25.0.90_cf9beb1)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --disable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --disable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20221125
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mpegts, from 'D:\movies\Sonic the Hedgehog 2 (2022).ts':
  Duration: 02:01:23.40, start: 1.462000, bitrate: 240 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1280x720 [SAR 90:67 DAR 160:67], 24 fps, 24 tbr, 90k tbn
  Stream #0:1[0x101](und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 134 kb/s
Output #0, mp4, to 'D:\movies\Sonic the Hedgehog 2 (2022).mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 90:67 DAR 160:67], q=2-31, 24 fps, 24 tbr, 90k tbn
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 134 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 1284750). speed=   0x
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 2293500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 3441000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 4292250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 4963500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 5631000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 6711000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 7513500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 9658500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 10637250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 11672250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 12186000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 13131000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 14162250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 14754750).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 16693500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 17537250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 19363500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 20927250).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] PES packet size mismatch
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 1, dts = 20913660).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 1
    Last message repeated 1 times
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 21988500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 22806000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 23758500).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 24696000).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 0, dts = 25629750).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 0
[mpegts @ 000001c045f72c00] PES packet size mismatch
[mpegts @ 000001c045f72c00] Packet corrupt (stream = 1, dts = 25615740).
D:\movies\Sonic the Hedgehog 2 (2022).ts: corrupt input packet in stream 1
    Last message repeated 4 times
[aac_adtstoasc @ 000001c0468d61c0] Error parsing ADTS frame header!
[mp4 @ 000001c04605e300] Error applying bitstream filters to an output packet for stream #1: Invalid data found when processing input
av_interleaved_write_frame(): Invalid data found when processing input
frame= 1230 fps=0.0 q=-1.0 Lsize=    9042kB time=00:03:09.64 bitrate= 390.6kbits/s speed= 609x
video:8253kB audio:770kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.203460%
Conversion failed!

https://146.19.24.137/sonic-the-hedgehog-2-2022/

filecxx commented 1 year ago

Usually, when the remote HTTP server doesn't want to send the segment file data, it will respond with an HTTP status code like 400, 401, or 403 ..., then FileCentipede will wait for a few seconds and try to re-download it.

If it fails, an error like "Address expired" or "Server busy" will be displayed.

But some special HTTP servers may return an HTTP status code 200 with incorrect data like this:

{"code:" 403, "message": "too many requests"}

So, FileCentipede will treat it as segment file data and write it to the disk.

Another case is remote HTTP server didn't have the "Content-Length" or "Transfer-Encoding: Chunked" field in the HTTP header, it will close the connection when the transfer is completed, but if the remote HTTP server closes the connection before all data is received, the segment data will be incomplete.

Unlike the BitTorrent protocol, HTTP doesn't have block hashes. So, it's uncheckable and unfixable.

The good news is the probabilities are very low, and it only happens when the connectivity is Unreliable.

image image

Answer your question: There's no option to download specified segments, but I will make one in the future.

candrapersada commented 1 year ago

but the problem occurs more often when using a chromium based browser code 404

filecxx commented 1 year ago

what 404?

candrapersada commented 1 year ago

image

filecxx commented 1 year ago

404 means the remote server doesn't have the file.

candrapersada commented 1 year ago

but if you press the start button or resume the download runs normally but the segment is broken

filecxx commented 1 year ago

Error code is only based on the HTTP server's response.

If the state is "Error code 404", after a while it becomes normal again, maybe you should not download it from the server.

candrapersada commented 1 year ago

but the problem is only in chrome and it doesn't happen to firefox

filecxx commented 1 year ago

I have no idea, FileCentipede chrome extension and firefox extension used the same code.

It's the website's problem.

when I download videos from https://embedy.cc/ It works fine on firefox, but I can't download the same video when I use chrome.

candrapersada commented 1 year ago

chrome page URL https://146.19.24.137/sonic-the-hedgehog-2-2022/ address https://hive.gdriveplayer.link/slzvXw3o5Cxb-2KLIniFwbFq0672-bqfyqo6CxyNFrv0a9ZxI_S0-2S3o1Azyw6MCSUyPlE5FJ5dbOUsI-7dzg/jlfRTrrrc4j6hLtzVeInYYnhpS1zDQNVHl2_o-WoDYw/360.m3u8 Referer: https://gdriveplayer.link/player/hykGobzjSz4VBer/ Origin: https://gdriveplayer.link Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36

filecxx commented 1 year ago

this m3u8 file works fine for me.

image

image image

candrapersada commented 1 year ago

Has it been tested too page URL https://146.19.24.137/sonic-the-hedgehog-2-2022 in chromium based browser?

filecxx commented 1 year ago

Yes, I tested.

filecxx commented 1 year ago

Have you tried clearing your browser cookies?

candrapersada commented 1 year ago

no

candrapersada commented 1 year ago

chrome_mv3