search

rogerfar / rdt-client

Real-Debrid Client Proxy
MIT License
929 stars 117 forks source link

Donwload restarts from zero on the very first error of a fragment #108

Closed StSixtus closed 2 years ago

StSixtus commented 2 years ago

The simple downloader loads in "fragments", at the very first download error in one fragment, it stops the full download. If the retry count isn't exceedd, the full download starts again. But usually fails in a fragment, and so on. Given my lunatic internet connection, it is very rare that a full download completes.

What version are you using? 2.0.5.0

Wat OS are you running? Windows or Docker/Debian/Hyper-V

Are you using Docker or as a service? Either command line on Windows or DockerCompose on linux

Please attach a log file here with the log setting set to debug Here is a log on Windows (where I'm sure there is disk space).

[16:11:51 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:11:51 DBG] Processing 1 torrents [16:11:51 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:51 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:11:52 DBG] Processing 1 completed downloads [16:11:52 DBG] Processing download for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] Download reported an error: The operation was canceled. for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] Download retry count 2/3, torrent retry count 1/1 for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] Retrying download for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] Removed from ActiveDownloadClients for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:52 DBG] Updating torrent info from Real-Debrid [16:11:55 DBG] Finished updating torrent info from Real-Debrid, next update in 10 seconds [16:11:55 DBG] Processing 1 torrents [16:11:55 DBG] Processing to download for download https://real-debrid.com/d/EZ2ACCFPC4WNS 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Unrestricting links for download https://real-debrid.com/d/EZ2ACCFPC4WNS 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Unrestricting link for download https://real-debrid.com/d/EZ2ACCFPC4WNS 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Marking download as started for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Setting download path to M:\Medias\Tmp\RDT\downloads\radarr for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Starting download for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:55 DBG] Starting download of https://53.download.real-debrid.com/d/ALHFFN4AGRN7W/Spi-----------------------------------------------------------.mkv, writing to path: M:\Medias\Tmp\RDT\downloads\radarr\Spi-----------------------------------------------------------.mkv\Spi-----------------------------------------------------------.mkv [16:11:56 DBG] No ID received for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:56 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:56 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:56 DBG] TorrentRunner Tick End (took 4017ms) [16:11:57 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:11:57 DBG] Processing 1 torrents [16:11:57 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:11:57 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267)

[...]

[16:13:22 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:13:22 DBG] Processing 1 torrents [16:13:22 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:22 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:23 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:13:23 DBG] Processing 1 torrents [16:13:23 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:23 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:24 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:13:24 DBG] Processing 1 torrents [16:13:24 DBG] Processing for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:24 DBG] Waiting for downloads to complete. 0/1 complete for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:25 DBG] TorrentRunner Tick Start, 1 active downloads, 0 active unpacks [16:13:25 DBG] Processing 1 completed downloads [16:13:25 DBG] Processing download for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:25 DBG] Download reported an error: The operation was canceled. for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:25 DBG] Download retry count 3/3, torrent retry count 1/1 for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267) [16:13:25 DBG] Not retrying download for download Spi-----------------------------------------------------------.mkv 0% (054ec476-dd74-4e3a-afcc-d1f617771bbc) for torrent Spi-----------------------------------------------------------.mkv (IG7LTVTBIRFCO - downloaded 100%) (07dd2383-926d-43df-8346-fdf772a31267)

StSixtus commented 2 years ago

My Internet connection, is from air (4G), and my operator applies a quality of service, which tends from time to time to freeze opened connections or tcp connection opening for a fews seconds (30 secs). Some downloaders like curl aren’t altered, the long tcp download is just stalled and continues afterward. But the download in rdtclient and the overall behavior, makes downloads almost impossible. In the logs, the full download was tried many times and failled at various % (from 3 to 7%).

Would it be possible, either/all : Work at the fragment download level. To add longer timeouts ? At least to provide option to do so ? Timeout in an opened stream, timeout to open a stream/resolve DNS… To work on the error handling at the fragment level. To retry at the fragment level ? Ideally with a configurable delay (in my case much more than 30 secs) ? To alter the retry. In case of error on a fragment to keep succeeded fragments, and only download the missing part ? Any other option, would be appreciated.

rogerfar commented 2 years ago

I suggest you switch to the Aria2 downloader as it handles those things for you properly.

StSixtus commented 2 years ago

Thanks for the answer, after some trials it is working.

In Docker, the rdt client seems to send its download path /data/downloads to the aria2c, so I had to create a volume /data/downloads for the aria2c container, with the same mapping as the one with the same name in rdt client docker container.