search

openzim / youtube

Create a ZIM file from a Youtube channel/username/playlist
GNU General Public License v3.0
46 stars 26 forks source link

Better handling of download failure #50

Closed rgaudin closed 4 years ago

rgaudin commented 4 years ago

ATM if the youtube server is unavailable or overloaded for a brief period of time, download would fail and the process would end.

Download is done via Youtube-dl, itself using aria2c. It seems that no retry was attempted. We should have this retry a few times at least.

@kelson42 what should be the script behavior in case multiple attempts fail? Give up as it is now or continue with a warning?

[youtube] Is3s8FuXWKE: Downloading webpage
[youtube] Is3s8FuXWKE: Downloading video info webpage
WARNING: video doesn't have subtitles
[youtube] Is3s8FuXWKE: Downloading MPD manifest
[youtube] Is3s8FuXWKE: Downloading thumbnail ...
[youtube] Is3s8FuXWKE: Writing thumbnail to: /output/build/videos/Is3s8FuXWKE/video.jpg
[download] Destination: /output/build/videos/Is3s8FuXWKE/video.webm

10/15 07:29:28 [NOTICE] Downloading 1 item(s)

10/15 07:29:28 [NOTICE] CUID#7 - Redirecting to https://r2---sn-vgqsrned.googlevideo.com/videoplayback?expire=1571146166&ei=VnWlXc6_LNTyhwb_iYnYDA&ip=185.15.56.1&id=o-AM_h_Ml3sqYSkkWDMgKPoHWkEO9qp8JKPOCHV6GvcGc2&itag=43&source=youtube&requiressl=yes&mime=video%2Fwebm&gir=yes&clen=121706110&ratebypass=yes&dur=0.000&lmt=1544093373899213&fvip=4&fexp=23842630&c=WEB&txp=2201222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ALgxI2wwRgIhALrK9lzzumFuBPa-_KJRUyN05yHknhe8qXwY8xJLlGsnAiEApfxg0TK6VZg0XbnWsnnYL43GcxL7G8i5lEIK1aGH0_Q%3D&redirect_counter=1&cm2rm=sn-p5qe7s7l&req_id=9c0e3d05fd5da3ee&cms_redirect=yes&mm=34&mn=sn-vgqsrned&ms=ltu&mt=1571124171&mv=u&mvi=1&pl=24&lsparams=mm,mn,ms,mv,mvi,pl&lsig=AHylml4wRQIgUGfhQ_UQmQyzWQ2VQESLIFAWdIUee_Tdto-s0Xz7HvkCIQDUd0fooUyfPxnsFhtZD-Vxxh3eheuckvnl4okJgYAt6g==

10/15 07:29:28 [NOTICE] CUID#7 - Redirecting to https://r2---sn-q4fl6ney.googlevideo.com/videoplayback?expire=1571146166&ei=VnWlXc6_LNTyhwb_iYnYDA&ip=185.15.56.1&id=o-AM_h_Ml3sqYSkkWDMgKPoHWkEO9qp8JKPOCHV6GvcGc2&itag=43&source=youtube&requiressl=yes&mime=video%2Fwebm&gir=yes&clen=121706110&ratebypass=yes&dur=0.000&lmt=1544093373899213&fvip=4&fexp=23842630&c=WEB&txp=2201222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ALgxI2wwRgIhALrK9lzzumFuBPa-_KJRUyN05yHknhe8qXwY8xJLlGsnAiEApfxg0TK6VZg0XbnWsnnYL43GcxL7G8i5lEIK1aGH0_Q%3D&cm2rm=sn-p5qe7s7l&req_id=9c0e3d05fd5da3ee&redirect_counter=2&rm=sn-vgqeez7l&cms_redirect=yes&mm=39&mn=sn-q4fl6ney&ms=ltr&mt=1571123964&mv=u&mvi=1&pl=24&lsparams=mm,mn,ms,mv,mvi,pl&lsig=AHylml4wRQIhAPSV3WsPlItIr0yDxGTc3FMLSYsYqzK7zhkZqco7VQbdAiBcD-A91uZHaT8Sh0AE8ej0_08VV1le2kj4Gmnp5DEHAg==

10/15 07:29:28 [NOTICE] CUID#7 - Redirecting to https://redirector.googlevideo.com/videoplayback?expire=1571146166&ei=VnWlXc6_LNTyhwb_iYnYDA&ip=185.15.56.1&id=o-AM_h_Ml3sqYSkkWDMgKPoHWkEO9qp8JKPOCHV6GvcGc2&itag=43&source=youtube&requiressl=yes&mime=video%2Fwebm&gir=yes&clen=121706110&ratebypass=yes&dur=0.000&lmt=1544093373899213&fvip=4&fexp=23842630&c=WEB&txp=2201222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ALgxI2wwRgIhALrK9lzzumFuBPa-_KJRUyN05yHknhe8qXwY8xJLlGsnAiEApfxg0TK6VZg0XbnWsnnYL43GcxL7G8i5lEIK1aGH0_Q%3D&cm2rm=sn-p5qe7s7l&req_id=9c0e3d05fd5da3ee&rm=sn-vgqeez7l,sn-q4fe7e7l&cms_redirect=yes&mm=39&mn=sn-q4fl6ney&ms=ltr&mt=1571123964&mv=u&mvi=1&pl=24&lsparams=mm,mn,ms,mv,mvi,pl&lsig=AHylml4wRQIhAPSV3WsPlItIr0yDxGTc3FMLSYsYqzK7zhkZqco7VQbdAiBcD-A91uZHaT8Sh0AE8ej0_08VV1le2kj4Gmnp5DEHAg==&redirect_counter=3&cmo=td=googlevideo.com&cmo=rr=11&cmo=rs=sn-q4fe7e7l&cmo=secure_transport=yes&prf=1

10/15 07:29:28 [ERROR] CUID#7 - Download aborted. URI=https://r2---sn-p5qlsndr.googlevideo.com/videoplayback?expire=1571146166&ei=VnWlXc6_LNTyhwb_iYnYDA&ip=185.15.56.1&id=o-AM_h_Ml3sqYSkkWDMgKPoHWkEO9qp8JKPOCHV6GvcGc2&itag=43&source=youtube&requiressl=yes&mm=31%2C26&mn=sn-p5qlsndr%2Csn-5uaeznzk&ms=au%2Conr&mv=u&mvi=1&pl=24&mime=video%2Fwebm&gir=yes&clen=121706110&ratebypass=yes&dur=0.000&lmt=1544093373899213&mt=1571124413&fvip=4&fexp=23842630&c=WEB&txp=2201222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ALgxI2wwRgIhALrK9lzzumFuBPa-_KJRUyN05yHknhe8qXwY8xJLlGsnAiEApfxg0TK6VZg0XbnWsnnYL43GcxL7G8i5lEIK1aGH0_Q%3D&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl&lsig=AHylml4wRgIhAJ22l8KPvp8auzwDGrf3e5ksKNwfCB205SuBtgaIy79RAiEAohLccN1687Oi1LoaCuQqPvdFXdytZPd0K26cm3FsnJg%3D
Exception: [AbstractCommand.cc:351] errorCode=29 URI=https://redirector.googlevideo.com/videoplayback?expire=1571146166&ei=VnWlXc6_LNTyhwb_iYnYDA&ip=185.15.56.1&id=o-AM_h_Ml3sqYSkkWDMgKPoHWkEO9qp8JKPOCHV6GvcGc2&itag=43&source=youtube&requiressl=yes&mime=video%2Fwebm&gir=yes&clen=121706110&ratebypass=yes&dur=0.000&lmt=1544093373899213&fvip=4&fexp=23842630&c=WEB&txp=2201222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=ALgxI2wwRgIhALrK9lzzumFuBPa-_KJRUyN05yHknhe8qXwY8xJLlGsnAiEApfxg0TK6VZg0XbnWsnnYL43GcxL7G8i5lEIK1aGH0_Q%3D&cm2rm=sn-p5qe7s7l&req_id=9c0e3d05fd5da3ee&rm=sn-vgqeez7l,sn-q4fe7e7l&cms_redirect=yes&mm=39&mn=sn-q4fl6ney&ms=ltr&mt=1571123964&mv=u&mvi=1&pl=24&lsparams=mm,mn,ms,mv,mvi,pl&lsig=AHylml4wRQIhAPSV3WsPlItIr0yDxGTc3FMLSYsYqzK7zhkZqco7VQbdAiBcD-A91uZHaT8Sh0AE8ej0_08VV1le2kj4Gmnp5DEHAg==&redirect_counter=3&cmo=td=googlevideo.com&cmo=rr=11&cmo=rs=sn-q4fe7e7l&cmo=secure_transport=yes&prf=1
  -> [HttpSkipResponseCommand.cc:231] errorCode=29 The response status is not successful. status=503

10/15 07:29:28 [NOTICE] Download GID#9e7fb10e0d0e18ee not complete: /output/build/videos/Is3s8FuXWKE/video.webm.part

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
9e7fb1|ERR |       0B/s|/output/build/videos/Is3s8FuXWKE/video.webm.part

Status Legend:
(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.

ERROR: aria2c exited with code 29
ERROR:FAILED. An error occured: ERROR: aria2c exited with code 29
stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

rgaudin commented 4 years ago

done