satyamtg
commented
4 years ago Looking at the logs, I found out that the download failed with the following error -
{"log":"[youtube] Z3f55mVm-F8: Downloading webpage\n","stream":"stdout","time":"2020-08-19T10:12:24.634261486Z"}
{"log":"[youtube] Z3f55mVm-F8: Downloading MPD manifest\n","stream":"stdout","time":"2020-08-19T10:12:25.137334248Z"}
{"log":"[dashsegments] Total fragments: 11\n","stream":"stdout","time":"2020-08-19T10:12:25.641686725Z"}
{"log":"[download] Destination: /output/tmpyoni6esv/course/core-english-01/topic-5-my-feelings-and-myself/intro/intro-and-objectives/xblock/video.f303.webm\n","stream":"stdout","time":"2020-08-19T10:12:25.641732472Z"}
{"log":"ERROR: giving up after 0 fragment retries\n","stream":"stderr","time":"2020-08-19T10:12:30.165035754Z"}
{"log":"[openedx2zim::2020-08-19 10:12:30,165] ERROR:Error while running youtube_dl: ERROR: giving up after 0 fragment retries\n","stream":"stdout","time":"2020-08-19T10:12:30.165694225Z"}Also, looking up about the error on youtube_dl, I found out this issue - https://github.com/ytdl-org/youtube-dl/issues/21660 The last comment in the issue says that its a server error from youtube. Also, in another ZIM that I have locally, I have that very video available. So it's not a problem in the scraper.
However, we might need a very robust youtube downloader soon if this persists. For now I'm closing this. We can reopen if we find this happening again.
It looks like the scraper fails to integrate some of the videos, returning a
The media could not be loaded, either because the server or network failed or because the format is not supportederror message from topic five onwards in the PHZH mooc. See, e.g. http://tmp.kiwix.org:9991/phzh_core-english-one_en_2020-08_zimfarm/A/course/core-english-01/topic-5-my-feelings-and-myself/intro/intro-and-objectives/index.html