search

blackjack4494 / yt-dlc

media downloader and library for various sites.
The Unlicense
2.9k stars 366 forks source link

youtube-dlc returns Error Code 1 when run with "-i" argument and engine encounters an error during runtime #202

Open etec-masterofsynapse opened 3 years ago

etec-masterofsynapse commented 3 years ago

Checklist

Description

YT-DLC has, as YT-DL, the option to ignore errors on download, with the "-i" argument. However I would expect that the .exe returns an error code 0 each time it is run with this argument since the user specifically wants to ignore errors that (maybe) happened. But even with the newest version it returns error code 1 if the program encountered an error during its run and skipped it due to "-i".

Is that behavior expected or should we change that?

pukkandan commented 3 years ago

-i is intended to continue downloading (the rest of a playlist, for example) even when there is an error. It should still return an error code. As far as I know, this has always been how it behaved. I even have scripts that depend on this behaviour

etec-masterofsynapse commented 3 years ago

-i is intended to continue downloading (the rest of a playlist, for example) even when there is an error. It should still return an error code. As far as I know, this has always been how it behaved. I even have scripts that depend on this behaviour

In the docs it states "Continue on download errors, for example to skip unavailable videos in a playlist", so I would use it if I have some private videos in a playlist and I still want the videos after that, so for it to just continue. But I would always start the playlist download from the beginning. For continuing a playlist I would use "--playlist-start x", "--playlist-end x" or "--playlist-items x,x".

And why should it still return an error code? If you are specifying the "--i" argument you know that it will fail along the way.

And for me its the other way round. Since I had some sites fail on me with random timeouts or "Unable to extract video params" errors many times during an archive run on a channel, I am executing a while loop as long as the error code is >0. So this while loop always gets triggered even though the execution itself was without errors.

pukkandan commented 3 years ago

For continuing a playlist I would use "--playlist-start x", "--playlist-end x" or "--playlist-items x,x".

I didn't mean continuing a playlist that was already downloaded partially. What I said is basically the same as what's in the docs. Just my wording seems to have confused you.

If you are specifying the "--i" argument you know that it will fail along the way.

No, I Use -i with ALL my downloads so that IF an item in the playlist happens to be unavailable, it will still download the rest of the videos.

even though the execution itself was without errors.

When using -i, under what circumstances do you think youtube-dlc should return an error code? how would you determine the "execution" was with or without error? The same error can often happen due to a variety of reasons. For example, a Unable to download webpage error could happen when the URL is wrong, when you have no internet, when the url was taken down or many other reasons. Some of these can be thought of us having "successfully executed" while other can't. As far as I know, there is no way to tell them apart.

Edit: After some digging in the source code, some errors are marked as "expected" and others not. This could conceivably be used to determine error code

Besides, this has always been youtube-dl's behaviour (correct me if I am wrong). So changing it could break countless third party scripts.

etec-masterofsynapse commented 3 years ago

If you are specifying the "--i" argument you know that it will fail along the way.

No, I Use -i with ALL my downloads so that IF an item in the playlist happens to be unavailable, it will still download the rest of the videos.

You can use that option with different mindsets, thats what I am talking about. If you expect an error you will specify -i and continue your run. I on the other hand DON'T expect an error everytime and will only specify -i if I know for a fact that there will be an error (a vlive video for example that will always be unavailable, because of Vlive+ or some other reason).

even though the execution itself was without errors.

When using -i, under what circumstances do you think youtube-dlc should return an error code? how would you determine the "execution" was with or without error? The same error can often happen due to a variety of reasons. For example, a Unable to download webpage error could happen when the URL is wrong, when you have no internet, when the url was taken down or many other reasons. Some of these can be thought of us having "successfully executed" while other can't. As far as I know, there is no way to tell them apart.

Well, thats what different error codes are for. There would be no problem defining an error code that means yt-dlc skipped errors but the operation in itself finished successfully, so no crash in between. Then I could just filter for that error code and the issue would be solved. But instead it is currently throwing a generic 1 as the error code, which can mean everything.

Besides, this has always been youtube-dl's behaviour (correct me if I am wrong). So changing it could break countless third party scripts.

Yes, could be. I am not using yt-dl or yt-dlc long enough to know how long this behavior has been there. I only noticed today that my script runs infinitely because I specified -i in the thought of getting through some errors but still receiving error code 0.

pukkandan commented 3 years ago

After some digging in the source code, some errors are marked as "expected" and others not. This could conceivably be used to determine error code

There would be no problem defining an error code that means yt-dlc skipped errors but the operation in itself finished successfully, so no crash in between. Then I could just filter for that error code and the issue would be solved.

If you want error codes only when there is a crash, you just need to remove this line

However, correctly distinguishing allowed/unexpected errors would take a lot more changes. The expected state of all the Error classes has to be propagated all the way to YoutubeDL.trouble function. Since various error are handled in different places, this is a bit tedious to do

etec-masterofsynapse commented 3 years ago

After some digging in the source code, some errors are marked as "expected" and others not. This could conceivably be used to determine error code

There would be no problem defining an error code that means yt-dlc skipped errors but the operation in itself finished successfully, so no crash in between. Then I could just filter for that error code and the issue would be solved.

If you want error codes only when there is a crash, you just need to remove this line

Yes, I could, however my use case is a bit more special than that. Let me explain: I wrote an engine that is able to download multiple Vlive channels, either full, so videos and subtitles, or subtitles only. Why I want the latter is because after Vlive videos get aired, there are sub teams that add subs later on, because the site is for Korean people, so I would need english subtitles to be able to understand the audio. And I dont want to download the whole video again just because new subtitles got published. So I use the channel in conjunction with the --skip-download flag and the --all-subs entry in my config file to only get "updated" subtitle files. But because Vlive doesnt deliver every video without error, I need to skip these with the -i option. On the full run however I want to cycle through every video successfully because I dont have everything yet and want to be able to correct the download errors during runtime. So it is not acceptable for me to skip some videos without me knowing the reason.

And the way I had to program the engine so I could cycle on error is with the error code obviously. But when I run the subtitle-only route and specify -i so I ignore any invalid videos, I still get error code 1 even though the run itself to update my subtitles was successful. There only happened to be some video that was unavailable for yt-dlc to throw code 1.

However, correctly distinguishing allowed/unexpected errors would take a lot more changes. The expected state of all the Error classes has to be propagated all the way to YoutubeDL.trouble function. Since various error are handled in different places, this is a bit tedious to do

And I totally agree that this would be tedious, however there are reasons the possibility of different error codes exist. When you first start developing a program you can go the easy route and throw 1 for error and 0 for successful. But later on when the program gets larger there should be some work gone into different error codes.

And I am by no means suggesting this has to happen anytime soon. I made some changes to my engine so I can still work with this "imperfection" in the error code usage. But I would like to be able to remove that change once yt-dlc gives some increased detail with the codes.