yausername / youtubedl-android

youtube-dl for android
GNU General Public License v3.0
934 stars 172 forks source link

Feature Request: Add callback to read the output of process whether it is progress or not #266

Closed denisnumb closed 4 months ago

denisnumb commented 4 months ago

At the moment, when starting a process via YoutubeDL.getInstance().execute(), you can specify a method as a callback parameter that will receive the video download progress.

This is useful, but I would like the library to provide the ability to also get the current output from a process.

Example:

I send this request to get brief information about each video on the channel:

val request = YoutubeDLRequest("https://www.youtube.com/@MrBeast/videos")
request.addOption("--skip-download")
request.addOption("--print-json")
request.addOption("--flat-playlist")

val result = YoutubeDL.getInstance().execute(request, "test", callback)

But at the same time callback does not receive a single call, because in fact, the download does not occur and yt-dlp does not output a line that could be read by the library as download progress.

Anyway, for each video, yt-dlp sends a separate json to stdout, which the library reads in line 34 of the StreamProcessExtractor file:

val line = currentLine.toString()

As a result, instead of sequentially getting information about each video from the process, I have to wait until it is completely loaded and take the final result of yt-dlp from the result variable in my code.


Current solution:

  1. Wait until the process is fully loaded and completed. For channels with 1000+ videos, this takes a lot of time.

  2. Run several processes in parallel at once and receive videos from the channel in portions, for example, at least 5 pieces, through the arguments --playlist-start $chunkSize and --playlist-end ${chunkSize+5}. This wastes a lot of hardware resources.

Ideal solution:

Add another argument to execute to specify the method that will receive the process output.

JunkFood02 commented 4 months ago

What do you mean by sequentially getting information about each video from the process? I believe that the yt-dlp process does only emit one output for your use case, and that's exact how it works.

denisnumb commented 4 months ago

Try the command

yt-dlp --skip-download --print-json --flat-playlist https://www.youtube.com/@MrBeast/videos

Or this one that takes longer to complete

yt-dlp --get-title https://www.youtube.com/watch?v=tWYsfOSY9vY https://www.youtube.com/watch?v=KOEfDvr4DcQ https://www.youtube.com/watch?v=krsBRQbOPQ4

And programmatically read the output from the process, for example through a program in Python or in any other language.

Or set a breakpoint on line 34 of the file StreamProcessExtractor.kt and see how many times the debugger stops at this line and what will be in the line variable

JunkFood02 commented 4 months ago

For a quick walkaround, i think we can just remove this if statement to expose every line of output to the callback

https://github.com/yausername/youtubedl-android/blob/ed652962e5caf2deb6ca5286eddfc8327c8ace75/library/src/main/java/com/yausername/youtubedl_android/StreamProcessExtractor.kt#L34-L35

denisnumb commented 4 months ago

You mean I should download the source code and do it myself?

JunkFood02 commented 4 months ago

You can use the snapshot build from the master branch for the walkaround. But a more robust solution might need modifications to API