[FEATURE] Explain execa, delimiters, argument usage better & include some helpful messages when people enter custom args

[bhugh]

Is your feature request related to a problem? Please describe.

I've been working for several weeks now to set up YoutubeDL-Material (or any other yt-dlp GUI) to download files the way I'm interested in doing it, which is:

• download by album, audio only
• include the regular metadata fields such as album name, track name, artist, and thumbnail embedded in the file
• include track number on every track
• include the album_artist metatag with value Various on every track

If this is all done as described above, then the playback software I'm using (Navidrome) automatically recognizes albums, has all the tracks in the right order, etc. If even one of the elements is missing, albums are completely scrambled, they play in random order (extremely annoying when listening to, for example, nearly any classical music, recordings of musicals or shows, and concept albums).

Most other GUIs can't seem to do this at all, because they break downloads into individual files and so seem to completely lose the album & track information.

YoutubeDL-Material can do this all, but I had a devil of a time figuring out how to handle the necessary custom arguments within YoutubeDL-Material.

The way YoutubeDL-Material handles arguments is actually very good, but it is definitely nonstandard. I wasn't able to figure out how to set up the args until I dove into the guts of the program a fair bit. Once I understood how you're doing it, then aha!

But I don't know that I ever would have figured out what to do without looking into the program details that way, and most people likely won't do that.

I think the problem can be fixed pretty easily by some additions to the documentation and also giving users a couple of hints when they enter args (especially if they look like they are wrong/problematic).

Describe the solution you'd like

#1. Add an additional line into the field help for the various args, something like: "Args are delimited using two commas like so: ,, Do not use other common delimiters such as space, single quote, or double quote. Example: --write-info-json,,--audio-format,,best,,--parse-metadata

#2. If the person enters a space, ' or " into their custom args, somewhere display a message something like this: You used a space or quotation mark to delimit your args. This usually causes an error or incorrect output. Use only double commas ,, to delimit the args. See documentation.

I don't know if this message should appear in the logs, or a pop-up, or maybe a little bolded message just below the entry field. Maybe more than one of those places - the logs and a pop-up, for example. But 99% of the time if space, ', or " appears in a person's custom args, those custom args are simply not going to work. So you might as well let people know that right up front.

#3. Add to the documentation something like this:

Including custom args

You can include global custom args (under settings/downloads), custom args for a subscription (when you create the subscription), and for an individual download (under Advanced).

At an ordinary bash command line, args are delimited by some combination of spaces, single quotation marks, double quotation marks, tick marks, etc. However, YoutubeDL-Material args are different. _Do not use any spaces, single quotation marks, or double quotation marks when you enter your custom args._

In YoutubeDL-Material, the only delimiter used in custom args is two commas: ,,

More detailed explanation of args and delimiters

YoutubeDL-Material uses an advanced command processor, execa, that is optimized for programmatic use. YoutubeDL-Material passes each individual segment of the command line to execa as a separate variable. YoutubeDL-Material separates the args into these segments via the double-comma delimiter.

For example, if you use custom args --extract-audio,,--audio-format,,best,,--parse-metadata, YoutubeDL-Material will pass to execa four chunks of data: #1: --extract-audio, #2: --audio-format, #3: best, #4: --parse-metadata. execa then passes those four command line chunks to youtube-dl/yt-dlp, which is able to interpret them properly.

For that reason, you don't need any further delimiters - such as spaces, tick marks, or quotation marks - in your custom args, In fact, if you do include those unneeded delimiters, they will be interpreted literally, included literally in the command line chunks passed to execa, and (in almost all cases) those literally quoted characters will not be recognized as a valid command and so will cause errors and unexpected behavior.

Custom args examples

Example working/correct custom args:

--extract-audio,,--write-info-json,,--audio-format,,best,,--parse-metadata,,playlist_index:%(track_number)s,,--embed-metadata,,--embed-thumbnail,,--audio-quality,,0,,--postprocessor-args,,Metadata:-metadata album_artist=Various

Example incorrect custom args:

`--extract-audio,,--write-info-json,,--audio-format best,,--parse-metadata,,"playlist_index:%(track_number)s",,--embed-metadata,,--embed-thumbnail,,--audio-quality,,0,,--postprocessor-args,,'Metadata:-metadata album_artist=Various'

Problems with the incorrect custom args:

1. ,,--audio-format best,, has a space between audio-format and best. Instead of a space, use double commas. Correct: ,,--audio-format,,best,,

2. ,,--parse-metadata,,"playlist_index:%(track_number)s" includes quotation marks. Most command line interpreters like bash expect quotation marks in an arg like this, and will strip them out, sending all the characters inside the quotation marks to the program. But YoutubeDL-Material doesn't work this way. Simply omit the quotation marks like this: ,,--parse-metadata,,playlist_index:%(track_number)s

3. ,,--postprocessor-args,,'Metadata:-metadata album_artist=Various' includes single quotation marks. Again, simply omit them: ,,--postprocessor-args,,Metadata:-metadata album_artist=Various

Notes:

• Check the logs carefully for errors. Errors can be difficult to interpret because there are typically several independent levels of software processing your download. Each uses different software with different error reporting standards.

• If the downloaded files stay in their original file format instead of being converted to your desired format, the reason is almost certainly that the post-processor has failed due to incorrect args. Including delimiters like ", ', or space in the args will almost always cause such an error.

• In any situation where you may typically need to escape a character in an arg - for example, \" \' or \` - simply include the needed character with no backslash or other method of escaping. In fact, if you do include an escape character like backslash in an arg, it will be interpreted literally and cause an error.

• The example above shows one situation where you might include delimiters (space or quotation marks) in your args: When a certain arg is being passed along by youtubedl/yt-dlp to another, later, processor. So this arg properly includes a space:

,,--postprocessor-args,,Metadata:-metadata album_artist=Various

The reason it includes the space is because everything after Metadata: is passed by yt-dlp to the Metadata postprocessor as a command line. yt-dlp uses an ordinary command line processor, not execa, so the argument being passed to the metadata postprocessor is formatted as typically expected on a command line. Because YoutubeDL-Material args are delimited solely by double commas ,,, YoutubeDL-Material is not confused by the space that is included in the arg, and passes it to yt-dlp correctly as one single arg.

Additional context

Being able to pass arguments in this way is a pretty cool and advanced feature of YoutubeDL-Material, but - as you can see from my explanations above - I don't think most people will be able to figure out how best to take advantage of it without a somewhat detailed explanation. On the other hand - I think people wanting this level of control will indeed be open to reading and understanding detailed instructions.

Thanks again for making a great program.

By the way, the set of custom args I put together to accomplish my four requirements (especially album_artist=Various and inclusion of track numbers) is exactly the example above:

--extract-audio,,--write-info-json,,--audio-format,,best,,--parse-metadata,,playlist_index:%(track_number)s,,--embed-metadata,,--embed-thumbnail,,--audio-quality,,0,,--postprocessor-args,,Metadata:-metadata album_artist=Various

This is working just fine now!

[Patriot2407]

Thank you for this, it's much appreciated!

[Patriot2407]

Any idea how to delimit ! such as the argument --match-filter,,!is_live

[Patriot2407]

nvmd, I figured it out