HaveAGitGat / Tdarr

Tdarr - Distributed transcode automation using FFmpeg/HandBrake + Audio/Video library analytics + video health checking (Windows, macOS, Linux & Docker)
Other
3.07k stars 96 forks source link

Auto beginn command & execute #1053

Closed CurlyFlow closed 3 months ago

CurlyFlow commented 3 months ago

Maybe its a dumb idea, but i wonder why we have to manually add them? Is there ANY reason that its not "automatic" or "hidden" added when you use a block which needs it? (just needs a small logic to dont put to many like when there are 2 or more)

I know its not much work or so, but in the long run, this could safe some frustration. (and i dont see any negative when doin so?)

Also there are no options in execute or begin command, so not very usefull?

HaveAGitGat commented 3 months ago

Maybe its a dumb idea, but i wonder why we have to manually add them? Is there ANY reason that its not "automatic" or "hidden" added when you use a block which needs it? (just needs a small logic to dont put to many like when there are 2 or more)

Are you able to clarify what you mean by this?

Also there are no options in execute or begin command, so not very usefull?

It's about what the blocks are doing under the hood. Begin initializes the ffmpeg flow variables (or clears any old ones), Execute then executes the created command. Let's say you have this flow, it's obviously needed to have the Begin and Execute blocks else it's not possible to determine where one command begins and the other ends:

image

CurlyFlow commented 3 months ago

I see why these steps are necessary, especially given the complexity of the workflow. However, I still wonder if they need to be manually added to the UI.

It might be more efficient if these segments could be handled automatically behind the scenes. For example, logic could be implemented to prevent starting new segments unnecessarily and to ensure segments end automatically only when needed, without requiring manual intervention from the user.

This could streamline the process and reduce the need for manual adjustments.

HaveAGitGat commented 3 months ago

I'm not sure what logic you are thinking of. Take these 2 flows for example:

image image

They have different outcomes like that.

With Start and Execute removed they look exactly the same:

image

There's no logic that can read the user's mind about which commands they want to happen where.

This is just a simple example and there are far more complex flows out there with 50+ parts. In the above flow it's not clear what should happen, are the first arguments meant to be with the second ones or should they be separate?

CurlyFlow commented 3 months ago

No need to read minds because users still put in their FFMPEG commands as usual.

My point is that adding BEGIN and EXECUTE tasks manually feels unnecessary since they don’t offer any options or configurations for the user. These steps are always required to ensure the commands work correctly, so it would be more efficient for the system to automatically insert them behind the scenes. This would streamline the process, allowing users to focus on their core FFMPEG commands without having to manually add these required but functionally redundant tasks.

++ Plus, it would save visible space in the UI.

++ Im also pretty sure that this way some support could be saved, because adding something which is not obv. needed isnt very clear when you start dealing with flows.

Example Workflow (as seen in the UI):

FFMPEG command: Set container Check video bitrate FFMPEG command: Custom arguments FFMPEG command: Remove subtitles ...other commands

What happens behind the scenes:

BEGIN FFMPEG command: Set container Check video bitrate FFMPEG command: Custom arguments FFMPEG command: Remove subtitles EXECUTE ...other commands

I understand that in more complex workflows, like the example you showed with a 'check' segment nested between BEGIN and EXECUTE, it might be tricky to determine where to place EXECUTE. However, a simple logic could be implemented to handle this. For instance, the system could automatically check if all 'check' segments eventually converge before placing EXECUTE.

A practical approach could be: if no BEGIN and (or) EXECUTE tasks are present in a flow with FFMPEG commands, the system should insert them automatically—BEGIN at the start (easy) and EXECUTE at the most appropriate end point.

The placement of EXECUTE could be fine-tuned to accommodate complex scenarios like the one you mentioned.

This way existing scenarios wouldnt change and only people forgetting to put BEGIN and EXECUTE would benefit, also if the "auto add" doesnt work correct, it could be corrected just the old way. (manually)

supersnellehenk commented 3 months ago

This way existing scenarios wouldnt change and only people forgetting to put BEGIN and EXECUTE would benefit, also if the "auto add" doesnt work correct, it could be corrected just the old way. (manually)

If people have followed the tutorials, it should be pretty clear that each ffmpeg command needs to be encased within a BEGIN and EXECUTE block.

the system should insert them automatically—BEGIN at the start (easy) and EXECUTE at the most appropriate end point.

BEGIN is easy, yes. It could be inserted before the first ffmpegCommand block. EXECUTE however isn't as easy, as in the examples shown by HaveAGitGat, how would the system know you want those to be two separate commands?

It would also confuse new users, as it's adding blocks that aren't in your flow and could cause unwanted side effects from merging commands which shouldn't be merged. This would be solved by manually adding the blocks, but then the benefit is already gone.

Example Workflow (as seen in the UI):

FFMPEG command: Set container Check video bitrate FFMPEG command: Custom arguments FFMPEG command: Remove subtitles ...other commands

What happens behind the scenes:

BEGIN FFMPEG command: Set container Check video bitrate FFMPEG command: Custom arguments FFMPEG command: Remove subtitles EXECUTE ...other commands

In the example HaveAGitGat showed, it was like this: BEGIN FFMPEG command: Set video encoder FFMPEG command: Set container EXECUTE BEGIN Check video bitrate FFMPEG command: Custom arguments FFMPEG command: Remove subtitles EXECUTE ...other commands

If you just go by "group all the ffmpegCommand blocks, put a begin at the front and execute at the end", it would end up being 1 command instead of 2. I can't see a way how Tdarr would know you want those to be separate commands, without the explicit blocks.

Example of my own flow (which might not be the best for this scenario): image

In this case I've got 3 commands, which have a fallback to each other and all lead to the same block on success. Removing the BEGIN and EXECUTE blocks would smush all of these 3 together.

All I'm trying to say is, it's rather complex to tailor to each and every scenario if Tdarr would be adding these blocks itself. It's easier and clearer to the user to add these blocks themselves. For simple flows that have a couple of steps it could be done, but it's quite a lot of work to get it right and it's much easier for the user to just drag & drop 2 additional blocks.

HaveAGitGat commented 3 months ago

@CurlyFlow not sure you read my comment as I gave a very clear example of 2 flows that are the exact same when Begin and Execute are removed, but perform very differently when Begin and Execute are added. They are far from functionally redundant.

system could automatically check if all 'check' segments eventually converge

This would not work either. Here's a flow where the Check blocks converge but the user may have wanted to produce a new output file in the middle and then start a new command.

image

I think you're underestimating just how complex FFmpeg can get, there are several thousand flags that do all sorts of things:

https://gist.github.com/tayvano/6e2d456a9897f55025e25035478a3a50

A user may be wanting to produce an output file in one set of blocks that they then want to use in the next set of blocks. Or they might want to scan the file for thousands of different properties in one set of blocks and use that in the next set of blocks. There are millions of other combinations of FFmpeg flags that would need to be accounted for.

EXECUTE at the most appropriate end point

It's impossible to determine what the most appropriate point is. Take this very simple flow:

image

Read through the thousand of FFmpeg flags, understand them, and then tell me what logic should be implemented to detect that the user wants to do this:

image

And not this:

image

If you give me the logic, I will add it.

CurlyFlow commented 3 months ago

I want to clarify that I didn't expect such complex flow structures; my question is more about what is commonly used.

I also have a possibly unconventional idea to improve this, though I'm not sure if it’s feasible: Could we 'brute-force' it? Specifically, is it possible to run FFMPEG with two conflicting commands? If it fails, the system would know not to place the BEGIN and EXECUTE around that segment. I’m not sure about the performance impact, but it could be a way to handle this, like caching it.

supersnellehenk commented 3 months ago

I want to clarify that I didn't expect such complex flow structures

These flow structures aren't all that complex, that's the whole point. They're rather basic.

Could we 'brute-force' it?

At that point, wouldn't it be easier for the user to just add two blocks? Easy to read, Tdarr doesn't have to do any guess work and it all runs quicker.

I don't see a benefit at all at Tdarr just guessing where blocks should be if the user can tell Tdarr exactly what they want it to do.

CurlyFlow commented 3 months ago

Sorry, i didnt meant it as replacement for current version. It was meant as a "easy" mode. Like when you click expert in other applications.

As soon as a user puts a block in, tdarr shouldnt try to guess at all. Just if the flow has no BEGIN & EXECUTE blocks in it.

Maybe that idea isnt that great as i though tho, i expected most flows only constisting of <10 execute blocks...

supersnellehenk commented 3 months ago

Yeah most flows will usually be < 10 execute blocks, but knowing when to place them is just guess work. You can't tell what the user intended, as most ffmpeg commands can be merged, but change what they do at the same time.

The easiest solution is to just remember to add a BEGIN and EXECUTE block after going through the tutorial flows. Chapter 3 specifically. image

The BEGIN & EXECUTE blocks are also helpfully positioned at the top of that list :) image

HaveAGitGat commented 3 months ago

Yeah a quick improvement may be to automatically add the begin and execute blocks around the ffmpeg command block when adding the first ffmpeg command block to a flow. Would make it a bit more convenient.