[Question] How to do a two-stage flow?

[jrruethe]

Describe the bug My problem is similar to this one, where I want to accomplish the following:

• (Stage 1) Clean the file, remux it, save the result
• (Stage 2) Attempt to reencode it, but if it gets larger, keep the result from Stage 1

The main issue is that the "Set Original File" resets back before Stage 1, regardless of my "Replace Original File".

My attempt to delete the working file at the end also fails. (This might be the actual bug?)

---

To Reproduce Steps to reproduce the behavior:

I attached both the flow and the job report flow.txt job_report.txt

The error at the bottom, in the "Delete File" step, shows:

2024-09-05T15:45:43.339Z l2MQH6T1jN:Node[tdarr]:Worker[rash-rhea]:Deleting working file /cache/tdarr-workDir-node-UlmXFsCYJ-worker-rash-rhea-ts-1725551093635/1725551107093/Bluey (2018) S03E30 Turtleboy.mkv

Followed by:

2024-09-05T15:45:43.341Z l2MQH6T1jN:Node[tdarr]:Worker[rash-rhea]:Final working file is still in the transcode cache, not allowed: /cache/tdarr-workDir-node-UlmXFsCYJ-worker-rash-rhea-ts-1725551093635/1725551107093/Bluey (2018) S03E30 Turtleboy.mkv. The file should do something such as replace the original file or be moved to an output folder. You can also use the 'Set Original File' plugin to reset the working file back to the original file if you've already copied the file somewhere else and not made it the new working file.

which implies that the delete didn't actually occur? No permission issues.

---

As a side note, in the Delete File plugin, if "fileToDelete": "workingFile" and "deleteParentFolderIfEmpty": "true", I expected it to delete the parent folder of the working file, NOT the parent folder of the original file:

2024-09-05T15:45:43.340Z l2MQH6T1jN:Node[tdarr]:Worker[rash-rhea]:Folder /media/cartoons/Bluey (2018)/Season 3 is not empty, skipping delete

---

Expected behavior

I expected two things:

• That after I use "Replace Original File", I could use "Set Original File" to essentially revert to the file state after the most recent "Replace Original File", instead of all the way to the very beginning. This is documented, but there doesn't seem to be a way to do it the expected way.
• That using "Delete Working File" after "Replace Original File" would allow the flow to exit cleanly with the results of Stage 1, without the "Delete File" step failing.

---

Please provide the following information: Using version 2.24.05 in Docker

[jrruethe]

My flow is a bit big, I can't get a good screenshot, but here it is. The left column is "Stage 1", the middle two columns are "Stage 2", and the rightmost column decides if the output of Stage 1 or Stage 2 should be kept.

I had thought that using "Replace Original File" after Stage 1, then either "Replace Original File" or "Delete Working File" after Stage 2 would accomplish what I want, but its not quite there.

[HaveAGitGat]

When the flow ends, the current working file is what will be kept in Tdarr's database (the file that shows in the Transcode: Success/Not Required and contributes to the Tdarr stats). So if the current working file at the end of the flow is deleted or in the cache then it will cause a flow error. So if you only replace the original file at the end of the flow then should be no issues. And you don't need to manually delete files still in the cache as they're automatically cleaned up.

[HaveAGitGat]

Are any of your plugins renaming the file? If not then using Set Original File after Replace Original File should still point to the new file as they have the same name.

[jrruethe]

I am not doing any renaming.

Ok, so maybe I misunderstood what was happening behind the scenes. You are saying:

• Do "Stage 1" (which is the cleaning / remuxing), then call "Replace Original File" to ensure that the result is kept
• Do "Stage 2" (which is the reencoding), decide if I want to keep the result (overwriting Stage 1)
• • If keeping, run "Replace Original File"
• • • The result will be that Sonarr has my Cleaned AND Reencoded file from Stage 2
• • If discarding, run "Set Original File" (NOT "Delete File")
• • • The result will be that Sonarr has my Cleaned file from Stage 1, but not the Reencoded file from Stage 2
• • • Calling "Set Original File" will NOT cause the result from Stage 1 to be lost

Is this correct?

[HaveAGitGat]

That should work but I assume Set Container will sometimes change the filename if the original container is different so would not work in that case. So we'd need a new type of plugin to handle this type of situation.

[jrruethe]

It seems like I need a way to end a flow successfully that does not require replacing the file.

"Replace Original File" appears to be the way to "end" a flow successfully, and results in the working file replacing the original file. But I'm not sure how to "end" a flow successfully while simultaneously discarding the working file. I thought "Delete Working File" was the way to do this. "Set Original File" doesn't appear quite right either, because I don't want to rewind all the way back to the beginning.

My current workaround is to use the "Review Required" node, which allows me to use the "Skip" button in the staging window. That is better than failing the flow and selecting "ignore". But this causes my staging area to fill up, which stops the node from processing, until I manually clear it.

I suppose another alternative would be a way to assign multiple flows to a library, and require all flows to complete (in a specified order). Then I could split my "Stages" into two flows, and the second flow would be able to use "Set Original File". However, that is a complicated feature request.

Thanks for your help and responses!

[jrruethe]

For what it is worth, I tinkered some more and came to the same conclusion as this issue, where I do the reencoding first, then the cleaning.

This way, my "Stage 1" determines whether to reencode. If it does, it determines if the reencoding should be kept. If so, replace, otherwise reset to original file. Then "Stage 2" does the cleaning operations, which will either be on the original file or the reencoded file.

Therefore, I think I have my problem solved. I'll close this issue.

Here is my new flow: new_flow.txt