HaveAGitGat / Tdarr

Tdarr - Distributed transcode automation using FFmpeg/HandBrake + Audio/Video library analytics + video health checking (Windows, macOS, Linux & Docker)
Other
2.92k stars 92 forks source link

Allow Auto-Cancelling when soucerfileSizeInGbytes < outputFileSizeInGbytes by a chosen ratio #571

Closed xFaultx closed 3 months ago

xFaultx commented 2 years ago

Is your feature request related to a problem? Please describe. When doing transcoding, sometimes an outputted file is larger than the input. If my goal is to just save space, this is creating more space, time, and power (energy bill) of having my CPU or gpu work on a task that I am going to cancel or delete. Currently I just have to check it manually, but if I am seeing the outputFileSizeinGbytes is significantly larger, like 150-200% the size of the original file, I cancel the task manually.

Describe the solution you'd like Instead of manually monitoring the input vs output, have a threshold that once the output exceeds a defined %, like the outputFileSizeInGbytes is 150% of the sourcefileSizeInGbytes, the task is canceled, and the node can move onto the next task.

Describe alternatives you've considered Currently I have 2 friends looking at the API and seeing what they can sus out, there was a similar quesion related to pausing the work through the API. I dont think what I need is there, but I think it would help.

Additional context I feel like between 'get-nodes' and other api functions this shouldnt be impossible, but I know very little about this to know where to go.

HaveAGitGat commented 2 years ago

Hi, please see this issue which is a more efficient solution: https://github.com/HaveAGitGat/Tdarr/issues/568

HaveAGitGat commented 2 years ago

https://github.com/HaveAGitGat/Tdarr/issues/573

This isn't really a fix for the issue however. What you're describing would only really work when writing with media that has either the same/similar source quality, and/or similar end file size. I have some things that the source bitrate is lower because I don't need it to be higher quality. An example would be maybe some holiday movie that is something we throw on in the background once a year. But in the same library I have movies that I want to have very little quality loss, and sold naintIn that very high bitrate. The goal is to have lower quality movies stay low quality, and higher quality stay high.

HaveAGitGat commented 2 years ago

FYI you can use this plugin to automatically error files that are not within the required boundaries: Tdarr_Plugin_a9he_New_file_size_check

xFaultx commented 2 years ago

FYI you can use this plugin to automatically error files that are not within the required boundaries: Tdarr_Plugin_a9he_New_file_size_check

I am using it . The issue is that does it at the end of the transcode and a script capable of checking the outputsize vs sourcefilesize would catch that way earlier, possibly an hour or two before a cpu transcode was finished. So it would not only save time by stopping it before the render was over, but save electricity and money by preventing it from running unnecessarily

xFaultx commented 2 years ago

I guess one of the big things is apart from manually clicking the buttons for the info then the cancel, I don't of a way to issue a cancel command, such as through a command line in either the server or node protocols

HaveAGitGat commented 2 years ago

Did you get anywhere with using the API to do this? The info in there has the live output file size, so you could use that and kill the worker when needed. Else I can add this in a future version.

xFaultx commented 2 years ago

Did you get anywhere with using the API to do this? The info in there has the live output file size, so you could use that and kill the worker when needed. Else I can add this in a future version.

No I didn't, started down the rabbit hole and got a bit lost. If your are able to add it that would be monumental. Hate having my pc running some big transcode only to find out that it could have been stopped an hour ago.

I think what would be best is if it gave you the option to either completely cancel the transcode, or send it to a different folder rather than replacing the original file so that you could approve if it falls within certain size parameters whether it's too big or too small. So just the same as if the file is too big, if the files too small, you can decide to go and visually see it to see if actually it still maintained its quality. In most cases I'd want it to cancel, but still options are nice

xFaultx commented 2 years ago

Did you get anywhere with using the API to do this? The info in there has the live output file size, so you could use that and kill the worker when needed. Else I can add this in a future version.

Just verifying because I was having issues updating my setup - was this ever added. I didnt see it in the changelog, but havent fully explored the update

Kedryn commented 11 months ago

an advice Switch to Flows there is a plugin that does exactly that.

samssausages commented 7 months ago

an advice Switch to Flows there is a plugin that does exactly that.

@Kedryn Do you know what plugin helps with that? I'm trying to find it and only finding ways to compare after the file is transcoded. We are looking for a way to not transcode if it results in a larger file.

HaveAGitGat commented 3 months ago

Added with flow plugin Compare File Size Ratio Live. Click Update community plugins on the classic plugins tab or will auto update within an hour or so.

image image

Can compare between either the estimated file size or the current file size of the output file.

  Compare either the estimated final size or current output size to the input size and 
  give an error if estimated final size or current size surpasses the threshold %.

  Works with 'FfmpegCommand', 'HandBrake Custom Arguments', 'Run Classic Transcode' and other flow plugins 
  that output a file.

  Can be placed anywhere before a plugin which outputs a new file.
longmpham commented 2 months ago

I'm using the Docker Tdarr version 2.22.01 which should contain that feature above after reading the release notes. I've updated both the node and the server to the same versions and I've gone ahead and updated the Community plugins like you said. I still cannot find that option "Compare File Size Ratio Live"

When I search for the plugin, these are all I get image Notice here, I've tried to update the Community plugins twice: image

Did I miss something? Not sure if this matters, but my server is under an Ubuntu environment using Docker, and my Node is under a Windows environment not using Docker.

davidry commented 2 months ago

I'm using the Docker Tdarr version 2.22.01 which should contain that feature above after reading the release notes. I've updated both the node and the server to the same versions and I've gone ahead and updated the Community plugins like you said. I still cannot find that option "Compare File Size Ratio Live"

When I search for the plugin, these are all I get image Notice here, I've tried to update the Community plugins twice: image

Did I miss something? Not sure if this matters, but my server is under an Ubuntu environment using Docker, and my Node is under a Windows environment not using Docker.

The logs you have provided is regarding pulling the plugins from the node to the server Have you updated the plugins using "Update Community Plugins" from the classic plugin tab?

image

Screenshot_20240626_192513_Chrome

Can you provide the Tdarr_Server logs?

HaveAGitGat commented 2 months ago

Go on the classic plugins tab and click update community plugins (this updates both flow and classic plugins). While this is happening check the server log on the logs tab, probably will be an error of some sort.

Could try deleting the community plugins within the server data folder and rerun the update.

Can also download this file: https://github.com/HaveAGitGat/Tdarr_Plugins/tree/master/FlowPlugins/CommunityFlowPlugins/file/compareFileSizeRatioLive/1.0.0

And place it in your community flow plugins (the folder structure is the same).

longmpham commented 2 months ago

Hmmm nothing reported in the logs, so I nuked the docker container again (docker compose down tdarr), and then started it back up (docker compose up -d) with explicit version tagged in the compose file (image: ghcr.io/haveagitgat/tdarr:2.22.01). That seemed to have worked.

Thank you for getting back to my problem so quickly.