Closed Routhinator closed 10 months ago
Routhinator
commented
10 months ago RE: Healthchecks and the hardware detection for encoding in Flow - I've thought about this more and seen more jobs complete and think the hardware encoding support built into the Set Encoder flow plugin is actually working fantastically. It would be really really good if we had the option to tell healthchecks to use this same logic to determine what flags to set with the check.
HaveAGitGat
commented
10 months ago Hi thanks for the feedback, yeah I think for some users flows can feel a bit more complicated at first but once you get into it it makes conditional branching stuff easier, overall feedback has been good.
Not sure on the language stuff atm will see if I can improve in the new year, for now would keep using the classic plugin from within the flow.
Regarding the hardware detection, you should be able to see this in the job report. It runs small test encodes on a virtual file to see what encoders are available which should show in the job report. Code for that is here:
QSV is a bit of a PIA as seems it has different commands for different FFmpeg versions and operating systems so we'd need to add a test for your hardware to the code above so that it works for you.
Routhinator
commented
10 months ago Regarding the hardware detection, you should be able to see this in the job report. It runs small test encodes on a virtual file to see what encoders are available which should show in the job report. Code for that is here:
QSV is a bit of a PIA as seems it has different commands for different FFmpeg versions and operating systems so we'd need to add a test for your hardware to the code above so that it works for you.
FWIW, it does seem that the hardware detection is working. It's using QSV on HEVC capable procs and falling back when they don't support the right codec. I've not test with an older codec to see if it uses QSV on the old procs when it can but looking at the logic I think it should.
What I've noticed is that health checks are not currently using the same test, and thus try to blindly use QSV/CUDA on workers based on the worker flags instead. It would be good to get healthchecks adopting this logic.
Any suggestions on the "Replace file" front? I'm trying to work how to set a flag or something if it enters any loops that actually change the file and only replace if it needs too - it's currently replacing any file that runs through transcode queue regardless of actions taken.
HaveAGitGat
commented
10 months ago Yeah the flows system is using different logic to what is used internally for the health checks but I'll try to get everything on the same page in the coming weeks.
Any suggestions on the "Replace file" front?
Are you able to share a job report?
No file replacement is done if the working file is still the original file (ie no changes have been done and no new cache file has been produce).
Can see the logic here: https://github.com/HaveAGitGat/Tdarr_Plugins/blob/master/FlowPluginsTs/CommunityFlowPlugins/file/replaceOriginalFile/1.0.0/index.ts#L44
Routhinator
commented
10 months ago Ah, looking at it more it did reorder the streams. So no file size change or transcode, but it did optimize the file. The report in the list says "Not Required" though which makes it misleading.
Routhinator
commented
10 months ago Going to close this. Thanks for the discussion (perhaps the Github discussions feature would be good here?).
Hey there, I want to start off by saying that this pro supporter really appreciates all the work you've done with Flow.
While I no longer use Reddit, I've stumbled across some posts there as I was digging into Flow where people were complaining that this makes things more complicated - and my experience really runs counter to that. In the plugin based system I was constantly weeding through multiple variants of plugins to get the exact logic I wanted in my plugin processing, and often times I couldn't find exactly what I needed and had to edit the plugin locally, and I also ended up with a lot of logic overlap between plugins.
With flow, I've been able to drop all my plugins and I have 90% of what they were doing recreated. One thing is left outstanding, which is replicating the behaviour of the
Remove all langs except native and Englishplugin. The core logic is recreatable with regard to removing the unwanted language streams, but what's missing is a way to query the radar/sonarr APIs for a TMDB id, and a way to query TMDB for the native lang. I imagine this could be done with a vanilla API query flow plugin, where given a specific URL and auth header (if required) a specific property in the response could be used in the next flow step. Not sure what you see as the way forward there. For now I've dropped this from my flow but if it really becomes a sticking issue I can bring that plugin into the flow.Thus ends the feedback portion, and into the question section! :)
How does the FFMPEG Set Video Encoder "auto" hardware setting work? I don't seem to be having success with it. I mean it seems to work when the job hits a CPU worker as it rightfully detects it has no hwaccel to work with and uses non-accelerated encoding, however it does not seem to be detecting the correct hwaccel on a GPU worker. In my case my workers support VAAPI and QSV, not CUDA, but it keeps trying CUDA.... I have thought that this may be improved if I set the workers to QSV explicitly however that excludes VAAPI should something need it.. I think? The help bubble for that setting on nodes only describes the nevc option so I'm uncertain if QSV is similar and includes VAAPI.
An additional complication that's exclusive to me, is that while my nodes have working QSV and VAAPI support, they are Gen 3 and Gen 4 CPUs and do not support HEVC - so IF the library is trying to convert to HEVC and lands on a GPU working on this node, it should be falling back to CPU based commands.
Whats the best way to approach the above scenario with Flow? Is there a way to have a decision tree based on worker capabilities?
EDIT: Y'know, just struck me that the failures I've seen are only with healthchecks, maybe I just need to drive them to the CPU workers only since there doesn't seem to be much control around healthcheck flows. Setting QSV on the nodes works to tell the healthchecks to try QSV instead of CUDA, however it fails with HEVC files as it doesn't understand the generation of the node and the codecs it supports.
One more. What's the best way to insert a "file was changed" condition in this flow before the "replace file" - Just realized the workers are still replacing the original file even when the transcode result is "Not required".