add flag to keep temporary files on hosts

[ipatch]

Is it possible to add a flag to keep the temporary files on the specified hosts?

Example

shepherd input output -c localbox.local,remotebox --length 120 --tmp $(pwd)

Running the above command does not keep any of the temporary files, which makes it difficult to know where the program failed or if one just wants to see what got split where and inspect the files that were used to generate the final output file.

It'd be nice to be able to run the above command with something ala -k for short, and --keep for long to store the temporary files if the process fails or completes.

Awesome piece of software by the way, spent a little while to see if even such a thing was possible, and this about the only app I came across.

cheers 🍩

[martindisch]

Hi Chris, thanks for the kind words. I'm super happy you find it useful, because when I searched for a tool like this, same as you did, I couldn't believe nobody had really done it yet. I'm actually pretty surprised you found this and curious now: do you have the exact same workflow I do (DNxHD in MXF files), do you have different files but it works anyway, or did you have to adjust the ffmpeg commands in the code?

Debugging when something goes wrong really is a pain, so your requested command makes complete sense. I don't have time to work on this as much as I'd like, but this is a pretty straightforward proposal, so I went ahead and implemented it. It only keeps the files on the encoding hosts around, not on the local one (because that's how I understood your request). If you'd like a different behavior, that should be easy to change.

[ipatch]

Hi Martin,

I actually thought it would be beneficial to keep all temporary files local and remote, very cool that you implemented that feature so fast, I wish I knew more about the rust programming language so i can send you a PR, but unfortunately I do not know much about rust at all, other that building packages.

I came across the package just doing some searches on github, and it seemed like the best app / program on github for distributing encoding across multiple machines as I couldn't find any other apps that would do such a thing.

As for my use case, I am not working with the same type of files as you are, but was more or less just looking for a project / solution to experiment with reencoding / transcoding video files across systems I have access to. I primarily do screen casting for youtube, and have recently switched from OBS to using ffmepg from a CLI using lossless audio WAV and video libx264 and really like the output put reencoding can take some time.

And yeah, I saw in your readme for the project that adding / passing ffmpeg commands to the program to where one could choose the ffmpeg settings of their choice as opposed to use the hard coded settings contained within the app, and having to modify those settings, then recompile, isn't exactly an eloquent solution, but you def have a great starting point for the whole process.

I hope that answers some of your questions.

[martindisch]

I actually thought it would be beneficial to keep all temporary files local and remote

No problem, I just changed it to do that. It makes sense anyway, since the audio is split off and encoded locally, so if you really want to look into the whole process, it's a good idea to retain the local files too.

Thanks for telling me about your workflow. It's cool you're making the switch to something that gives you more control over the output. I like to pursue the best possible quality myself and know that it's a real rabbit hole once you go into individual encoding parameters.

I wish I had the time to make this more universally useful, maybe in a few months if all goes well.