Log file name after failed transcode is not predictable

[zb140]

I'm working on what is basically a batch tool with a UI that shows things like which files are still pending, which have been completed, etc. It also detects when transcoding failed and will offer a few options for handling it (skip, retry, that kind of thing).

I would like for my tool to be able to actually display the failed log file (to make it easier to see what the failure actually was), but the issue I'm running up against is that in the failure case the log file name is a temporary one. This makes it difficult to correlate the log file with which video it belongs to. In a situation where I start a bunch of encodes running and go away for a while, if several have failed when I come back, the only way I can see to figure out which ones they were is to try to parse it out of each log file.

One thought I had is, would it be possible to rename the file to match the video file name not just on success but on failure too? I'm also open to any other ideas anyone might have that don't involve parsing the logs :smile:

(I hope this makes enough sense. If not, just ask :smile:)

[lisamelton]

@zb140 I must say that this is a unique request. I never imagined anyone would depend upon that filename. :)

I definitely don't want to make a failure .log file have the same name as a successful .log file. But perhaps I can still make that filename predictable. For example, on failure I could rename the temporary .log file using a different file extension. But that trick would only work if I can recover from the failure to do the renaming.

You see, the problem is the way ffmpeg generates .log files. I can't name the temporary .log file anything I want because there are restrictions on whichcharacters I can pass to ffmpeg. It's really weird and annoying. So I can't just start out with the temporary filename being the same as either the success or failure name.

Obviously renaming the temporary file on success is easy because my code is still running normally. But some failure modes can mean that execution never returns to my code.

Anyway, I will consider this. Thanks for the idea!

[zb140]

Thanks for the response, and for being willing to think about this! I definitely agree with not wanting to make a failure log look exactly like a successful one -- predictability of the filename would be enough to solve my problem.

Here's another idea that just came to mind: would you consider, in --debug mode, writing the name of the temporary log file to stdout (or stderr for that matter, either one works for me)? I'm already parsing that output to track progress, so it wouldn't be hard to also grab the file name that way. That would save any need to rename the file at all, so you wouldn't need to worry about being able to recover from failures in order for it to work.

[lisamelton]

@zb140 Yes! I would consider that. And I would have to write the name to stderr because ffmpeg pretty much owns stdout when it's running.

[zb140]

Awesome, that would completely solve my problem. I know it's a low-priority request for an uncommon use case, but it would definitely make my experience better :smile:

[skj-dev]

Just making a "peanut gallery" comment here. From my perspective you are on the right path with

...parse it out of each log file

Since you're already in the context of writing code to do stuff, adding a parser that lines up a _ffmpeg file with a source might be a simpler task and easier to troubleshoot than trying to build in predictable temporary file name support. Just off the cuff, something like this would produce the input file associated with a _ffmpeg filename stored in a $FFMPEGLOG variable.

grep '^Input file' $FFMPEGLOG | cut -d '(' -f2 | cut -d ')' -f1

I realize that your environment might not line up with shelling out to a UNIX/Linux system, and just wanted to provide an example of what I am talking about. FWIW, using that example, it could be tossed into a loop that cycles through many _ffmpeg files, etc.

[lisamelton]

Thanks, @ttyS0!

[zb140]

Thanks, @ttyS0 -- something like that would certainly be better than reinventing the wheel by writing all the necessary code myself. But to be clear, I'm no longer asking for the temporary log name to be predictable, just for an extra output line in the --debug mode output that says what it is. @donmelton, if you don't want to do that, then that's the way it goes -- that's why it's a "feature request" and not a "feature demand" :smile: -- but in my view it's a better solution than reading the log file(s) after the fact.

Arguably it might even be helpful to other users too. Right now, if you run other-transcode on a file that ffmpeg doesn't like for whatever reason, the output will be a zero-length .mkv file and a weirdly named log that wasn't even obviously created by other-transcode. Telling them, even just in --debug mode, that that file is being created could help connect those dots.

[lisamelton]

@zb140 So it turns out that I'm already writing the temporary filename to the --debug output. :)

I just did a --debug build and noticed this line:

Report written to "_ffmpeg_21244_18848.mkv.log"

Are you able to parse that?

[zb140]

@donmelton Huh, I could've sworn I checked that... In any case, that's great! As long as it's going to stdout or stderr (surely it must be if you saw it on your console) I shouldn't have any problem pulling it out. I won't have time to actually double check until Thursday night, but it seems likely that that will solve my problem. Thanks!

[lisamelton]

@zb140 No worries, I didn't know it was there either. :) To be clear, it's ffmpeg that's writing that line, not other-transcode explicitly. Which means it may be a new behavior in ffmpeg version 4.3. I don't know. I'll try to find out for sure this week.

[zb140]

@donmelton Yep, that works! Also, I checked and I'm definitely using ffmpeg 4.3, so maybe you're right that that's where it came from. I don't think I recently upgraded it on purpose, but it could easily have happened automatically.

Thanks again (and for the excellent tools)!

[lisamelton]

@zb140 Thanks for checking and you are very welcome!