unknownbrackets / maxcso

Fast cso compressor
ISC License
390 stars 23 forks source link

How do I redirect/export maxcso text output to a log? #68

Closed jeffrey-lebowski closed 2 years ago

jeffrey-lebowski commented 2 years ago

Since maxcso can be operated via the command line, I was hoping to batch script the conversion process for my entire PSP collection. However, from what I can tell, maxcso text output isn't redirectable via streams at all. Consequently, I don't know how to create a log that would allow me to double check that the conversion was successful for each game or inform me of any errors...

How can I export/redirect any errors as well as output from maxcso --crc input.iso or maxcso --crc output.cso to a log?

Many thanks to all the people who worked on this project.

unknownbrackets commented 2 years ago

On Linux, you can use script (I assume.) For example:

script
maxcso --crc *.iso *.cso
^D
cat typescript

That said, for crc and the final status of compression for a line, it probably makes sense to output that even when not attached to a tty. Currently it intentionally suppresses output when not using a tty (which happens sometimes in scripting and usually means the ANSI it uses to display progress % won't be processed properly.)

I may get to it later, I would also accept a pull request as long as it maintains the current output pattern and progress display on a tty.

-[Unknown]

unknownbrackets commented 2 years ago

I've made it so that stderr can be piped to a file and show final results, including CRC.

Use:

maxcso --crc input.iso 2> maxcso-crc.log

The 2> pipes stderr, while > or 1> pipes stdout. I might in the future detect if stdout is a TTY or pipe, and send success to stdout instead so you could do > and get progress still in the terminal. If you have an opinion on whether that would be useful or detrimental, let me know.

-[Unknown]

jeffrey-lebowski commented 2 years ago

Thanks for adding this feature. I'm sure that it'll be useful to many people.

I should've mentioned before that I'm using maxcso on Windows. I used a workaround with powershell and converted my entire PSP library. If you were to release an updated exe, I'd be happy to test it and use it to verify checksums in the future.

In terms of your question, then I'd say that having the progress still displayed in the terminal and having a "SUCCESS"/"ERROR"/<crc> output send to a log after the process completes would be best.

For me, the most the most important features (on top of the conversion itself) is to have the conversion result (success/error) and the CRC (of the iso) logged when I batch convert multiple games.

Example:

cmd input

(1) for %a in (*.iso) do (echo %~na & maxcso "%a") >> maxcso.log or for %a in (*.iso) do (echo %~na & maxcso "%a") >> maxcso.log 2>&1 (2) for %a in (*.cso) do (echo %~na & maxcso --crc "%a") >> maxcso_crc.log

Log output

(1) <name of the game> SUCCESS/ERROR (2) <name of the game/cso> <crc of the iso used to create the cso>

When pointed at a .cso file maxcso already calculates the crc of the underlying iso and not of the .cso file itself, which is phenomenal. Sadly, the most popular compression format/program of them all, i.e. chd/chdman, doesn't offer such luxuries. This makes it extremely painful to verify original/source files/isos against redump/NI and, afaik, necessitates decompressing the files first...

unknownbrackets commented 2 years ago

You can try this: https://github.com/unknownbrackets/maxcso/actions/runs/2405125828#artifacts

These aren't release builds, so they're not using PGO (profile guided optimization) and are therefore a bit slower. But the speed impact is most significant with zopfli, and already small for general compression. But for CRC calculation the speed difference should be unnoticeable.

Also, in case it helps, maxcso will give a failure exit code if any file compression fails in any case. For compression at least, you could use this to fail early:

for %a in (*.iso) do (
  maxcso "%a" || (echo %a FAILED && exit /b 1)
)

Or to move successfully processed files to a subfolder:

if not exist compressed (
  mkdir compressed
)
for %a in (*.iso) do (
  maxcso "%a" && move "%a" compressed
)

That doesn't help for CRC, though (the CRC operation will essentially never fail unless there's a disk read error.)

Thinking I may make it so if you pipe stdout to something other than a TTY, it will output to stdout.

-[Unknown]

jeffrey-lebowski commented 2 years ago

Thanks for your continued support @unknownbrackets!

I did some quick testing and just wanted to note two things.

There seems to be a small error here:

for %a in (*.iso) do (
  maxcso "%a" && move "%a" compressed
)

It'll move the iso to the "compressed" folder rather than the cso. This should fix it:

for %a in (*.iso) do (
  maxcso "%a" && move "%~na.cso" compressed
)

The following code works as expected, which is really great and what I always wanted to see:

maxcso --crc input.iso 2> maxcso-crc.log maxcso input.iso 1> maxcso-error.log

However, it's been my understanding that errors are supposed to be redirected via 2>, whereas command outputs are supposed to be redirected via 1>. It seems to be the other way round in this case. It makes no difference to me personally, just wanted to mention it :)