Idea: DNG recompression

[leahneukirchen]

If dnglab could read uncompressed DNG and export exactly the same DNG, just with lossless compressed image data, that would simplify long term storage. AFAIU no such tool exists for Linux so far, even tho all necessary parts of it are implemented multiple times.

[cytrinox]

What's your workflow to start first with uncompressed dng instead of lossless compression? Can't imagine a situation where this is useful. I've ideas for implementing a reconvert and/or recompress command to reconvert the embedded original raw (or strip it) or for recompress to non-linear or 8 bit lossy jpeg.

[leahneukirchen]

I have an old camera (Pentax K200D) which can shoot uncompressed DNG only, which take up twice the space then. So compressing after the fact sounds useful for storage. This is probably the case for a bunch of other cameras too.

(It can also shoot compressed PEF, but rawspeed doesn't support it currently... we're working on that, though.)

[leahneukirchen]

I also learned that Adobe SDK validate_dng can be used to recompress DNG, but trying to compile it on Linux seems like a rabbithole, so having a modern Rust solution would be great. :)

[cytrinox]

Okay, that makes sense. Yeah, with my idea for reconvert/recompress, your problem should be automatically solved, but it was not the first intention :)

[cytrinox]

Can you please attach CC-0 sample raw files from your K200D in DNG and PEF? If PEF can be compressed/uncompressed or some other mode, please add all modes. Thanks. Maybe the zip get's to big, so you can use services lie wetransfer.com Already found the new sample set on raw.pixls.us, thanks.

[cytrinox]

@leahneukirchen not what you requested but: https://github.com/dnglab/dnglab/pull/120 just landed in main. If you want to test I would be happy about feedback. It's not yet feature complete but most of things should work.

[leahneukirchen]

I'll try! thx

[leahneukirchen]

Some notes:

• conversion fails when output file exists already
• --compression lossless --dng-embedded false turns 11MB PEF to 9.7MB DNG (nice!)
• i could not make dnglab analyze do anything

[cytrinox]

Thanks! Existing files should be overriden with -f. analyze command is not yet stabilized, but --meta --yaml should already show some output.

[leahneukirchen]

Works for .PEF, for a (camera-made) .DNG I get

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: General("File has not makernotes")', bin/dnglab/src/analyze.rs:21:57

[leahneukirchen]

For the dnglab-converted DNG I get

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Unsupported("Couldn't find camera \"PENTAX\" \"K200D\" mode \"\".\nPlease submit samples at https://raw.pixls.us/")', bin/dnglab/src/analyze.rs:21:57

[embeeweezer]

What's your workflow to start first with uncompressed dng instead of lossless compression? Can't imagine a situation where this is useful. I've ideas for implementing a reconvert and/or recompress command to reconvert the embedded original raw (or strip it) or for recompress to non-linear or 8 bit lossy jpeg.

I would be very interested in an integrated lossy dng compression option! I also tried using the https://github.com/gopro/gpr for compressing dng files into gpr but it wouldn't accept the dnglab dng files :(

[madwyn]

SONY cameras with uncompressed ARW options all can benefit from converting to compressed DNG. I converted all my RAW files to DNG, which might be a bad idea since you can't revert them, but for ARW files, there is in general 40-50% space saving. Even some cameras with compressed RAW still get further compressed, usually around 12-20%.

I didn't dive into lossy DNG, but I couldn't tell the difference between a lossy DNG and a lossless one.

I'd be very interested to see in future DNG versions to include JEPG XL for lossless and lossy compression, which can bring even better compression rate.

[se6astian-pi]

+1 for feature request of lossless compressing uncompressed DNGs.