Canon CR3 support. - Githubissues

LebedevRI commented 6 years ago

Though, until the production camera is actually released, and the full set of freely-licensed samples is contributed to the https://raw.pixls.us/, it won't be merged.

LebedevRI commented 6 years ago

Does not appear to be TIFF. Appears to be ISO Media / ISO/IEC 14496-12. The actual content of the 'mdat' box appears to be JPEG-ish, but not old-good lossless-JPEG, because it contains M_DQT (define-quantization-tables marker)

$ file canon_eos_m50_01.cr3  
canon_eos_m50_01.cr3: ISO Media

$ mpv canon_eos_m50_01.cr3
Playing: canon_eos_m50_01.cr3
 (+) Video --vid=1 (*) ( 6000x4000 1.000fps)
     Video --vid=2 (*) ( 1624x1080 1.000fps)
     Video --vid=3 (*) ( 6288x4056 1.000fps)
Failed to initialize a video decoder for codec ''.

LebedevRI commented 6 years ago

Hmm, ok, this is strange. Got the ISO Media parser working, and hooked-up dummy Cr3Decoder. The mdat chunk is NOT lossless jpeg. But it just loads via libjpeg. Those CR3 appear to be plain demosaiced JPEG's... (Yes, this is an actual decoded image) 0001_

LibRaw commented 6 years ago

According to ffprobe, mdat chunk contains four substreams. First one is, exactly, JPEG preview, so from libjpeg's point of view this is JPEG file + ~30Mb of garbage (real raw data + makernotes) on tail.

LebedevRI commented 6 years ago

Aha, thank you, that makes more sense. Let's see...

LibRaw commented 6 years ago

unfortunately, two data chunks (1624x1080 and 6288x4056, first one probably is low-res preview) are not lossless jpegs too :(

LebedevRI commented 6 years ago

(I'm still hoping they are JPEG w/arithmetic coding)

LibRaw commented 6 years ago

It does not looks like JPEG (e.g. no FF doubling).

LebedevRI commented 6 years ago

There are 0xFF0xFF sequences in files 05, 11, 19, (looking at the second half of the file in okteta hex editor)

clanmills commented 6 years ago

Thank you folks. I'm going to keep out of a discussion about the pixels and keep to my domain of metadata. Thanks to your comments, I've now heard of ffprobe.

822 rmills@rmillsmbp:~/gnu/github $ exiv2 -pR foo.jpg 
STRUCTURE OF JPEG FILE: foo.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffdb DQT   |     132 
     136 | 0xffc0 SOF0  |      17 
     155 | 0xffc4 DHT   |     418 
     575 | 0xffda SOS  
823 rmills@rmillsmbp:~/gnu/github $ ffprobe foo.jpg 
ffprobe version 3.4.1 Copyright (c) 2007-2017 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-librsvg --enable-libtheora --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-jack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-indev=jack --enable-opencl --disable-outdev=xv --enable-audiotoolbox --enable-videotoolbox --enable-sdl2 --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-x86asm --enable-libx265 --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, image2, from 'foo.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 7605132 kb/s
    Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 6000x4000, 25 tbr, 25 tbn, 25 tbc
824 rmills@rmillsmbp:~/gnu/github $

I'll work on the metadata. We've found the XMP data and I'm confident the Exif (and IPTC) data is in the Canon uuid box at the top of the file.

Later, I will investigate previews and ICC profiles.

LebedevRI commented 6 years ago

I'll work on the metadata. We've found the XMP data and I'm confident the Exif (and IPTC) data is in the Canon uuid box at the top of the file. Later, I will investigate previews and ICC profiles.

Thank you, @clanmills! Sounds awesome! :)

lclevy commented 6 years ago

Phil Harvey exiftool 10.82 can parse Canon CR3 it seems there are at least 3 pictures: THMB, PRVW, and other in mdat

lclevy commented 6 years ago

for http://img.photographyblog.com/reviews/canon_eos_m50/photos/canon_eos_m50_01.cr3 mdat contains "trak" picture at 0x6d940, 0x37b030, 0x546c70, and TIFF metadata at 0x2565b98

LebedevRI commented 6 years ago

Yep, that much was understood. Need to continue working on that code.. Hopefully tomorrow?

lclevy commented 6 years ago

I started a CR3 format description here https://github.com/lclevy/canon_cr3 if you are interested in contributing... I've done it for CR2 here http://lclevy.free.fr/cr2/ and https://github.com/lclevy/libcraw2

clanmills commented 6 years ago

Laurent

This is very helpful indeed. I wrote an email to you to ask if you were considering working on CR3. Then I made good progress and decided not to send the email.

I've documented my progress: https://github.com/Exiv2/exiv2/issues/236

Team Exiv2 will have a meeting on May 5 at my home in England. This project will be discussed. There are many demands on the team and only a few contributors. In honestly, I don't think we have the resources to implement support for CR3 in 2018. https://github.com/Exiv2/exiv2/issues/225

lclevy commented 6 years ago

Hi, a lots of updates at https://github.com/lclevy/canon_cr3, you can subscribe to the project to see updates

clanmills commented 6 years ago

Thanks for doing this. It's looking good.

I'm puzzled by the claim (also made by @lclevy and @LibRaw) that there are 4 images in MDAT. I only see a single JPEG. I see there are 4 trak objects which are a hierarchy of items (CRAW, stsz, co64). Perhaps you could document the meaning of the objects in the trak hierarchy.

532 rmills@rmillsmbp:~/gnu/github/isobmff/pyke369/isobmffdump $ ./isobmffdump --dump stsz ~/Downloads/canon_eos_m50_01.cr3 
@0         | ftyp [24]
@24        | moov [28440]
@32        |   uuid [26216]
@26248     |   mvhd [108]
@26356     |   trak [484]
@26364     |     tkhd [92]
@26456     |     mdia [384]
@26464     |       mdhd [32]
@26496     |       hdlr [33]
@26529     |       minf [311]
@26537     |         vmhd [20]
@26557     |         dinf [36]
@26565     |           dref [28]
@26593     |         stbl [247]
@26601     |           stsd [128]
@26617     |             CRAW [112]
@26729     |           stts [24]
@26753     |           stsc [28]
@26781     |           stsz [20]
                       00000000  00 00 00 00 00 31 ea 2f 00 00 00 01                                                               .....1./....
@26801     |           free [15]
@26816     |           co64 [24]
...

lclevy commented 6 years ago

There are 4 parts in mdat: 3 pictures (JPEG, plus 2 with new compression), then metadata like in tiff

clanmills commented 6 years ago

@lclevy I do not agree that there is a tiff in the MDAT. Here's my evidence: https://github.com/Exiv2/exiv2/issues/236#issuecomment-372625849

We agree that there is at least one JPEG in the MDAT. I'm not convinced there are 3 pictures, however I have no proof to assert there is only 1 picture.

clanmills commented 6 years ago

@lclevy Apologies. I am mistaken. There is a valid tiff in the MDAT and ExifTool reads it.

@LebedevRI Thanks for pointing this out. By coincidence I discovered and documented this before your email arrived. Two heads thinking of the same thing! That's better than no heads thinking! https://github.com/Exiv2/exiv2/issues/236#issuecomment-372633645

Steady progress. Great Teamwork. Thanks everybody.

lclevy commented 6 years ago

updates: THMB and PRVW detailed. confirmed 3 jpeg and their sizes

clanmills commented 6 years ago

Well done, @lclevy. Very impressive progress. It's possible I now have sufficient information to parse all the CR3 metadata and thumbnails in Exiv2.

lclevy commented 6 years ago

please find parse_cr3.py ...

clanmills commented 6 years ago

@lclevy Happy Easter. We were on vacation last week and now I'm home and would like to look at parse_cr3.py. Where have you put it?

lclevy commented 6 years ago

https://github.com/lclevy/canon_cr3

clanmills commented 6 years ago

Thanks. That worked first time. I'll study that. Thanks very much.

simonszu commented 5 years ago

Please allow me this one question: Is this still worked on? I have recently bought a M50 and was hoping for linux support since Lightroom already had it. But for now i purely rely on JPEG output, which is a bit sad.

LebedevRI commented 5 years ago

Presently - no. It will eventually be done. I should probably take a stab at this.

clanmills commented 5 years ago

I haven't been able to work on this either. Since July 2017, Exiv2 has been swamped by security issues. I'm glad to say that every issue has been fixed and we will release Exiv2 0.27 on 28 December 2018. We intend to offer "dot" releases as necessary during 2019 and 2020 to address security issues.

It has been a major undertaking to deal with security and very little feature development has been possible. CR3 support is on the "highlight" features for v0.28 and will be discussed at a team meeting in December. https://github.com/Exiv2/exiv2/projects

The Team may, or may not, agree to undertake this work in 2019.

For sure, I will not do this work. I will be 68 in January and have been contributing daily to Exiv2 for more than 10 years. During 2019 and 2020, I will be the release engineer (to do the dot releases), and deal with user questions. However I want to retire and no longer contribute to the C++ code.

I'm delighted to say that the future of Exiv2 is secure. In 2017 I recruited two outstanding engineers and they have dealt with most of the security issues. They have also rewritten the build and test infrastructure.

Several observations about open source:

1) Many people request changes 2) Few folks volunteer to help 3) User contributed patches usually result in lots of work concerning platforms and testing

Adobe have 10,000+ people including several thousand C++ Engineers. I know, I worked for Adobe in San Jose, California for more than 10 years. Team Exiv2 has been mostly a single volunteer. Surprise, Lightroom Supports CR3 before Roman, Laurent and Exiv2 can catch up.

If you have any good ideas about how to recruit engineers to join Team Exiv2, I'll be delighted to receive suggestions.

clanmills commented 5 years ago

I do have a practical suggestion to make about this. Canon should be contributing to open-source. When they decide to launch a new format on their users, I believe they have engineers in "Developer Relations" who will engage with Adobe, Google, Apple, Microsoft and others to ensure that PhotoShop and other major applications provide support for their format.

I have never received any communication from Canon or any other camera manufacturer. The MakerNote Support in Exiv2 has been reversed engineered. Laurent has done very good work to reverse-engineer CR3. Canon sold the camera and contributed nothing to the open-source community.

So, @simonszu Would you write to Canon and express your disappointment that they have not supported the open-source Community to adopt CR3.

simonszu commented 5 years ago

@clanmills That's an excellent idea. I'll do it right now.

lclevy commented 5 years ago

Adobe is certainly using code from Canon

⁣Envoyé par Blue

Le 27 nov. 2018 à 11:43, à 11:43, Robin Mills notifications@github.com a écrit:

I haven't been able to work on this either. Since July 2017, Exiv2 has been swamped by security issues. I'm glad to say that every issue has been fixed and we will release Exiv2 0.27 on 28 December 2018. We intend to offer "dot" releases as necessary during 2019 and 2020 to address security issues.

It has been a major undertaking to deal with security and very little feature development has been possible. CR3 support is on the "highlight" features for v0.28 and will be discussed at a team meeting in December. https://github.com/Exiv2/exiv2/projects

The Team may, or may not, agree to undertake this work in 2019.

For sure, I will not do this work. I will be 68 in January and have been contributing daily to Exiv2 for more than 10 years. During 2019 and 2020, I will be the release engineer (to do the dot releases), and deal with user questions. However I want to retire and no longer contribute to the C++ code.

I'm delighted to say that the future of Exiv2 is secure. In 2017 I recruited two outstanding engineers and they have dealt with most of the security issues. They have also rewritten the build and test infrastructure.

Several observations about open source:

1) Many people request changes 2) Few folks volunteer to help 3) User contributed patches usually result in lots of work concerning platforms and testing

Adobe have 10,000+ people including several thousand C++ Engineers. I know, I worked for Adobe in San Jose, California for more than 10 years. Team Exiv2 has been mostly a single volunteer. Surprise, Lightroom Supports CR3 before Roman, Laurent and Exiv2 can catch up.

If you have any good ideas about how to recruit engineers to join Team Exiv2, I'll be delighted to receive suggestions.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/darktable-org/rawspeed/issues/121#issuecomment-442012160

lclevy commented 5 years ago

Canon does have a sdk for developpers. But it is closed source and not public

⁣Envoyé par Blue

Le 27 nov. 2018 à 12:29, à 12:29, Robin Mills notifications@github.com a écrit:

I do have a practical suggestion to make about this. Canon should be contributing to open-source. When they decide to launch a new format on their users, I believe they have engineers in "Developer Relations" who will engage with Adobe, Google, Apple, Microsoft and others to ensure that PhotoShop and other major applications provide support for their format.

I have never received any communication from Canon or any other camera manufacturer. The MakerNote Support in Exiv2 has been reversed engineered. Laurent has done very good work to reverse-engineer CR3. Canon sold the camera and contributed nothing to the open-source community.

So, @simonszu Would you write to Canon and express your disappointment that they have not supported the open-source Community to adopt CR3.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/darktable-org/rawspeed/issues/121#issuecomment-442025365

simonszu commented 5 years ago

Sehr geehrte Damen und Herren,

ich möchte gerne meine vor einem Monat neu erworbene Canon M50 unter Linux in Betrieb nehmen. Hier habe ich vor allem das Problem, dass das neue CR3 RAW Format nicht mit den gängigen RAW-Bearbeitungsprogrammen (bspw. RAWtherapee oder darktable) kompatibel ist, was sehr schade ist, da ich die M50 momentan ausschließlich über die JPEGs nutzen kann. Meines Wissens nach arbeitet Canon bereits mit Adobe und anderen Studios zusammen, um RAW-kompatibilität bei kommerzieller Software zu gewährleisten. Leider wird hier die Linux- und OpenSource-Gemeinschaft völlig außer Acht gelassen. Ich würde mich freuen, wenn Canon hier ebenfalls mit den Entwicklern zusammen arbeiten würde, sodass auch Fotografen, die lieber quelloffene Software, bzw. Linux verwenden, das volle Potenzial aus ihrer Kamera schöpfen können. Ein mögliches Projekt wäre hierbei z.B. exiv2 (http://www.exiv2.org/), da dies als Grundlage von vielen anderen RAW-bearbeitungstools verwendet wird.

Bitte sorgen Sie dafür dass ich meine Kamera vollständig benutzen kann.

Vielen Dank und freundliche Grüße,

Dear Sir or Madam,

i'd like to use my Canon M50 which i have bought around a month ago with Linux. I currently have the problem that the new CR3 RAW format isn't compatible with the usual RAW editing tools (like RAWtherapee or darktable), which is sad, since i currently only can use the M50 via JPEG output. As far as i know, Canon is working with Adobe and other developers to ensure RAW compatibility in commercial software. Unfortunately, the Linux and OpenSource community is completely left behind. I would be very happy if Canon could also work with these developers, so that photographers who would rather use open source software or Linux could use the full potential of their cameras. A possible project would be exiv2, since that's the underlying library which is used by many other RAW editing tools.

Please make sure that i can use the full potential of my camera.

Thank you so much and best regards

I have sent the german text to Canon Germany, since i'm from Germany as well. Maybe other users who stumble upon this issue can re-use it, so i have provided an english translation as well.

lclevy commented 5 years ago

See references: https://github.com/lclevy/canon_cr3/blob/master/readme.md

⁣Envoyé par Blue

Le 27 nov. 2018 à 12:30, à 12:30, Simon Szustkowski notifications@github.com a écrit:

@clanmills That's an excellent idea. I'll do it right now.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/darktable-org/rawspeed/issues/121#issuecomment-442025662

LebedevRI commented 5 years ago

Ein mögliches Projekt wäre hierbei z.B. exiv2 (http://www.exiv2.org/), da dies als Grundlage von vielen anderen RAW-bearbeitungstools verwendet wird. ... A possible project would be exiv2, since that's the underlying library which is used by many other RAW editing tools

exiv2 only has the knowledge to decode the metadata. It does not, and should not, do any of the actual raw image (i.e. the actual pixels) decoding. Making exiv2 aware of cr3 will not be required for me to make darktable understand cr3.

Please make sure that i can use the full potential of my camera.

Uh huh. Regardless of whether this is directed at canon, or any of us, i really don't think it works that way :)

simonszu commented 5 years ago

It is directed at Canon. In the past i have experienced Canon as a very consumer friendly company which makes sure that their users are happy.

We will see. I have got an automated response which assured me that i'll get an update by a service guy in 1-2 days.

The usual problem with Linux support is in most cases that there is a quite small user base, so that it is too expensive to write official tools for it. But maybe once the company is aware that there is already a community which develops quite good applications themselves (and, most importantly, for free), and all they need is a little help, some companies are happy to jump in (just look at microsoft).

If their marketing guys realize that they can "unlock" an even bigger potential user base almost for free...well, i'm just thinking loudly. I'll keep you guys updated once canon has sent me an answer.

clanmills commented 5 years ago

I only mentioned Canon because we're discussing CR3. I worked for Canon for 3 years. I'm a printing software guy and that's what lead me to work on Adobe PostScript and later Acrobat. Never worked on metadata at Adobe.

It's hard to say how Canon will react. It's a business matter. Historically, Developer Relations would have focused on big commercial software vendors such as Adobe and Microsoft. The contracts and contacts are well defined. To work with the "open-source" community sounds vague and undefined. How do they know to contact me, Roman and so on? There are thousands of projects after all. They could focus on 5-10 major applications (GIMP, darktable, digiKam, and others). They should openly publish their format specifications and avoid NDAs.

Thanks for making the effort to bring this to their attention. Very helpful. It'll be interesting to hear their response.

LebedevRI commented 5 years ago

How do they know to contact ...

Who do they contact? Good question. How do they contact? Not an issue. Every git commit already has an email address.

hfiguiere commented 5 years ago

Just a little story about Canon and Linux support:

Over decade ago some of the developers of gphoto2 contacted Canon to try to get some documentation to improve Canon support in gphoto. Canon answer was that they didn't support Linux and that there was that open source project that work on Linux. That project name was... gphoto.

True story.

clanmills commented 5 years ago

There are very smart people at Canon. Clearly this reply didn't come from one of their best people!

And as you said, it was 10 years ago. The world is changing. Open-source is a "tour de force".

lclevy commented 5 years ago

I bet Dave Coffin will release in 2019 a dcraw version that supports cr3 wavelet decompression based on the Canon patent

⁣Envoyé par Blue

Le 27 nov. 2018 à 15:05, à 15:05, Robin Mills notifications@github.com a écrit:

There are very smart people at Canon. Clearly this reply didn't come from one of their best people!

And as you said, it was 10 years ago. The world is changing. Open-source is a "tour de force".

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/darktable-org/rawspeed/issues/121#issuecomment-442069652

rrenomeron commented 5 years ago

I know it's not ideal, but I was able to import some sample Canon M50 CR3 files into Darktable after converting them to DNG with an appropriate version of Adobe DNG Converter running under wine, and work with the DNGs. Depending on how demanding your workflow requirements are, it might be a viable workaround until this issue is figured out.

Disclaimer: I don't actually own an M50, I tried this as part of my camera-buying research earlier this year.

pszaban commented 5 years ago

I also asked Canon's EMail support to release this information. Will post here if anything of value comes back. By the way, the dcraw web page doesn't look like it has been updated in quite a while.

Hello, I'm a Canon EOS R camera owner, and a Linux O/S user. EOS R camera uses a new .CR3 file format that none of my Linux applications can decode. To be brief, I wanted to ask if it would be possible for Canon to release sample code, or documentation describing this new file format and, more importantly, information on how to extract and decompress raw image data from these files, so Linux programs like TheGIMP, Darktable, RAWTherapee and ufraw will be able to read CR3 files. Thank You!

LebedevRI commented 5 years ago

I also asked Canon's EMail support to release this information.

I really hope all these mails this won't cause an epic (legal) blow-back at me/us/project :)

In principle the patent linked at https://github.com/lclevy/canon_cr3#crx-compression has sufficient into. At least that is how it looks like, without starting to implement it.

Like i said previously elsewhere(?), i think it will be almost the same as the GoPro's VC5 decompressor, with the notable differences being:

different wavelet, 5/3 instead of 2/6 (but the same number of wavelet levels, 3)
not hardcoded RLV, but Golomb-Rice
looks like all the wavelet levels are Golomb-Rice (as opposed to the smallest wavelet being uncompressed low-pass)

pszaban commented 5 years ago

Well, that was quick. Canon responded with a polite no. The response stated that Canon provides a free Digital Photo Professional program and has decided to support only Microsoft and Apple operating systems. I'm going to let it go and move on; Canon heard a customer's request and I heard their response.

Thank you LebedevRI, I'll take a look at the link to the patent you supplied to see if I can understand how the compression works.  This isn't the kind of thing I typically code.  If nothing else, reading it will be a learning experience.

simonszu commented 5 years ago

I have got an answer from Canon as well. I think it's a polite no as well.

We are very sorry that you are not completely satisfied with your EOS M50.

We for ourselves are just offering our free software Digital Photo Professional for developing RAW photos of our cameras. Of course, Adobe and other software developers receive information from our headquarters in Tokyo which enable them to use our RAW photos.

We are pleased to forward your request regarding Linux. The ultimate decision which operating systems we support and with which providers we work with is taken by our mother company Canon Inc. Therfore we cannot tell you if a support in this case would be possible in the future.

We are sorry that we cannot give you other informations regarding this request and want to apologize for that.

Best regards...

So, i think, reverse engineering it is, then.

neo22s commented 5 years ago

Hello,

Thanks for the job, and joining the conversation, any news on this?

Bought a canon eos R and I can no use the RAW files. I work only on ubuntu :(

Any temporary fix?

LebedevRI commented 5 years ago

Hello,

Thanks for the job, and joining the conversation, any news on this?

Bought a canon eos R

Aha, finally someone with that camera! Please help, we need samples for https://raw.pixls.us/ :)

Details: Pick some static scene (outdoor landscape, no people), use tripod There are total of 2 2 4 samples needed (i think?), let me break it down:

There are two raw modes on that camera (i think?): RAW, C-RAW.
That camera has dual-pixel mode, which can be disabled and enabled
That camera can produce raws in 4 different aspect ratios We need the full coverage of all that, so
RAW, dual pixel off, 3:2
RAW, dual pixel off, 4:3
RAW, dual pixel off, 16:9
RAW, dual pixel off, 1:1
RAW, dual pixel on, 3:2
RAW, dual pixel on, 4:3
RAW, dual pixel on, 16:9
RAW, dual pixel on, 1:1
C-RAW, dual pixel off, 3:2
C-RAW, dual pixel off, 4:3
C-RAW, dual pixel off, 16:9
C-RAW, dual pixel off, 1:1
C-RAW, dual pixel on, 3:2
C-RAW, dual pixel on, 4:3
C-RAW, dual pixel on, 16:9
C-RAW, dual pixel on, 1:1

Please, do rename the files appropriately!!! Afterwards, upload each one of these files separately(!!!) to https://raw.pixls.us/

and I can no use the RAW files. I work only on ubuntu :(

Any temporary fix?

neo22s commented 5 years ago

@LebedevRI tomorrow you have it ;)

darktable-org / rawspeed

Canon CR3 support. #121