LJpegDecoder doesn't support Hasselblad style files

[pedrocr]

I've been trying to get the .3FR format supported by rawspeed. Here's my current effort so far:

https://github.com/pedrocr/darktable/compare/rawspeed-mamiya-support...rawspeed-hasselblad-support?expand=1

I've created a new decoder and hooked into TiffParser. I've also fixed ByteStream::skipToMarker() to correctly identify the starting marker of the DHT section in the hasselblad files (ignoring 0xffff markers which apparently are FILL sections that we don't care about, since the DHT section was being confused with a FILL section). I've also accepted predictor number 8 as apparently that's what is in the hasselblad files.

So right now I just need to actually do the decoding. I guess I need to implement a decodeScanSomething() function. The relevant dcraw code is this one:

void CLASS hasselblad_load_raw()
{
  struct jhead jh;
  int row, col, pred[2], len[2], diff, c;

  fprintf(stderr, "In hasselblad_load_raw with c=%d\n", c);

  if (!ljpeg_start (&jh, 0)) return;
  order = 0x4949;
  ph1_bits(-1);
  for (row=0; row < raw_height; row++) {
    pred[0] = pred[1] = 0x8000;
    for (col=0; col < raw_width; col+=2) {
      FORC(2) len[c] = ph1_huff(jh.huff[0]);
      FORC(2) {
        diff = ph1_bits(len[c]);
        if ((diff & (1 << (len[c]-1))) == 0)
          diff -= (1 << len[c]) - 1;
        if (diff == 65535) diff = -32768;
        RAW(row,col+c) = pred[c] += diff;
      }
    }
  }
  ljpeg_end (&jh);
  maximum = 0xffff;
}

It seems pretty straightforward. But before I try and reimplement this blindly do you know how this relates to the code already there? Is it a special case? Any tips on how to go about it?

[pedrocr]

Humm, it seems the DHT and SOF sections aren't being correctly read either. As usual the dcraw code is terse but actually not that bad:

  do {
    fread (data, 2, 2, ifp);
    tag =  data[0] << 8 | data[1];
    len = (data[2] << 8 | data[3]) - 2;
    if (tag <= 0xff00) return 0;
    fread (data, 1, len, ifp);
    fprintf(stderr, "Found tag %x\n", tag);
    switch (tag) {
      case 0xffc3:
        jh->sraw = ((data[7] >> 4) * (data[7] & 15) - 1) & 3;
      case 0xffc0:
        jh->bits = data[0];
        jh->high = data[1] << 8 | data[2];
        jh->wide = data[3] << 8 | data[4];
        jh->clrs = data[5] + jh->sraw;
        if (len == 9 && !dng_version) getc(ifp);
        break;
      case 0xffc4:
        if (info_only) break;
        for (dp = data; dp < data+len && (c = *dp++) < 4; )
          jh->free[c] = jh->huff[c] = make_decoder_ref (&dp);
        break;
      case 0xffda:
        jh->psv = data[1+data[0]*2];
        jh->bits -= data[3+data[0]*2] & 15;
        break;
      case 0xffdd:
        jh->restart = data[0] << 8 | data[1];
    }
  } while (tag != 0xffda);

Right now rawspeed only cares about SOF3 (0xffc3) and seems to parse it roughly the same way that dcraw parses SOF0 (0xffc0). Although the rawspeed LJpegDecompressor seems to have been made for Canon, the dcraw ljpeg_start parsing seems to be used for everything, so hopefully this can be made to work.

[klauspost]

Do you happen to have a link to some sample files, I can give it a shot once I get a little time?

[pedrocr]

I'm going through the rawsamples.ch files. The 3FR files are here:

http://rawsamples.ch/html/en/hasselblad.html

I've now done another two Kodak formats (DCR and KDC) and the Epson R-D1 format (EFR). I'll submit those as a new pull request.

After those to cover all of rawsamples.ch we're missing these Hasselblad 3FR files, the old Canon CRW format, another Kodak format from some compacts, and the Nikon D1X and D100. I'll probably have a look at the Nikon and then the Canon next.

Getting all of rawsamples.ch would probably mean most of the old oddball formats are supported. And hopefully the modern formats are stable enough to not cause too many headaches.

[klauspost]

OK. I have pushed a working decoder to develop.

It seems like Phase One had their own idea of how an LJPEG should be ;)

I have added a "pixelBaseOffset" hint, since dcraw seem to have some "load_flag" magic which could indicate that the base offset per line could be different. However in both test images it is '0', so I don't know when it occurs.

I have not added camera definitions, since I currently don't have a way of confirming the CFA colors.

[pedrocr]

Cool, thanks for looking into it. What do you mean by confirming the CFA colors? At least for the CFV and H3D the rawsamples.ch files should be enough to verify the working CFA, no? I'll try them now and see how it works.

[klauspost]

I don't have a working Linux setup at the moment, and on my Windows machine I can only see a greyscale CFA image, so verifying that the CFA colors are correct is a bit tricky :)

[pedrocr]

Here's a pull for this:

https://github.com/klauspost/rawspeed/pull/30

[klauspost]

Cool. I will close this.