Pentax K1ii DNG images corrupted after capture

[jesshurch]

Describe the bug Using Ekos (kstars) to capture an image on a remote indi server results in a valid .DNG file being copied from the camera, but when copied back it arrives both corrupted, and with added FITS headers. Removing the FITS header does not leave a valid DNG

To Reproduce

Select 'Native' in the ekos module, and 'DNG' as the format in settings capture an image either locally or remotely (neither seem to make a difference) Image will not open as a valid DNG in Rawtherapee, or a valid FITS in Siril

Expected behavior A valid DNG image should be copied to the local folder, or store remotely on the indiserver machine Note: this did work with whichever version was current around the end of 2021

Screenshots settings in ekos hex header of copied image hex header of image saved by indiserver for debug

Desktop (please complete the following information):

• OS: OpenSuse 15.4
• kstars-3.6.1-lp154.35.2.x86_64
• Remote indiserver
• OpenSuse tumbleweed
• indi-pentax-1.9.8-1.1.x86_64
• libindi-1.9.8-1.3.x86_64

Log Files

INFO 129.770058 sec : Session log file /root/.indi/logs/2022-10-13/indi_pentax/indi_pentax_10:50:18.log INFO 157.802243 sec : Taking a 2 seconds frame... INFO 163.820461 sec : Capture finished. Waiting for image download... DEBUG 166.303567 sec : new bulb

DEBUG 166.303772 sec : Shutter pressed. DEBUG 174.830551 sec : Reading temp file from: /tmp/indipentax.tmp-0001.dng DEBUG 180.855593 sec : read_libraw: raw_width: 7392 top_margin 18 left_margin 6 first_visible_pixel 133062 DEBUG 180.855808 sec : read_libraw: rawdata.sizes.width: 7376 rawdata.sizes.height 4932 memsize 72756864 bayer_pattern RGGB DEBUG 180.989681 sec : read_libraw: memsize (72756864) naxis (2) w (7376) h (4932) bpp (16) bayer pattern (RGGB) WARNING 180.989885 sec : Camera image size (7376x4932) is different than requested size (7360,4912). Purging configuration and updating frame size to match camera size. DEBUG 181.255823 sec : Uploading file. Ext: fits, Size: 72763200, sendImage? No, saveImage? Yes INFO 181.496080 sec : Image saved to /telescope/Light/Light_004.fits DEBUG 181.496695 sec : Upload complete INFO 358.815886 sec : Debug is disabled. INFO 358.894540 sec : Debug is enabled. DEBUG 358.970657 sec : Toggle Debug Level -- Driver Debug INFO 364.959450 sec : Debug is disabled. INFO 365.038430 sec : Debug is enabled. DEBUG 365.110698 sec : Toggle Debug Level -- Driver Debug INFO 389.859743 sec : Taking a 2 seconds frame... INFO 396.778351 sec : Capture finished. Waiting for image download... DEBUG 398.381798 sec : new bulb

DEBUG 7785.512590 sec : Shutter pressed. DEBUG 7794.430237 sec : Reading temp file from: /tmp/indipentax.tmp-0001.dng DEBUG 7794.455982 sec : Copied to frame buffer. Leaving temp file for debug purposes. DEBUG 7794.721600 sec : Uploading file. Ext: raw, Size: 72763200, sendImage? Yes, saveImage? No DEBUG 7794.727927 sec : BLOB transfer took 0.00616128 seconds DEBUG 7794.728030 sec : Upload complete INFO 13362.046152 sec : Taking a 2 seconds frame... INFO 13368.313555 sec : Capture finished. Waiting for image download... DEBUG 13370.562990 sec : new bulb

DEBUG 13370.563162 sec : Shutter pressed. DEBUG 13379.324118 sec : Reading temp file from: /tmp/indipentax.tmp-0001.dng DEBUG 13379.350019 sec : Copied to frame buffer. Leaving temp file for debug purposes. DEBUG 13379.613811 sec : Uploading file. Ext: raw, Size: 72763200, sendImage? Yes, saveImage? No DEBUG 13379.620325 sec : BLOB transfer took 0.00633663 seconds DEBUG 13379.620426 sec : Upload complete INFO 15984.476086 sec : Upload settings set to local only. INFO 15984.486863 sec : Taking a 2 seconds frame... INFO 15990.886597 sec : Capture finished. Waiting for image download... DEBUG 15992.984083 sec : new bulb

DEBUG 15992.984570 sec : Shutter pressed. DEBUG 16001.896462 sec : Reading temp file from: /tmp/indipentax.tmp-0001.dng DEBUG 16001.934099 sec : Copied to frame buffer. Leaving temp file for debug purposes.

[jesshurch]

Note: Images passed to the focuser and plate solving appear to be valid

[knro]

Is this using INDI Pentax (Native) driver? I just tested with Nikon and couldn't find any issues. Also, when using Focus/Align modules, the captured images are FITS. However, the images are only native when capturing via the Capture module.

[jesshurch]

Correct, this is the INDI Pentax (Native) driver. My apologies for not being able to recall which version I was using at the tail end of last season when everything was functioning well. It does look like the capture is fine, just the images is being corrupted and a FITS header added prior to transfer. Is this handled by the Pentax driver ?

[knro]

Thanks, @karlrees is the expert on this driver. @karlrees Can you please check this out?

[knro]

I see Image Format is missing. @karlrees I added in pentax_ccd.cpp line 101 a loop to add capture formats... do you know why are they still missing?

[karlrees]

Off-hand, I recall that there was some hack I implemented originally because the DNG files weren't being processed properly. I believe I was saving the DNG files with a .raw extension to workaround the problem. Perhaps the changes have broken an assumption I was making later on about the image formats. I am probably at least a week out before I can commit any serious time investigating though.

[jesshurch]

If it's any help I just built and tested 1.9.4 and .raw files are written correctly with no FITS header information

[jesshurch]

I've now built and tested version 1.9.5 and it fails in the same way as the current release PkTriggerCordCCD::grabImage handles the data as native, but CCD::ExposureCompletePrivate treats it as FITS

not knowing the code too well nor having much time I haven't got further than this, but if it helps pin down anywhere where it's likely to be broken ...

[jesshurch]

It looks like pktriggercord_ccd.cpp is using transferFormatSP to determine whether a FITS format is needed whereas it needs to use EncodeFormatSP[FORMAT_FITS] as per pentax_ccd.cpp and it seems to work

[jesshurch]

Fairly confident I've fixed it now