search

indigo-astronomy / indigo

INDIGO is a system of standards and frameworks for multiplatform and distributed astronomy software development designed to scale with your needs.
http://www.indigo-astronomy.org
Other
153 stars 70 forks source link

Debayering issues with Fujifilm files #567

Closed esox13 closed 3 weeks ago

esox13 commented 1 month ago

Hello,

There is something I can't manage to fix in AIN. I use a Fuji DSLR (X-H2) and if I create FITS files they are in greyscale. If I import them in SIRIL I have this message (french translated by chatGPT, so the original message in english may differ) :

The Bayer pattern found in the header (GRBG) is different from the Bayer pattern set in the preferences (RGGB). The latter has been replaced.

In fact I think Fuji Xtrans is a 6 pixels pattern, not 4. I purchased the macOS version, indigo A1, I think it has the same issue.

rumengb commented 1 month ago

I'm sorry, we don't actually support Xtrans color matrix. But do not worry it is only related to the visualisation. Debayering is done only for visualization purposes. Neither INDIGO nor Ain do any modifications to the original image. So it should be ok and it can be debayered by software that supports Xtrans. When it comes to the message from Siril - i do not know what is means, I never used Siril and I do not develop Siril.

Maybe saving the file in camera raw format will help try that.

jeanmartin13 commented 1 month ago

The thing is that Siril debayers correctly raf fuji raw files but not Ain generated fits files. You sure Ain do not change the pattern infos ? Because fits generated by Ain from fuji appear in b&w no matter the viewer. Siril, kstars...

esox13 commented 1 month ago

fits

This is a screen shot of the header of the fits file opend in the KStars viewer. There is a BRBG bayer pattern indicated.

fits_indigo_header And this one the fits header of a fits file from Indigo A1 unlocked version

esox13 commented 1 month ago

If the BAYERPAT paramter of the header is chenged to GGRGGBGGBGGRBRGRBGGGBGGRGGRGGBRBGBRG then the FITS files seem to be displayed and debayered correctly by SIRIL and when saved by SIRIL it can be displayed with colors in the indigo image viewer.

As I told you I'm just a user, but it seems FITS files generated by AIN/Indigo A1 include the wrong Bayer Pattern info.

rumengb commented 1 month ago

We will try to fix the pattern in the fits file for those exotic colour masks. If not, we will remove it.

When it comes to files saved by Siril they are no longer raw but manipulated and converted to 3 channel FITS which is a destructive transformation. INDIGO saves the image data as read from the camera, and as I already mentioned BAYERPAT is used by Ain and A1 to perform this transformation in memory for visualization purposes only and are never saved. Also some Fuji cameras use some specific color patterns that we do not support in indigo. This is why we display it as grayscale. On the other hand the files saved by Siril are 3 channel FITS files which are standard, therefore displayed correctly by INDIGO apps but at the cost of losing the raw data.

On Tue, Oct 1, 2024 at 2:03 PM esox13 @.***> wrote:

If the BAYERPAT paramter of the header is chenged to GGRGGBGGBGGRBRGRBGGGBGGRGGRGGBRBGBRG then the FITS files seem to be displayed and debayered correctly by SIRIL and when saved by SIRIL it can be displayed with colors in the indigo image viewer.

As I told you I'm just a user, but it seems FITS files generated by AIN/Indigo A1 include the wrong Bayer Pattern info.

— Reply to this email directly, view it on GitHub https://github.com/indigo-astronomy/indigo/issues/567, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5EZBJIIT32WMHATY7O3FLZZJ6OJAVCNFSM6AAAAABPEDDFBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBVGQ3TQMRQGQ . You are receiving this because you commented.Message ID: @.***>

rumengb commented 1 month ago

The issue is fixed. It will be part of the next indigo release.

esox13 commented 1 month ago

Thanks. Will it also fix the flipped upside down issues in FITS file from FUJI/IndigoA1/AIN issue ?

I didn't notice it becaue I tried on pictures that are difficult to say they are upside down, at least I didn't pay attention to that, but I tried with a non astro image where orientation can be easily noticed and all FITS images from my Fuji camera are flipped upside down, not 180° rotation but upside down flipped. The RAF raw are fine but the FITS aren't. In SIRIL, ASTAP and FITS liberator. But FITS previw app opens it in the right orientation, but of it can't debayer it. I guess there is a explanation to that. I don't have other software to test them on mac or linux. ASTAP also opens RAF files upside down...

The problem is that if I flip the images (with a numpy script) and apply the BAYERPAT I gave you above, it doesn't debayer correctly again, the xtrans pattern also has to be flipped upside down...

Maybe I shouldn't worry about this flipped problem, as I told you I'm not beginner as a photographer but a beginner in astrophoto.

I wrote a small astropy/numpy script that can correct the FITS so they are opened and processed correctly in SIRIL. It may also be a SIRIL issue, but SIRIL has no problem processing FUJI RAF files. This script has a flipped xtrans matrix and make the FITS fils (apprently) processed correctly in SIRIL.

I join you the script. fits_fuji_bayerpat.py.zip

polakovic commented 1 month ago

I tried FITS in both PixInsight and FITS Preview and images are not flipped compared to RAF. So it looks like SIRIL and ASTAP issue (as we consider PixInsight as most reliable "standard").

polakovic commented 1 month ago

Mystery solved, SIRIL doesn't use ROWORDER in FITS metadata.

rumengb commented 1 month ago

I did some experiments and it looks like Siril does not observe the ROWORDER keyword. and it assumes the images are always bottom-up. And we produce TOP-DOWN images and indicate this in the HEADER.

esox13 commented 1 month ago

Ok thanks for the answer. Understood, the funny thing is that the RAF are OK in SIRIL. I keep my script for SIRIL use, as it is the tool I'm using now.

Can't wait for the update ;)

Edit : I have an answer from some SIRIL guys about the fact SIRIL has no issues with RAF raw files : it uses LibRaw to process the raw files. Mystery also solved ;)

rumengb commented 1 month ago

It will take some time as we are preparing a big release. We are still developing. Then we need to freeze and test everything which will take several weeks.

On Thu, Oct 3, 2024 at 11:06 PM esox13 @.***> wrote:

Ok thanks for the answer. Understood, the funny thing is that the RAF are OK in SIRIL. I keep my script for SIRIL use, as it is the tool I'm using now.

Can't wait for the update ;)

— Reply to this email directly, view it on GitHub https://github.com/indigo-astronomy/indigo/issues/567#issuecomment-2392236685, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5EZBNQDDTOWAAHLCTJOA3ZZWPVTAVCNFSM6AAAAABPFPT6MKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJSGIZTMNRYGU . You are receiving this because you commented.Message ID: @.***>

esox13 commented 1 month ago

Hi !

You're planning en. rlease of Indigo server for windows ? ;)

Here is the answer of Vincent Hourdin one of SIRIL's developpers :

`that’s a funny thing to say because ROWORDER was created by the Siril team.

So that’s not RAW images if they are FITS, and if the acquisition software is not able to give the correct BAYERPAT, it’s not really our problem.

Yes Siril displays images bottom-up, like the FITS convention: Siril - FAQ 2 . When plate solving the result image in the usual processing workflow, if it’s flipped it will be put back in the correct orientation.`

https://siril.org/faq/#why-are-my-images-upside-down

rumengb commented 1 month ago

I'm sorry for my ignorance, I did not know that but I am glad for SIRIL developers :) Please bear in mind, that half of the software threats ROWORDER in one way and the other half in another. And it looks like we are in the wrong camp, together with PixInsight and many others. Life is tough and we have to live with that... Sorry, we are not going to change that.