search

ermig1979 / Simd

C++ image processing and machine learning library with using of SIMD: SSE, AVX, AVX-512, AMX for x86/x64, VMX(Altivec) and VSX(Power7) for PowerPC, NEON for ARM.
http://ermig1979.github.io/Simd
MIT License
2.03k stars 406 forks source link

Feature request: 16-bit support for Bayer demosaicing #177

Open s-trinh opened 2 years ago

s-trinh commented 2 years ago

I think it could be useful to have Bayer demosaicing support for 16-bit datatype. If other people agree, please add a +1 to this post.

Also, what is the demosaicing method? I believe it is bilinear demosaicing. Could the doc be updated with some details/reference about the algorithm used?

ermig1979 commented 2 years ago

Hi! Is the output 16-bit per chanel BGR or BGRA?

s-trinh commented 2 years ago

Hi.

For my use case, it would be more like 16-bit per channel RGBA but I believe for consistency in the lib supporting both BGR and BGRA mode would be great.

Also, the camera SDK allows retrieving 10-bit and 12-bit Bayer images.

ermig1979 commented 2 years ago

Could you give any reference to description of these (10-bit and 12-bit Bayer) image format?

ermig1979 commented 2 years ago

As to description of algorithm used in Simd unfortunately I can't remember original paper or reference code which I used to implement it. I remember that the algorithm is smarter than simple bilinear interpolation - it uses information from farer pixels for correction of corners coloring.

s-trinh commented 2 years ago

What I am getting is txt files with header description. Something like mode (10 bits per pixel or 12 per pixel), width/height (2048), etc. I don't have much more information. I don't have access to the camera / software anymore.

Something like uint16_t in input (e.g. 2048x2048) and uint16_t in output (2048x2048x4 or 2048x2048x3) should do the trick for my use case. Conversion from uint16_t to uint8_t for visualization + vertical flipping (I am getting data vertically flipped) if not already implemented would also be useful.

Too bad for the reference. Matlab uses this reference High-Quality Linear Interpolation for Demosaicing of Bayer-Patterned Color Images.

s-trinh commented 2 years ago

Looks like I am not the only one interested into Bayer conversion :smiley:

https://github.com/ermig1979/Simd/issues/135

ermig1979 commented 2 years ago

There are fast and slow ways to implement 16-bit Bayer to BGR conversion: 1) I use current algorithm for 8-bit and update it to 16-bit. 2) I try to implement algorithms from your paper - it may take much more time.

ermig1979 commented 2 years ago

There is third way: you implement Base (scalar) desired algorithm of 16-bit Bayer to RGB and then I will perform it optimization for different SIMD.

s-trinh commented 2 years ago

Thanks. I will see what I can do. There is no urgent need from my side.

s-trinh commented 2 years ago

@ermig1979 My attempt at Bayer demosaicing (still WIP): https://github.com/lagadic/visp/pull/936/files There are two implemented methods, bilinear and Malvar interpolation. I am interested into 16-bit (10/12 bits or even more) and 8-bit handling. Border handling is tricky. I tried to do something but this is certainly not conventional.

There is this paper that could be interesting: Efficient, High-Quality Bayer Demosaic Filtering on GPUs It eliminates branches for GPU architecture, but not sure if the mentioned optimizations are relevant for CPU since CPU should have advanced branch predictors able to handle that? From my limited knowledge, SIMD and minimizing cache misses are the keys. So tiling is the key if it is applicable?

Just found also this paper: Malvar-He-Cutler Linear Image Demosaicking

ermig1979 commented 2 years ago

Hello! I have seen your implementation of demosaic algorithms: https://github.com/s-trinh/visp/blob/feat_Bayer_demosaic/modules/core/src/image/vpImageConvert.cpp I can add these functions to Simd (with some modifications of course) and make their SIMD optimizations.

But this will be a great amount of work. So I have to be shure that it is exactly that you need. In other words: is the quality of these demosaic algorithms is enough to you?

s-trinh commented 2 years ago

To summarize for my use case, ideally:

ermig1979 commented 2 years ago

I found a small project which implements many debayering algorithms: https://github.com/jdthomas/bayer2rgb It may be useful in order to choose best algorithm.