search

sygslhy / image-io

Image io python APIs to wrap cxx_image C++ lib by using pybind11
MIT License
6 stars 2 forks source link

Can't convert files. #1

Closed Uzver123 closed 1 month ago

Uzver123 commented 1 month ago

I am on Windows 10, Python 3.12.4 installed cxx-image-io 0.0.7 and numpy 1.26.0 trough PIP.

This are the files i am working with https://www.upload.ee/files/17182598/raw_and_yuv.zip.html

This is my code

from cxx_image_io import read_image
from cxx_image_io import ImageMetadata, ImageWriter, FileFormat, PixelType, ImageLayout
from cxx_image_io import write_image

import numpy as np

image, metadata = read_image('D:\\raw_and_yuv\\raw.RAWPLAIN16')
assert isinstance(image, np.ndarray)

print('Type:', image.dtype)
print('Shape:', image.shape)

print('Type:', metadata.fileInfo.pixelType)
print('Precision:', metadata.fileInfo.pixelPrecision)
print('Layout:', metadata.fileInfo.imageLayout)

write_options = ImageWriter.Options(metadata)

assert isinstance(image, np.ndarray)
write_image('D:\\raw_and_yuv\\image.tif', image, write_options)

This produces TIF that can't be opened in any image viewer, if i switch extension to DNG its also unreadable.

when i try PNG write_image('D:\\raw_and_yuv\\image.jpg', image, write_options) getting following error:

Traceback (most recent call last):
  File "D:\Settings and Profiles\PyCharm Projects\RAW Dump Image Reader\index.py", line 35, in <module>
    write_image('D:\\raw_and_yuv\\image.jpg', image, write_options)
  File "C:\Users\User\AppData\Roaming\Python\Python312\site-packages\cxx_image_io\io.py", line 113, in write_image
    image_writer.write(image)
TypeError: write(): incompatible function arguments. The following argument types are supported:
    1. (self: cxx_image.io.JpegWriter, arg0: cxx_image.ImageUint8) -> None

Invoked with: <cxx_image.io.JpegWriter object at 0x0000020A468A7630>, <cxx_image.ImageUint16 object at 0x0000020A46AF76F0>

when JPG write_image('D:\\raw_and_yuv\\image.png', image, write_options) getting following error

Traceback (most recent call last):
  File "D:\Settings and Profiles\PyCharm Projects\RAW Dump Image Reader\index.py", line 35, in <module>
    write_image('D:\\raw_and_yuv\\image.png', image, write_options)
  File "C:\Users\User\AppData\Roaming\Python\Python312\site-packages\cxx_image_io\io.py", line 113, in write_image
    image_writer.write(image)
TypeError: write(): incompatible function arguments. The following argument types are supported:
    1. (self: cxx_image.io.PngWriter, arg0: cxx_image.ImageUint8) -> None
    2. (self: cxx_image.io.PlainWriter, arg0: cxx_image.ImageUint16) -> None

Invoked with: <cxx_image.io.PngWriter object at 0x000001D1A59E5C70>, <cxx_image.ImageUint16 object at 0x000001D1A59E60F0>

when instead of raw.RAWPLAIN16 i load yuv.YUV420NV12

from cxx_image_io import read_image
from cxx_image_io import ImageMetadata, ImageWriter, FileFormat, PixelType, ImageLayout
from cxx_image_io import write_image

import numpy as np

image, metadata = read_image('D:\\raw_and_yuv\\yuv.YUV420NV12')

getting memory access violation error: Process finished with exit code -1073741819 (0xC0000005)

P.S. i also tried your tests none of them completed successfully.

sygslhy commented 1 month ago

Hello, Thanks for using this package. I tried your test file and code with python 3.12.4, I hope the following can help you:

1. Tif cannot be viewed

I see in your code, it is saving a Bayer pattern image to tif file, Bayer pattern raw 16bit is caught from image sensor, not yet processed by ISP(image singal processor), neither converted to RGB image. The common image viewers usually support well to display RGB image, but not support display Bayer image, I opened successfully your result image.tif by imageJ: https://imagej.net/ij/, you can just drag the tif file to the imageJ window then the image will be display like this:

tif-preview-imagej

2. DNG no readable

This is an issue I am work on it, because in current C++ code, the dng writing API doesn't write yet the preview image, it writes only the main image. but to read the dng file correctly by common image viewer, we need preview image. I will try to fix this DNG preview issue, and before issue fixed I will disable this DNG writing API.

3.PNG,

Thanks, because of your question, I find a typo in my code, that's why png 16 bit writing is not usable. this will be fixed in v0.08 version. However png 16bit file dosesn't supports the bayer format, you can see the I/O table in README.md: io-table

I suggest you to write png 16bit file in grayscale format for image viewing, you need to add only one line in your code: metadata.fileInfo.pixelType = PixelType.GRAYSCALE Then you can use imageJ to see it, because unlike png 8bit RGB, png 16bit grayscale file isn't widely supported by common image viewer.

png-preview-imagej

4.JPG,

As shown in the above io-table, in this project, JPG support only 8 bit image writing, doesn't support 16 bit image writing. your image read from RAW is 16bit. that's why JPG reader throws the compatibility exception. In the version v0.08, in python level I made some "catch" to make these C++ exceptions look like friendly. so the error message will be less spam.

5.YUV or NV12 file support

Unfortunately, it is an under-developing feature (not yet done) in python package where I am still looking for the technique solutions. because in pybind11 numpy buffer protocol only supports buffer mapping on the image planars where has the same size, Since YUV and NV12 always has resampled planar, the planars have different size from each others. It was me I didn't successfully disable this unfinished feature in python binding code, I disabled YUV, NV12 in python package in version v0.08.

pytest case failed

On my machines linux and windows in different python version, the pytest always have all passed for test result, can you give me some detail trace from your side for analyze please? pytest-passed

I hope all the above can help you for your development, the v0.08 is already published on PYPI. please feel free to contact me if you have any problem or questions.

Best regards.

Uzver123 commented 1 month ago

Thank you for fast response., methods you suggestions work.

My final goal is:

  1. Export RAW image file format that can be opened in ART/RawTherapee or Darktable to debayer and work on same way as when i open camera RAW file.

if currently not possible

  1. Debayer in Python and export TIF/PNG/JPG with color.

From what i understand CXX Image Library does not convert anything it simply change container file format, in order to export normal image would need to use another python library in middle to import->debayer->export

Do you happen to have any code examples how to do so?

P.S. CXX Image Library repo got updated two weeks ago, did you happen to include the new version in cxx-image-io 0.0.8?

sygslhy commented 1 month ago

Hello, Yes CXX image io in python or in C++ just provide the import or export API, the Bayer conversion is not included.

If I understand well your question, you are looking for :

  1. a software which can export raw image buffer to a compatible format which can be processed by the dartable Bayer conversion pipeline, then output RGB color image.
  2. Or you want a python Bayer conversion pipeline which can take directly a raw image buffer as input then output RGB color image.

I am not sure you can find directly the python code which do the generic bayer conversion to RGB, even it exists some code in github, the image quality can not be ensured as good as dartable or lightroom does. because raw bayer conversion is the part of image postpossing software work, it includes lots of complex algorithms where every company has their IP on them.

So I think it might be better to still try to find a compatible file with Dartable, at least dartable's processing quality is ensured, it could exist some dng or Tif bayer writing code on the internet, but I don't have directly the example.

No I didn't include the new version in v0.0.8.

have a good day.

sygslhy commented 1 week ago

@Uzver123 Hello, I hope you are well, wish your project goes well too, I'd just like you know, I have a new version of cxx-image-io=0.1.0 which support DNG writer, it doesn't support preview image in Dng yet, still working on it, but I tested it can be openand read by RawTherapee now.

So maybe you can use the version cxx-image-io=0.1.0 and export you raw file to dng, then process with RawTherapee to an RGB jpg image.

I also found a workround to generate preview image in dng too, it is dng_validate in https://github.com/emmcb/adobe-dng-sdk. you can build this project to generate dng_validate binary then it can help to generate preview-image as this command like this ./dng-validate -dng output_with_preview.dng output.dng

PS:

By the way, I forgot to tell, to add the preview image, it is necessary to have these fileds whiteBalance, blackLevel, whiteLevel, and colorMatrix in your .json file. because preview image is RGB, they need these parameters to convert Bayer to RGB. In these fields: whiteBalance is linked to every shot for the scene. the other 3 are linked the camera senor model.

"cameraControls": { "whiteBalance": [ 2.2502453327178955, 1.493620753288269 ] }, "calibrationData": { "blackLevel": 258, "whiteLevel": 4095, "colorMatrix": [ [ 1.7485008239746094, -0.9061940312385559, 0.15769319236278534 ], [ 0.017115920782089233, 1.318513035774231, -0.3356289267539978 ], [ 0.03677308186888695, -0.3459629416465759, 1.309189796447754 ] ] }

Uzver123 commented 1 week ago

Great news, i will test at first opportunity.