Chroma Keying

[tvanfossen]

Hello,

Running on an ESP32S3 with LVGL near master, with color depth set to 16_SWAP with an ST7789 driving a 240x320 LCD.

I am attempting to take the below PNG image, with a transparent background and running through lv_img_conv.js

Passing CF_TRUE_COLOR_CHROMA as the color format to converter

My chroma key as set in kconfig is 0xFF00FF

The converter translates the transparent background to black, as I would expect which in turn is displayed on a given view.

In my case, I want to use the chroma key as an effective replacement of transparency from the original PNG.

This image has been altered from the original in GIMP. A black layer is added with burn blending, and a pink layer is added. This is coming at the cost of dithering on the PNG.

If I instead provide this image to the converter, where the background has been translated to 0xFF00FF, the image ALMOST converts as I'd expect. With or without dithering enabled, I am seeing pink pixels make it through to the LCD with this method. If I dither, the pink pixels form a complete edge around the image. If I dont dither, a few scattered pixels make it in.

---

Suggested Fix:

When I am calling lv_img_conv.js, I'd like to be able to pass the chroma color expected by LVGL when using CF_TRUE_COLOR_CHROMA (or any other chroma format), such that the conversion translates the original PNG with transparency, with or without dithering to a chroma keyed c array.

The converter then should be able to translate fully opaque pixels to the desired chroma color, while preserving any dithering that may have already been performed in the PNG.

Dithering should also accomodate chroma color, and not dither against a color that is not intended to be rendered in LVGL. This will actively form a pink edge around the content of the image, while removing the background.

I am not familiar with script, but am happy to help on changes if someone more familiar can help point me in a direction to start.

[embeddedt]

@tvanfossen Thanks for opening this. I agree that the dithering logic could be improved in this scenario so I'll see what I can do. I also like the idea of having the converter handle chroma-keying the image for you.

I cannot reproduce the first part of your issue, where pink pixels are still visible on the LCD even if dithering is not enabled. Here is the command I am running:

./lv_img_conv.js img.png -f -c CF_TRUE_COLOR_CHROMA -o ~/work/lv_newproj/lv_sim_eclipse_sdl/test.c -i test_img

I am then using the resulting image in the PC simulator by creating an image object and setting its source - no other changes.

When I tried putting the simulator in 16bpp mode, I can see that the edges of the hand might be very slightly pink, but only if I zoom in on the individual pixels in GIMP. On the simulator itself I can't tell.

[tvanfossen]

Original image used in both examples below to generate c array is slightly different than chroma colored image above. It leaves the dithering from original transparent image from above in place instead of making it all white.

Using the above as part of an invoke call to generate images from a provided folder produces the below image on LCD, hard to see but see a non-uniform pink edge around the image (so sounds like a 16bpp issue here, or indirectly an issue with how I was trying to leave dithering from png in place)

Using the above with dithering on, seeing a uniform pink edge around the image. Nominally would prefer to leave dithering on

Obviously a bit hard to see given the resolution of the screen against camera artifacts, but in all cases, LVGL is doing what its supposed to in not drawing the chroma color (0xFF00FF)

[embeddedt]

@tvanfossen Sorry for the delay in getting back to you. I'm still unable to reproduce this so I must be doing something wrong. I tried your second image with this command:

./lv_img_conv.js -d true -f -c CF_TRUE_COLOR_CHROMA -o hand.c hand.png

However, it looks perfect on my screen:

I don't doubt that there's an issue here, and your explanation makes sense. However, I need to figure out how to reproduce it to fix it. Did I miss something?

[tvanfossen]

What color depth are you displaying on-screen? Could it be an issue only present in 565 displays?

[embeddedt]

I am using LV_COLOR_DEPTH = 16.

On Sun, May 8, 2022 at 10:20 AM tvanfossen @.***> wrote:

What color depth are you displaying on-screen? Could it be an issue only present in 565 displays?

— Reply to this email directly, view it on GitHub https://github.com/lvgl/lv_img_conv/issues/17#issuecomment-1120427564, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKHTVAFM3LGJSHKZJHHDCTDVI7ESBANCNFSM5TZHZQGA . You are receiving this because you commented.Message ID: @.***>

[tvanfossen]

hand.zip hand_dither.zip

These are the c array files being converted (with and without dithering)

I can confirm that I do not see this issue on the TFT simulator, but DO see this issue on the LCD

Tried both dithered and undithered c arrays on simulator and see no pink outline in either case. Simulator set to use color depth 16 with swap

On the LCD, the same c arrays both produce a pink outline

[tvanfossen]

Overlaid the hand images onto a parent lv_obj_t grey circle as in the examples on LCD screen provided above for sanity, seeing the same case of no pink outline showing in simulator.

[embeddedt]

@kisvegabor Any ideas why SDL and an actual LCD would display an image differently, if they are both using the same color depth?

[kisvegabor]

It seems there is a problem with the image:

So the image has alpha channel too. When I removed the Alpha channel in GIMP, the image looks like:

Okay, the extra pixels are not not pink but blue, but I believe it causes the problem.

[tvanfossen]

interesting, odd that it converts to blue? For the SDL sim to not show an outline, but on an LCD to show, with the same bitmap, seems to imply a problem in the display driver as well?

[kisvegabor]

Does the other colors look correct on your device? E.g. if you set the bg color of rectangle to lv_color_hex(0xff0000) will it be red?

[tvanfossen]

Yes, all other colors appear as intended

[kisvegabor]

I have no idea why they turn to purple . :slightly_frowning_face: Probably we could track it, but I think there is no defined behavior (at least I not defined it when implemented the image renderers :smile: ) when chroma keying and alpha are used together.

Does chorma keying works well if you remove the alpha channel?

[tvanfossen]

Removing every bit of transparency from PNG will give a properly colored image, but then results in jagged edges due to lack of dithering

Have tried a few different ways in gimp to keep the dithering in place while replacing alpha with chroma, with no luck

[kisvegabor]

For an image like this I suggest using e.g. INDEXED_4BIT. With this you have 16colors in a palette and any of these colors can have any alpha value.

[tvanfossen]

I'll give indexed a shot - was trying chroma keying to be able to layer images/widgets on top of each other nicely

[kisvegabor]

indexed should do an even better job as you can have a semi-transparent colors too. E.g. opa=25%, 50%, 75%, etc.

[tvanfossen]

Would there be any issues with that on an RGB565 screen without an alpha channel?

[embeddedt]

I don't think the screen needs to have an alpha channel; the alpha values are consumed by LVGL in the rendering process (e.g. for rendering widgets behind the image).

[tvanfossen]

This 100% resolves the issue with dithering in chroma keying, but introduces a separate issue

When producing bitmaps from PNG assets (the hand png from above with alpha channel) with CF_INDEXED_8_BIT, image looks perfect on screen, exactly as desired.

However, rotation and zoom animations no longer perform as expected on these images. I might just be missing something simple as I havent used the indexed color in LVGL as of yet.

....

Which after a moment of checking the docs

So while an indexed image resolves the issue with chroma keying, it breaks animations that would otherwise be applied normally to a chroma keyed image

[tvanfossen]

For reference, the same screen/animations with the chroma keyed image asset

[kisvegabor]

If transformations are also required you can do 4 things:

• Use chroma keying without having alpha channel (result in jagged edges)
• If you always place the images on this dark background you don't need alpha channel. Just use TRUE_COLOR format and the dark background on the image too
• Use TRUE_COLOR_ALPHA images (3 bytes/pixel)
• Use the new transformation feature of LVGL. It renders the widget into a buffer first and rotates that buffer. The only difference is that you need to use style_transform_angle and lv_img_set_angle(). It will temporarily allocate a buffer equal to the size of the transformed widget. To great thing in it is that the children are transformed as well.

[tvanfossen]

Will give the transform a try!