lvgl / lvgl

Embedded graphics library to create beautiful UIs for any MCU, MPU and display type.
https://lvgl.io
MIT License
17.53k stars 3.3k forks source link

v8.2 discussion #2790

Closed kisvegabor closed 2 years ago

kisvegabor commented 3 years ago

v8.1 is released so I open this issue to plan v8.2. In order really have a "Planning phase" for the next release I'm about to leave this issue open until 31 December and focus on implementation after that.

I add my current ideas here (see explanation in the next comment), please comment if you also have some features ideas in mind or share your opinion about the planned features.

Rendering

Features

Others

kisvegabor commented 3 years ago

The initial list and the features ideas are not written into stone, please tell what you think about them.

Fix/Release scripts

IMO it has the highest priority. I meant these:

Widgets to image buffer

Make possible to buffer widgets with all children into a single image to

Scenes

It's a brand new concept about how a UI can be managed. It's a very simple but very powerful thing. Each widget has an int scene_active field and there is an lv_obj_set_scene(obj, secene_id) function which sends LV_EVENT_SCENE_UNLOAD, saves the scene_id and sends LV_EVENT_SCENE_LOAD to the widget. In other words, the scenes can be considered as "states" of the widget (I do NOT mean pressed, focused, etc style states here).

Use cases:

Add/Subtract/exclusive mask types

Now the masks of LVGL can only subtract from an 0xff filled mask. It'd be great to allow "building back" parts too.

Style components

Right now it's a little bit cumbersome to manage differently looking versions of the same widget type. E.g. button, error button, warning button, border only button, border only error button, rounded button, rounded error button, etc. To manage it effectively there should be btn, btn_border_only, btn_rounded, btn_error, btn_warning styles. The combination of these styles needs to be added manually to the created buttons. It's possible to create btn_error_create() and similar functions but it's also cumbersome. If the "role" of the button changes meanwhile, it's also hard to manage now (i.e. remove styles and add new ones). The idea to solve it looks like this:

uint32_t ERROR_BTN_ROUNDED = lv_component_create_from(LV_COMPONENT_BTN);
lv_component_add_style(ERROR_BTN_ROUNDED, &error_btn_style, LV_PART_MAIN);
lv_component_add_style(ERROR_BTN_ROUNDED, &rounded_btn_style, LV_PART_MAIN);

lv_obj_t * btn = lv_btn_create(parent);
lv_obj_set_component_type(btn, ERROR_BTN_ROUNDED);

lv_obj_set_component_type can be called any time to change the type and ERROR_BTN_ROUNDED can be updated anytime too, and all objects will be updated.

The implementation is quite simple: If functions like lv_obj_style_get_bg_color don't find a style on the object, they check the styles assigned to the component ID too. It will save memory and give more performance because the styles with the same selector can be merged in ERROR_BTN_ROUNDED and only the component ID needs to be saved in the object, instead of many styles.

Overflow:visible

Same as in HTML

Hover

Same as in HTML

More grid features

I'm specifically thinking about repeat(auto-fill, minmax( <value> px, 1fr)) but ideas are welcome

SVG support

See https://github.com/Samsung/thorvg/issues/1014

lishaoenxm commented 3 years ago

@kisvegabor I've created a new issue #2800. Could it be added in this version?

guoweilkd commented 3 years ago

Modify qrCode lv_qrcode_delete() interface, replace it with lv_obj_del().In other words, rewrite Qrcode using obj template.

kisvegabor commented 3 years ago

@lishaoenxm I'll comment in the opened issue.

@guoweilkd It's quite straightforward, so I've just updated the qr code widget. See 318edd8a3

embeddedt commented 3 years ago

Sorry for the delay in replying here. Here are my thoughts:

a CI action to merge commits with fix message to staging/v8. branches

I think it might be safer to check for the word "backport" in the commit description, otherwise the CI will end up cherry-picking fixes for a v8.2 feature into the v8.0 and v8.1 branches, which is not what we want.

add CI to port to automatically update submodules and conf files and minor and patch releases

Are we moving lv_drivers into lvgl for v8.2? That would allow us to have only a single submodule per port, which is ideal, otherwise we have to keep the two repositories working with each other.

Style components

It seems to me that all that is needed here is a way of linking styles to a parent style. This is what I'm envisioning:

static lv_style_t btn, btn_rounded, btn_warning, btn_error;

static lv_style_t error_btn_component;

lv_style_init(&error_btn_component);
lv_style_add_child(&error_btn_component, &btn_rounded);
lv_style_add_child(&error_btn_component, &btn_error);

Now error_btn_component can be used just like any other style, but has no properties of its own.

kisvegabor commented 3 years ago

No problem :slightly_smiling_face:

I think it might be safer to check for the word "backport" in the commit description, otherwise the CI will end up cherry-picking fixes for a v8.2 feature into the v8.0 and v8.1 branches, which is not what we want.

True, although it can happen that the part being fixed already changed in v8.2. But, I agree "backport" is safer and probably will result in fewer issues.

Are we moving lv_drivers into lvgl for v8.2?

I think we should.

It seems to me that all that is needed here is a way of linking styles to a parent style. This is what I'm envisioning:

As styles don't store part and state e.g. a complete slider can't be described this way where there are main, indicator, and knob parts with any combination of states.

C47D commented 3 years ago

18/24 bit color format support

How hard could it be to add this feature? I'm interested in helping out with this one.

Should we also consider gray scale support?

New drivers architecture New driver architecture #2720

I'm also working on this, we had a couple of really busy weeks at work, I'm continuing with the new drivers architecture next weekend.

kisvegabor commented 3 years ago

How hard could it be to add this feature? I'm interested in helping out with this one.

I suggest waiting a little with it. We started to work on abstracting the drawing enginesand therefore the all drawing parts will be hugely refactored and rewritten. So it should be added once the architecture is stabilized to avoid complicated merging.

I'm also working on this, we had a couple of really busy weeks at work, I'm continuing with the new drivers architecture next weekend.

Awesome! It might be also related to the abstraction of drawing. See https://github.com/lvgl/lvgl/pull/2803#issuecomment-971426684

C47D commented 3 years ago

I suggest waiting a little with it. We started to work on abstracting the drawing engines and therefore the all drawing parts will be hugely refactored and rewritten. So it should be added once the architecture is stabilized to avoid complicated merging.

Sounds fair, I will focus on the drivers.

puzrin commented 3 years ago

Scenes

It's a brand new concept about how a UI can be managed. It's a very simple but very powerful thing.

IMO this looks like mixture of CSS cascading & transforms animation. I would implement more classic features instead of invent new ones.

Style components

Right now it's a little bit cumbersome to manage differently looking versions of the same widget type. E.g. button, error button, warning button, border only button, border only error button, rounded button, rounded error button, etc.

I don't understand the problem. https://getbootstrap.com/docs/5.1/components/buttons/ twbs is ok with mixins.

If you need syntax sugar - leave it to CPP wrapper, instead of balloon C api. It's normal to add multiple styles (modifiers) to widget.

Hover

That's very expensive thing & not available on touch interfaces. IMO added value does not worth efforts. I'd suggest to postpone that as much as possible.

Markup language

IMO that's standalone project, described as "GUI creator tool with code generator". I'm skeptic about inventig/promoting new languages. Sugar can be partially solved via CPP wrapper. And visual gui builder is usually faster to create with js/html (electron and other).

Anyway, i'd prefer this thing separate from lvgl core. It may take too much efforts and slow down other development.


Also, it would be nice to temporary pause with adding new features & focus on pending tails - true RGBA support in styles, benchmarks infrastructure, CPP wrapper and so on.

kisvegabor commented 3 years ago

IMO this looks like mixture of CSS cascading & transforms animation. I would implement more classic features instead of invent new ones.

It's more than that and it's new only in LVGL. It's inspired by QT. See here. It helps to make otherwise public states private and encapsulate them into a component resulting in a much-much cleaner implementation. The transition can be part of it, but it's "just" a tool to organize your UI in a much cleaner way.

I don't understand the problem. https://getbootstrap.com/docs/5.1/components/buttons/ twbs is ok with mixins.

It's possible but not convenient. E.g. a React button component is much more powerful where the component's style and logic are encapsulated into one object. My suggestion is not about logic only styles though.

If you need syntax sugar - leave it to CPP wrapper, instead of balloon C api. It's normal to add multiple styles (modifiers) to widget.

The problem came up while I was working on several UI projects. It's just tedious to manage styles as they are now. And if it was inconvenient for me, I suppose it's problematic for others too.

Of course, the idea can be polished, I just wanted to raise the topic and a possible implementation direction.

That's very expensive thing & not available on touch interfaces. IMO added value does not worth efforts. I'd suggest to postpone that as much as possible.

I'm personally interested in it because we are planning to support desktop app development too with EdgeLine so, it will come in handy. It needs to be designed in a way to not be expensive. :slightly_smiling_face:

IMO that's standalone project, described as "GUI creator tool with code generator". I'm skeptic about inventig/promoting new languages. Sugar can be partially solved via CPP wrapper. And visual gui builder is usually faster to create with js/html (electron and other).

I consider having a markup language very important. I can't mention exact use cases due to NDAs but the need for a platform independent textual UI description came from many sources. It's will also help to find illogical parts in LVGL's UI, and clean it up as much as possible.

Also, it would be nice to temporary pause with adding new features & focus on pending tails - true RGBA support in styles, benchmarks infrastructure, CPP wrapper and so on.

RGBA, would be also a new feature. :slightly_smiling_face: I don't consider this as high priority task as it just joins to properties into one (color+opa). Cpp wrapper is on its way, but the development slowed down a little in the past month. Benchmarking would be great, but some contribution would be also great with it. I add it to the list

embeddedt commented 3 years ago

Let's discuss the topic of managing old & new releases.

I think what we are thinking of right now (assuming 8.2 is the next release) is the following:

@kisvegabor You and I had discussed before the idea of requiring every commit to master to be a PR with approval.

This could work for automating backporting as well, as I could make an action which tries to backport to a specific branch when someone with appropriate privileges writes a comment like "@lvgl-bot backport v8.0" in the PR.

However, it means that every commit (including simple ones like these) would need a PR opened. I suppose it would ensure that CI passes on master all the time, so maybe it's a good step?

kisvegabor commented 3 years ago

Meanwhile, I was thinking about if it's really worse the effort to establish a complex backporting system. After v8.0 was released I manually backported some fixes but not that much. As I haven't heard complaints about bugs in v8.0. What if we say that if someone reports a problem in v8.0 (or an older release) we manually fix that problem. Seemingly there won't be too many cases.

We can find a proper solution if there is a real need for massive backporting (maybe never :slightly_smiling_face: )

What do you think?

embeddedt commented 3 years ago

True. The majority of the backporting only tends to be needed across major version boundaries, because a decent majority of users will upgrade to the latest 8.x release if asked.

kisvegabor commented 3 years ago

Great! :)

The question of a good release script is still open. I've tried to do it for v7 but wasn't confident with it at all. I always run the Python script with a debugger because it always has some problems...

For v8 IMO the requirements are:

xiaoxiang781216 commented 3 years ago

Is it possible to merge lv_demos into lvgl git too? The demo is very useful to verify the porting and performance.

kisvegabor commented 3 years ago

Is it possible to merge lv_demos into lvgl git too? The demo is very useful to verify the porting and performance.

With this, the repo structure would be very clear. My only concern is that it'd significantly increase the repo size due to the many assets. lvgl is now 40MB, lv_demos is 65MB. With drivers and other stuff, we could easily reach 150-200 MB. It's not the end of the world, but something that's worth discussing.

xiaoxiang781216 commented 3 years ago

Yes, but compare with today's disk capacity and network bandwidth, most developer will more enjoy the convenience.:)

kisvegabor commented 3 years ago

I agree. This way LVGL core, 3rd party lib, example, demo, and driver would be in one place. As part of the CI tests, we could easily un the stress demo for a while too.

mysterywolf commented 3 years ago

Is that possable to add LV_MEMCPY_CUSTOM and LV_MEMSET_CUSTOM to let users or OS to re-implement lv_memcpy and lv_memset by themselves, just like LV_SPRINTF_CUSTOM? RT-Thread has rt_memset and rt_memcpy in assembly language, which is faster than lv_memcpy and memcpy(no matter under GCC or Keil), and to let users to accelerate the drawing and plotting speed.

kisvegabor commented 3 years ago

Could we rather integrate the faster versions into LVGL? :smile:

Can you give a link to your implementation?

mysterywolf commented 3 years ago

Yes. of course, this is one of the RT-Thread software packages. It's for Cortex-M. But I didn't translate the readme to English yet. 😅https://github.com/mysterywolf/rt_memcpy_cm

kisvegabor commented 3 years ago

But I didn't translate the readme to English yet.

No problem, Google Translate kindly translated it :slightly_smiling_face:

Copy 10,000 bytes 10,000 times in keil environment rt_memcpy is 1261ms This compilation version is 883ms C ​library memcpy is 906ms

Do you still have this test environment? If so could you try LVGL's memcpy?

mysterywolf commented 3 years ago

I retest under GCC Keil and IAR, and here is the result:😄

copy 10,000 bytes for 100,000 times (@STM32L745)

Time Cost (Unit: ms) Keil-MDK (-O0) GCC (-O0) IAR(-O0)
rt_memcpy (assembly language) 8819ms 7254ms 7227ms
lv_memcpy 17406ms 47879ms 16951ms
memcpy (C Standard Library) 9073ms 6940ms 10242ms
amirgon commented 3 years ago

@mysterywolf - Why would you do any performance measurement with -O0 ???

mysterywolf commented 3 years ago

@mysterywolf - Why would you do any performance measurement with -O0 ???

To test the worst case. I want to measure the performance of different memcpy version without compiler's help. I will test the -O2 case if you want.

amirgon commented 3 years ago

To test the worst case. I want to measure the performance of different memcpy version without help compiler's help. I will test the -O2 case if you want.

Comparing performance with -O0 is meaningless. The compiler makes no effort to achieve performance.
Showing that one compiler gives better performance than the other with -O0 does not prove anything. The results could be opposite with -O2 or -O3.

mysterywolf commented 3 years ago

Done. Here is the -O2 result.

Time Cost (Unit: ms) Keil-MDK (-O2) GCC (-O2) IAR(-O2)
rt_memcpy (assembly language) 8806ms 7212ms 7224ms
lv_memcpy 9510ms 8727ms 11056ms
memcpy (C Standard Library) 9063ms 6894ms 10237ms
embeddedt commented 3 years ago

@amirgon I'm not sure that optimization matters for something simple like a memcpy implementation. It makes sense for higher-level code, but memcpy is really just two pointers being incremented, dereferenced, and assigned. It tends to be sped up more by manual tricks like copying words instead of bytes. (edit: didn't notice the 50 seconds for lv_memcpy at the time of writing)

@mysterywolf Thanks for these measurements.

In general lv_memcpy seems to be reasonably performant and has the advantage of being 100% portable. However, I see nothing wrong with offering an option to use a custom memcpy implementation.

mysterywolf commented 3 years ago

@embeddedt Same like lv_memset, rt_memset also is implemented by C. The assembly rt_memset version (for Cortex-M) only can be used with the corresponding architecture BSP. e.g. if you are using the STM32 series, you can use the assembly version.

And you are right. In RT-Thread side, we also find that GCC-Newlib's memcpy is not reliable. When we try to use newlib’s memcpy to operate the hardware registers, such as SDIO https://github.com/RT-Thread/rt-thread/pull/4015/files. It’s will crush. So lv_memcpy or rt_memcpy will be safer than newlib’s memcpy. The assembly rt_memcpy version also has considered the 4 bytes alignment.

mysterywolf commented 3 years ago

In general lv_memcpy seems to be reasonably performant and has the advantage of being 100% portable. However, I see nothing wrong with offering an option to use a custom memcpy implementation.

For sure, We need to keep the lv_memcpy and memcpy this two options simultaneously, and lv_memcpy should be set as default option to be safty-guaranteed. My suggestion is that it would be great that providing a marco to let users can reimplement the lv_memcpy in assembily language for specific CPU to accelerate the speed, e.g. LV_MEMCPY_CUSTOM.

kisvegabor commented 3 years ago

Glad to see that lv_memcpy is not that bad with -O2. :slightly_smiling_face:

I wonder why Keil is so much slower than the other two compilers when rt_memcpy isn't even written in C.

It's really interesting.


I'm ok with LV_MEMCPY_CUSTOM and LV_MEMSET_CUSTOM but we should maintain the compatibility with the current LV_MEMCPY_MEMSET_STD. It can be done via some hardcoded wrappers in lv_conf_internal.h.

X-Ryl669 commented 2 years ago

Adding dithering to the list, maybe, since it's a low hanging fruit now ? See #2872

mysterywolf commented 2 years ago

I will translate the homepage's readme to Chinese language version as README_zh.md before the next version releases of LVGL.

tyler-gilbert commented 2 years ago

How about adding the ability to customize the LV_SYMBOL_* values? This would allow using different icon sets without having to change the keyboard icons. It should be super easy to implement, just add some guards to the current LV_SYMBOL_* definitions that allow the symbols (either individually or collectively) to be over-ridden by values from lv_conf.h

kisvegabor commented 2 years ago

@tyler-gilbert It makes sens to me. Could you send a PR?

kisvegabor commented 2 years ago

I've added LV_OBJ_FLAG_OVERFLOW_VISIBLE in e7ac0e419

kisvegabor commented 2 years ago

According to (hopefully) the final plans EdgeLine will be released in end of January and it'd be great to make it work with the latest LVGL version. The do this there should be a release in few weeks. IMO only the fact that the demos were moved to lvgl would be worth a release as it makes the usage of LVGL with Ardunio and PlatformIO much easier. Besides #2957 also requires a new release.

Of course, we can roll the ideas in the first post to v8.3.

So is there any objections against it?

@embeddedt, @X-Ryl669, @rzr, @brgl, @mysterywolf, @tore-espressif, @MouriNaruto is there anything you want to be included in v8.2?

rzr commented 2 years ago

nothing special, do what you think is best to have it released for fosdem ;)

embeddedt commented 2 years ago

Are we merging the drivers in this release or waiting for 8.3? I don't think it matters too much. Just want to confirm that that's still the plan.

kisvegabor commented 2 years ago

@embeddedt I think it needs more time, thinking and work to merge the drivers so I'd keep it for a later release.

MouriNaruto commented 2 years ago

I have no issue because @embeddedt has been asked the question I wonder to know.

Kenji Mouri

MouriNaruto commented 2 years ago

@kisvegabor

If you start to merge the drivers, I will try my best to help you merge the win32drv driver.

Kenji Mouri

mysterywolf commented 2 years ago

is there anything you want to be included in v8.2?

Nothing special

We will release v4.1.0 Beta version to cooprate with your v8.2 release plan. We are promoting more chip vendors to port LVGL to their RT-Thread BSPs. I think Nuvoton has connected with you.

tore-espressif commented 2 years ago

We are getting compile warning and Kconfig warnings when building LVGL in our internal pipeline with strict compiler settings.

I'll publish PR that fixes this, so it can be included in v8.2 after review

X-Ryl669 commented 2 years ago

I think I'm done with the dithering code & gradient code. I've added a cache for the gradient map computation. But it breaks previous code compatibility if one were to set the draw_dsc.bg_color field directly instead of using the style functions. The change is easy (replacing .bg_color = by .bg_grad.stops[0].color =, but I don't know the policy for small breaking changes and minor version bump.

tyler-gilbert commented 2 years ago

@tyler-gilbert It makes sens to me. Could you send a PR?

Yep. Here is the PR: https://github.com/lvgl/lvgl/pull/2973

kisvegabor commented 2 years ago

@mysterywolf

We are promoting more chip vendors to port LVGL to their RT-Thread BSPs. I think Nuvoton has connected with you.

Amazing, thank you very much! Yes, Nuvoton just contacted me. I didn't know you directed to them to me. :slightly_smiling_face:

@X-Ryl669

I think I'm done with the dithering code & gradient code

I just reviewed it, thank you!

@tyler-gilbert

Yep. Here is the PR: #2973

Merged, thank you!

lanistor commented 2 years ago

Should support epaper. vroland/epdiy can be used.

C47D commented 2 years ago

See https://github.com/lvgl/lvgl_esp32_drivers/pull/75 for related topic