search

lvgl / lvgl_esp32_drivers

Drivers for ESP32 to be used with LVGL
MIT License
324 stars 279 forks source link

willing to give a lift #1

Open zladay opened 4 years ago

zladay commented 4 years ago

I am into Sitronix ST7735 style (e.g. ili9341, ili9488 and the like) TFT controllers. I am also working with different TCs (STMPE811, XPT2046 etc.) I am interested in a more ESP-IDF-ish driver infrastructure with a streamlined lvgl interface. Willing to share ideas, designs and code examples. Looking for some guidance since this is an empty space now.

kisvegabor commented 4 years ago

Hi @zladay

Glad to see that you are interested in contributing to the project. Every help is welcome! :slightly_smiling_face:

@C47D, the maintainer of lv_port_esp32, is also into this topic.

We are still in the planning phase, finding the right API, layers, etc. Do you already have any ideas?

C47D commented 4 years ago

Hi @zladay,

The initial work is already done on the lv_port_esp32 repo, we will copy the lvgl_esp32_drivers directory into here, so users can use the drivers without needing to use the whole lv_port_esp32 project (we are also working on the lvgl and lv_examples repo to make them easier to use with ESP-IDF framework and other frameworks that use Kconfig). We already have support for ILI9341, ILI9488, ILI9486, HX8357B/HX8357D, ST7789, ST7735S, SH1107, SSD1306 and IL3820 display controllers, and XPT2046, FT3236 and STMPE610 touch controllers.

Support for those drivers will be gradually added into the lvgl_drivers repo, this repo requires work.

Hopefully I can start working in this repo this weekend, I'm currently trying to fix a bug in the IL3820 display controller.

Any ideas let us know.

Regards

zladay commented 4 years ago

I am aware of the drivers you mention. I started working with an original version of @kisvegabor's ESP32 demo. (There was only ili9341 and xpt2046 support at that time). I put together a completely new driver architecture initially targeting ili9341 and STPME811 only (this was the only hardware I had back then). Design objectives included

kisvegabor commented 4 years ago

@zladay That's great! Could you upload your code somewhere to see how it looks like?

zladay commented 4 years ago

Sure. I need some time to clean it up, though. I will link it if it is done.

C47D commented 4 years ago

I can upload the existing code here and you take over with your improvements, I'm sure there's a lot to be done with the existing code on the lv_port_esp32 display and touch (indev) code.

zladay commented 4 years ago

I have already set up a repository with my drivers. In order to demonstrate the new approach, I need to come up with a new demo program, too. I have also set up a repository for this. I have been using a pretty old version of lvgl and an lvgl example. The "Write List Chart" one. I decided to include lvgl and lv_examples as submodules in this repo. It is now done. In this process, I switched to the master branch of both of these lvgl parts. At this moment, my ancient main.c does not compile with this. I have just recently realised that my favourite "Write Chart List" is gone. I need to find another demo program. (Any suggestions?) I had to fight the build system a bit, but it works now. Nothing serious, I am on track. It has, however, delayed my attempt to clean up and reorganise the driver code. I am asking for a little patience still. I am planning to link my repos when it is worth to show them. Given the above circumstances, however, I am not sure if it is worth to populate this repo with the old drivers now. At least it does not help me at this moment.

zladay commented 4 years ago

I have just changed the demo to lv_widgets_demo. Demo compiles and works on first try with the latest lvgl, wow. ili9341 (320x240) and xpt2046. Touch works, too. Demo and drivers are committed to the Github repos. I can start the cleanup and reorganisation now. I will keep you updated.

C47D commented 4 years ago

Hi,

~For start you can only show a label in the display, once it's working we can work on showing any of the lv_example demos.~

Maybe we can use your architecture as a base and add the old drivers on top of that, updating any necessary code, we have to remember this repo will be used as submodule on the lv_port_esp32 repo, what do you think? I have some drivers so we can test with them.

The lv_port_esp32 repo has three components, lvgl, lv_examples and lvgl_esp32_drivers. I'm working on separate those, currently writing the Kconfig file for LVGL.

Do you have your repos available somewhere?

OFFTPIC: It would be nice to have a discord server about LVGL.

C47D commented 4 years ago

Also, are we going to compare both approaches? I haven't measured how busy the SPI master is in the current approach.

zladay commented 4 years ago

As I wrote earlier, the lv_example problem is solved. This widget demo is scrolling through the TFT right now. (I am missing the "Write List Chart" BTW, that looked much nicer :) Oh yes, the repo question! I did not think it is going to be this complex when I showed up here. @kisvegabor asked me to upload the code somewhere. This is why I had to set up a repo for the drivers. Later, I realised that the driver code can not be used in itself. It has a completely new initialisation and lvgl interfacing. I could come up with some sort of documentation to describe how it is intended to be used, but I thought it is better to show it in a working demo application. This is why I started to build it and created a separate repo for this. The demo repo is constructed in a way that it uses lvgl, lv_examples and my drivers as submodules. My drivers are organised in a separate demo as multiple ESP-IDF components (yes, not one component, but more components). You can exclude certain components from the build. If, e.g. you do not have STMPE811 in your hardware setup, you can leave the corresponding component out. The driver repo is designed in a way that it is not strongly connected to the demo app. It can be used for any application preferably as a git submodule. This is meant to contain all the code. If you want to exclude something from the build, you need to make the necessary changes in CMakeLists.txt under main. (All components are included by default). The components in the driver repo are organised in a hierarchical manner. There are common components (e.g. SPI handling for TC aka tc_spi). These are required for all the TCs. If you want to use STMPE811 e.g., it is enough to specify that component, it will pull in dependencies automagically (e.g. tc_spi). This driver repo can, of course, also be a part of lv_port_esp32. lv_port_esp32 (its main.c) needs to be heavily reworked in order to use this drivers, though. Until that happens, we can use the repo I put together. It is not meant to be final. I have not yet implemented all my ideas in the drivers & the demo. I have yet not touched lvgl itself yet, but I have ideas how we could make interfacing much better with that. My code is meant to be a work in progress. I do not want to break the applications of many people using the publicly available code.

zladay commented 4 years ago

You have a lot of questions :) I am trying to answer all, however. Discord server (or any other real-time comms application)? I am definitely in. I have suggested this to @kisvegabor earlier.

zladay commented 4 years ago

"Maybe we can use your architecture as a base and add the old drivers on top of that" This is my intention, too. Later. This time, I think, it is a bit early. As I said, I need to reorganise the code first. It does not yet fully reflect my ideas and objectives yet, not to talk about other people. Your suggestions are also welcome. My initial plan is to support the following hardware:

embeddedt commented 4 years ago

Discord server (or any other real-time comms application)? I am definitely in. I have suggested this to @kisvegabor earlier.

We typically prefer to use communication mediums that can be searched more easily (i.e. GitHub issues and the forum). It helps reduce the number of duplicate questions.

EDIT: I forgot to mention that the forum actually behaves very similarly to a real-time chat system; I'm pretty sure you get notified almost instantly if browser notifications are enabled.

zladay commented 4 years ago

What do I mean by reorganisation? Maybe it is news for you, but the above TFTs do not need separate drivers. Only the initialisation is different, and that difference is only one bit in the MADCTL register (RGB order). I want to write a generic driver (not writing it actually, since any will do, of course) as a common component and a very thin specific driver for each of those. Similarly, there is no visible difference between the STMPE811 and STMPE610. One generic driver is enough. But, just not to confuse people, there is going to be two different initialisations for it for each device.

zladay commented 4 years ago

Performance measurements? I did some with an oscilloscope and logic analyzer. The color buffer is transmitted on the SPI bus via DMA (from day one). This is very fast. It can not be improved. Neither could I. What you can optimise is the setup of those transfers. Probably I did, but it is insignificant. I did a measurement with @kisvegabor's original demo (at 40 MHz if I remember correctly) and the bus was utilised at about 16% with the demo app slide show. I, hopefully, could improve the robustness of the TFT driver, however. I saw that the public demo has a pinned task for some reason. I never used that technique and never experienced any problems (maybe I was not trying hard enough). I could optimise the TC drivers. I added support for the FIFO (it is available on the STMPE devices only) and simplified the handling substantially. Unfortunately, TCs work in polling mode. Further improvement could be achieved if lvgl supported interrupt driven indev handling.

zladay commented 4 years ago

"Do you have your repos available somewhere?" Yes, here on Github. I will link them if it is going to be worth to see them. I need some days still.

zladay commented 4 years ago

There is one catch with the TFT drivers. As I said, they do not need separate drivers. There is a problem with the supported RGB modes (on the 4 wire MCU interface, aka SPI), indeed. If I remember correctly, the supported modes are as follows:

The RGB 666 seems to be Satanic for lvgl (not just because of its name). It takes 24 bits on the SPI. 1 octet per channel. The 2 LSBs are ignored (do not care). This goes well with 24bit color depth (some resolution is lost, only 262k colors instead of 16.7M). Unfortunately, however, this does not go well with lvgl. Lvgl dropped support for 24 bit color depth. This is a real pain with the ili9488, since it does not support 16 bit RGB (on SPI). One ugly workaround exists (as demonstrated in the public driver, too). It is the conversion of the 16 bit RGB 565 into 18 bit (effectively 24 bit) RGB 666 in the driver code before transfering the buffer to the DMA to send it out. Ahhh. Performance? Memory footprint? The only correct way would be if lvgl supported 24 bit mode. I looked at the relevant code portions, and it was visible that it would take substantial effort to add it. In fact, it would necessitate a serious refactoring of the low level color handling there. Actually, the situation is even worse. Only one color depth is supported concurrently. Color depth needs to be decided at compilation time. Even though my new driver architecture supports the concurrent handling of any number or any type of displays, lvgl is limited to a single RGB model. It is, therefore, impossible e.g. to use a monochrome OLED together with a color TFT in one device. Could you please inform @kisvegabor about it? What do you suggest with the ili9488? Shall the driver do the ugly workaround?

kisvegabor commented 4 years ago

Chat Can you describe why do you think the chat is required?

Personally I'm happy with forum/github. I got a lot of emails and notifications from these platforms. On top of these a chat - that requires real-time presence - would be too much for me. Besides - as @embeddedt mentioned - the issues and forum topics can be easily linked to answering questions quickly.

TFT + monchrome You can use the set_px_cb to write the buffers as you wish. They are used anyway because some monochrome displays pack 8 pixels into a byte.

Architecture design We should keep in mind that the new drivers' interface shouldn't be specific to ESP only.

So I see these abstraction layers:

What do you think?

zladay commented 4 years ago

Chat

@C47D proposed this. I will leave this to him to answer.

TFT + monochrome

I am aware of this. I never said that the drivers can not be written for MC OLEDs (especially because those are already written). I just said that drivers tend to be complicated because of the sub-byte addressing. TFT drivers are different since the minimum practical resolution (16 bit) exceeds one byte. There are problems with color depth support. There is sometimes a mismatch between lvgl and the TFT capabilities (ili9488 is the only known example for me so far). Currently, my drivers support 16 bit color depth for the TFT only. It is, however, not difficult to add support for any RGB model the HW supports. Let's see what is available in general:

Lvgl does support 32 bit depth (RGB 8888). This is incompatible with RGB 666, so (down) conversion is required. It is just slightly better than up converting from RGB 565, since it is offering only 2 bits extra color resolution at a cost having 2x bigger buffer sent from lvgl compared to the RGB 565 case. My preference is to sort out the RGB model issue (What is supposed to be supported by the driver?) before going to an entirely different area of monochrome displays (there are no questions related to MC displays on my side so far). I am, however, open for different suggestions. I was, however, making the point that mixing entirely different displays (e.g. monochrome and color TFT) in one device is currently impossible because of an lvgl limitation (even though my driver architecture supports that scenario apart from the fact that monochrome drivers are not yet added).

Architecture design

I fully agree on your objectives in theory, but there are some problems in practice. When designing the driver architecture I kept the guidelines you mention in my mind. I have (distant) plans to extend this to other frameworks, too (e.g. mbed, ST Cube, etc.)

Work with or without OS

I am not sure if we understand the same for "with OS". I suppose that it means that the development framework contains a scheduler with task handling and synchronization primitives (e.g. FreeRTOS). When a framework has this (e.g. ESP-IDF), it is actually a must to do proper synchronization and CPU monopolization (busy waits) should be avoided. In an environment like this, a "no OS" version does not make sense. When there is an OS, FreeRTOS can not be taken for granted. Some frameworks (e.g. mbed, Nuttx) use something else for this purpose. The only way to make the drivers OS independent is to develop something like an "OS abstraction layer". It is not impossible, but I consider this idea a bit farfetched now. What is the reason behind having drivers for a "without OS" scenario? I can not recall a useful framework that does not have an "OS". (Fun fact: Even Arduino for ESP32 does have an OS. AFAIK, the ESP32 Arduino core is implemented using ESP-IDF. The "OS" stuff is, however, carefully hidden under the hood.)

Support for multiple transport interfaces (I2C, parallel, etc.)

I fully agree on this, but current code supports SPI only. Transport layer can be detached and a multiplexer can be built in to use multiple different types of interfaces. As I said, I have not done it yet. Simply because I did not want to jump that long right now.

Support for vendor-specific drivers (e.g. ST)

I do not really get it. All the drivers are vendor specific (Sitronix, Illitec, Solomon Systech, Xptek). The only difference in the mentioned ST case is that the display driver hw is within the MCU (but the touch controller is external, it is STMPE). The GRAM can be reached via an internal high speed parallel bus. This is technically (as far as the drivers are concerned) not too different from the situation when an external controller is connected via a parallel interface. (BTW, I have been having an STM32F7 Discovery kit (TFT+touch) for a while. It was set aside, because I could not find a good graphics library. I do not want to look at it now, because ESP32 drivers would be delayed then :)

Abstraction layers

My architecture is similar to what you wrote. As I said, it is missing the transport multiplexing layer yet. I want to make it available before it happens, because I am interested in the opinion and suggestions of the people showing up here as early as possible. It is not perfect, but I hope that it is going te be a good starting point and we could develop it further.

Anyways, I think I need to stop talking too much. It takes away time from reorganization of the code. I am progressing with that. I will keep you updated.

kisvegabor commented 4 years ago

RGB 666 and RGB888 are really not supported and I don't see an easy way to do it because the current formats can be natively addressed in every case (1,2 or 4 byte color types). Let's postpone it a little.

mixing entirely different displays (e.g. monochrome and color TFT) in one device is currently impossible because of an lvgl limitation

"Impossible" is not entirely true. You can convert the colors to the desired format in the flush_cb. It's ugly though but not impossible. For small displays, it shouldn't make a big difference,

If support for e.g. ST framebuffers fit well into the design then all good fro my side :)

zladay commented 4 years ago

Just a small clarification for the RGB problem. There is no need to support RGB666 in lvgl. RGB888 would suffice. Why? TFTs (including ili9488) expect RGB666 in 3 bytes. One byte for each color channel. These 6 bits are expected in the MSB part of the byte (D7-D2). TFT controller disregards D1-D2. Using RGB 888 here is fine. The HW does the "conversion". Two LSBs would be lost for each channel. Application graphics designed for RGB 888 would work with a slight image quality degradation.

The workaround you mention makes the concurrent use of a monochrome and a color TFT possible indeed. It is not impossible, you are right, but this is so ugly, that I was not even considering this as a "possibility". If color TFT uses 16 bit RGB, the same is forced to the monochrome. This requires 16 times bigger buffer than normally needed plus the CPU cycles of converting and the 1 bit/pixel buffer that needs to be sent on SPI. RAM is pretty limited on the ESP32. I had to decrease the flush buffer size even when I had only one ill9488 with the conversion needed, otherwise the app would not fit (linker error). With small flush buffers performance suffers even more.

C47D commented 4 years ago

I suggested the chat because it's more real time, only intended for developers, not for general use. But thinking again about it, we should make the discussion public so anyone can add suggestions.

Work with or without OS I am not sure if we understand the same for "with OS". I suppose that it means that the development framework contains a scheduler with task handling and synchronization primitives (e.g. FreeRTOS). When a framework has this (e.g. ESP-IDF), it is actually a must to do proper synchronization and CPU monopolization (busy waits) should be avoided. In an environment like this, a "no OS" version does not make sense. When there is an OS, FreeRTOS can not be taken for granted. Some frameworks (e.g. mbed, Nuttx) use something else for this purpose. The only way to make the drivers OS independent is to develop something like an "OS abstraction layer". It is not impossible, but I consider this idea a bit farfetched now. What is the reason behind having drivers for a "without OS" scenario? I can not recall a useful framework that does not have an "OS". (Fun fact: Even Arduino for ESP32 does have an OS. AFAIK, the ESP32 Arduino core is implemented using ESP-IDF. The "OS" stuff is, however, carefully hidden under the hood.)

We need to remember that the drivers supported by the lv_port_esp32 are planned to be ported to lv_drivers. Which are going to be used in bare metal and RTOS projects.

I fully agree on this, but current code supports SPI only. Transport layer can be detached and a multiplexer can be built in to use multiple different types of interfaces. As I said, I have not done it yet. Simply because I did not want to jump that long right now.

SPI and I2C are currently supported because that's the interface of the displays we have available, I can't do any parallel because I have no display to test it and sometimes is slower to wait for test results. Sure we can do an abstraction layer so it doesn't matter what SPI, I2C, parallel or custom interface is used.

So I see these abstraction layers: Display (mainly commands to configure, write pixels, set orientation (?)) Display communication (I2C, SPI, ..., OS or no-OS) Peripheries (How to initialize and use SPI, I2SC ... )

I think we shouldn't do everything, users should init the SPI, or any peripheral they are using. They should set some callbacks (functions that takes an array and send it over whatever interface they are using), we only need to arrange that array of bytes with the color data.

As @zladay suggested, we can make a generic ILI9x driver and make the initialization and rotation code driver dependent.

kisvegabor commented 4 years ago

@zladay

RAM is pretty limited on the ESP32. I had to decrease the flush buffer size even when I had only one ill9488 with the conversion needed, otherwise the app would not fit (linker error). With small flush buffers performance suffers even more.

If you use 32 bit color depth for LVGL you can make the conversation in the color buffer as well without allocating a new temporary buffer.

Seemingly we are on the same page. :slightly_smiling_face:

zladay commented 4 years ago

@zladay

current code supports SPI only

@C47D

I2C are currently supported

Let me resolve the contradiction here. I referred to my code. @C47D talks about the public version. I am aware that the public version has support for eye squared see.

I2C is practical (or available at all) at small, mostly monochrome displays. I have already written SSD1306 (and SH1107 which is almost the same) drivers for an other platform (mbed, if I remember correctly). I have experience with that even on I2C. So, I can tell you that I am not afraid of them. The reason I would not focus on that now is that I want to finish the work what I started (written above) first. This is bringing a lot of news into the driver arena already. My second priority is to extend this to other transports (I2C and parallel). @C47D seems to have a different agenda and priorities. He is focused on the a breadth of HW support instead (whatever architecture those are in).

zladay commented 4 years ago

We need to remember that the drivers supported by the lv_port_esp32 are planned to be ported to lv_drivers. Which are going to be used in bare metal and RTOS projects.

I am not sure that I know what "drivers supported by the lv_port_esp32 are planned to be ported to lv_drivers" means.

I am also not sure if your comment was intended to support or dispute my thoughts.

If it is RTOS, it is obviously not a "without OS" scenario. If bare metal, you are not confined to a certain framework. You can choose whatever scheduler or OS you want.

zladay commented 4 years ago

I can't do any parallel because I have no display to test it and sometimes is slower to wait for test results.

I do have some TFTs with parallel interface, but I do not want to work with them now, because

  1. Parallel does not go well with ESP32, because there are not many pins to form the bus
  2. I do not want my efforts to be too fragmented
zladay commented 4 years ago

I think we shouldn't do everything, users should init the SPI, or any peripheral they are using. They should set some callbacks (functions that takes an array and send it over whatever interface they are using), we only need to arrange that array of bytes with the color data.

I disagree strongly. (It is a pretty long story why.)

zladay commented 4 years ago

As @zladay suggested, we can make a generic ILI9x driver and make the initialization and rotation code driver dependent.

It is almost ready now. I am currently fighting with the rotation story. It is not simple if you want to give a general solution. There are a couple of mines there. I am sorting this out at this moment. I hope that you are going to be able to see the results soon.

zladay commented 4 years ago

If you use 32 bit color depth for LVGL you can make the conversation in the color buffer as well without allocating a new temporary buffer.

Maybe I do not fully understand how RGB 8888 (32 bit color depth) works, but I am afraid that your above solution does not work.

RGB 8888 mode populates the flush buffer with 32 bits/pixel. Four bytes for pixel one, then four bytes for pixel two and so on. For RGB 666 e.g., the TFT expects 3 bytes/pixel. If I am receiving a buffer from lvgl in RGB8888 format, how could I transfer it to the SPI without converting it and allocating a buffer for conversion? The received buffer should be sent out en bloc, not per byte, since SPI/DMA does the heavy lifting. It expects an entire buffer that is already properly formatted.

kisvegabor commented 4 years ago

I think we shouldn't do everything, users should init the SPI, or any peripheral they are using. They should set some callbacks (functions that takes an array and send it over whatever interface they are using), we only need to arrange that array of bytes with the color data.

I disagree strongly. (It is a pretty long story why.)

Why? I like Carlos's approach to make user init the bus(es) and set callbacks to the driver. Writing init and send functions for every vendors every MCU is an endless task. It's important to allow users to add their own drivers. We can provide built-in support for some platforms though but it couldn't be the only way.

If I am receiving a buffer from lvgl in RGB8888 format, how could I transfer it to the SPI without converting it and allocating a buffer for conversion?

Conversion is required but to allocation isn't. E.g. (not tested, probably wrong color channel order)

uint8_t * c8 = (uint8_t * ) color_p;
for(i = 0; i < lv_area_size(area); i++) {
  c8[i * 3 + 0] = c8[i * 4 + 0];  
  c8[i * 3 + 1] = c8[i * 4 + 1];
  c8[i * 3 + 2] = c8[i * 4 + 2];
}
C47D commented 4 years ago

Yeh, I struggled with rotation as well, mainly on the ili9341 driver because we support the Wrover Kit, M5Stack and the Chinese display with that driver, all of them seem to be wired differently.

We can try to do everything by ourselves, it's the current implementation on the lv_port_esp32 repo, the users only call the helper function and the interface peripheral is initialized, flush, set_px and round callbacks are called when necessary.

Looking forward for your proposal, I think we can make the esp32 driver support much better with it.

zladay commented 4 years ago

Conversion is required but to allocation isn't. E.g. (not tested, probably wrong color channel order)

uint8_t * c8 = (uint8_t * ) color_p;
for(i = 0; i < lv_area_size(area); i++) {
  c8[i * 3 + 0] = c8[i * 4 + 0];  
  c8[i * 3 + 1] = c8[i * 4 + 1];
  c8[i * 3 + 2] = c8[i * 4 + 2];
}

If I understand it correctly, the conversion is in place. The buffer offered to the driver is compacted in place by the driver. In this case, you are right. No additional buffer allocation is necessary. I seem to be too well-mannered, I was considering the buffer as const, a property of lvgl, something I should not mess with.

zladay commented 4 years ago

As I said, I am struggling with the orientation (and rotation) stuff. What setup works with one panel is broken on the other. What is completely normal on <st7735s 128x160> is broken on <ili9341 240x320>. On the <ili9341 240x320>, X starts from the right edge, and because of this, the image is mirrored on the y axis and also shifted to the right. Shift is only visible if you are not using the full X range, of course. This is how I could figure it out. First I thought that the problem lies in the difference between ili9341 and st7735s. But no! The chips are identical in the way how they handle column and row (page on ili9341) addresses! (Column -> X coordinate; Row or Page -> Y coordinate). What makes this mess then? The ili9341 based module designers were real geniuses (or rather idiots or tricksters). They connected the column lines between the TFT and the chip in an abnormal order. Not from left to right, but from right to left. Of course, I have not disassembled the module to check the wire routing. I compared the data sheets. There is only one difference, but that is not semantic. Illitek refers to the Y coordinate as page address, while Sitronix uses row address instead. There is no logical difference between the two controllers. I did not expect that I would need to deal with these "smart" panel design decisions, too.

I wrote the above comment yesterday, but I forgot to push the "Comment" button.

zladay commented 4 years ago

The ST7735S and the ILI9341 works with exactly the same generic driver. No changes are necessary. No difference in the initialization, either. Earlier I thought, the two controllers only differ in RGB order (it is in the MADCTL register). At least, that was what the data sheets said. After testing, I concluded that RGB order handling is identical. (The ILI9341 DS might have a mistake, or I remember incorrectly.)

The ILI9488 is expected to be the same as the others. In this case, however, the generic driver has to be modified (the flush function replaced), because buffer conversion is needed due to lack of RGB 888 (including RGB 666) support in lvgl.

zladay commented 4 years ago

Orientation

It seems to me that I could get past this issue last night.

Driver handles orientation properly. This is done the following manner.

Base orientation can be set at init (it is then affected at calling reset on the display, base orientation is restored after any subsequent reset call). This handles different user scenarios including:

  1. User wants to use the display in a rotated mode (e.g. 320x240 on an ili9341 while the native is 240x320 instead).
  2. The panel is mounted physically upside-down in the hardware (for mechanical or electrical reasons e.g.)
  3. The combination of the two above

Base orientation can be set in 90 degree increments clockwise. Zero degrees corresponds to the native orientation. Base orientation belongs to a specific hardware setup, how the device is constructed.

In addition to base orientation, another orientation can also be set. Why is it that that way ? Base orientation can be set at reset (either after init or later), Orientation can be set at any time during display operation with an orientation call on the display. Orientation can be set in the same manner as base orientation. It is, however, relative to it. Orientation is meant to be used during the operation of the device (as opposed to at initialization). Orientation call effects the MCU interface only. Nothing is changed on the display after a call until a refresh (flush) happens. This is put in there for preparing for dynamic orientation handling. Orientation can be changed when the user rotates the device physically (as it is on phones). 180 degree rotations are possible even with the current lvgl version. Users should call orientation and a subsequent redrawing (refreshing) of the entire screen area. @kisvegabor, is this really possible in lvgl now (provided that orientation calls are funnelled up to lvgl)? 90 or 270 degree rotations is a different story. lvgl resolution needs to be changed as well (240x320 instead of 320x240 e.g.). This does not work now. lvgl orientation is static, can not be changed run-time.

zladay commented 4 years ago

Weird panel wiring scenarios

As I mentioned earlier, some panels are having a weird wiring arrangement between the controller chip and the TFT. I have found a couple of panels where column wiring was reversed (right-to-left instead of left-to-right). I could address this issue, too. You can configure the panel with reverse wiring for both columns (X) and rows (Y) at init. This can not be changed later (after init), but no one is expected to re-wire the panel while it is in operation :)

These settings do not interfere with orientation, they are symbiotic.

Wiring configuration and orientation settings are achieved by hw settings (MADCTL) purely. There was no need for any sw fiddling (manipulation of coordinates or the buffer during flush - flush is left intact).

zladay commented 4 years ago

Other orientation related considerations

Mirroring

Mirroring might be useful where the screen is not seen directly, but reflected from a surface. E.g. from a mirror or a car windshield (HUD scenario). Current version does not support this, but it is very easy to add it later for both X and Y as mirroring axes. This could even be done dynamically (run-time).

Touch controller

As I wrote earlier, screen orientation can be set in any way. It can even be changed run-time. It has some implications, though. Lvgl related implications were written above. The other problem is the TC. TC coordinates do not yet follow screen orientation changes. It is to be added later. It is not particularly difficult in my implementation, because translation between TC and display coordinates are done by matrix multiplication. This translation matrix can handle any orientation scenario (even when the rotation angle is not a multiple of 90 degrees, e.g. when the touch panel is stuck skewed on the TFT). This matrix is square (or at least it can be handled as it was square). Translation matrix shall be determined for the base orientation (it is constant). If a screen rotation appears, then the basic translation matrix should be transformed into a new one. This can be done (because the matrix is square) by another matrix multiplication (multiplication of the basic matrix with a translation matrix reflecting the required amount of rotation). This means that a new (static) matrix can be derived. It does not impose a performance penalty, because TC still uses one matrix multiplication (required for any touch event). Translation matrix needs to be recalculated only if orientation changes. It is not happening frequently. Even when it is going to be added to the TC drivers, it is going to be a bit awkward from the application developer's perspective. She is expected to issue two orientation calls (one for the display, one for the TC). For this reason and some other similar reasons, it would be desirable to handle the display and the TC together. One orientation change call would suffice among other benefits. What do you think?

zladay commented 4 years ago

I think we shouldn't do everything, users should init the SPI, or any peripheral they are using. They should set some callbacks (functions that takes an array and send it over whatever interface they are using), we only need to arrange that array of bytes with the color data.

I disagree strongly. (It is a pretty long story why.)

Why? I like Carlos's approach to make user init the bus(es) and set callbacks to the driver. Writing init and send functions for every vendors every MCU is an endless task. It's important to allow users to add their own drivers. We can provide built-in support for some platforms though but it couldn't be the only way.

I am not sure if I understand what you (@kisvegabor & @C47D) say. I do not really want to discuss this now. I hope that I can make my drivers and their application in a demo program available soon. That is also meant as my statement about what direction drivers should go. As I said earlier, it is not complete, because I have more ideas than I have implemented so far (and I am not too optimistic, it will stay that way forever). In spite of this, it is, I think, a good starting point for the discussion. You can then tell me what is wrong and what you could improve.

It's important to allow users to add their own drivers.

That is the only thing I understood. BTW, I completely agree with that statement.

zladay commented 4 years ago

I hope that I can make my drivers and their application in a demo program available soon.

Almost everything is ready for rolling it out from the hangar.

I have still troubles with ili9488 driver. It worked (and still works) in a standalone implementation, but I want to rearrange that as a derivation of the generic driver. I.e. to keep the generic intact and redefine the flush function only in a separate unit. I have a nasty problem with that, however. The whole device has gone astray when I tried to use this chimera. It is yet to be seen wether the cause is a direct sw bug, or the device does not tolerate the decreased amount of free memory.

I do not want to mess with the generic driver, I do not want to decorate that with normally unnecessary code portions. I want to keep the workaround separate. We can then jettison that as a sand bag from a balloon once lvgl is taken to the next level.

zladay commented 4 years ago

Quick update on the chimera.

It did not work because of a trivial, not driver related problem. It is a higher resolution screen, so I set the resolution to 480x320 in lvgl. Flush buffers became so big that it exceeded the maximum transfer size configured for the SPI bus (not the device). When I increased that, everything had become working.

So, the chimera is kept separate. Can be jettisoned as a sand bag.

Note:

Even though this chimera is a really ugly beast, but this exercise is very useful otherwise. It shows how a new display driver can be derived from the generic one with minimal effort.

zladay commented 4 years ago

More of the chimera story

The chimera is not ugly because it is a chimera. It is ugly because of the extra memory needed for the conversion and the necessity of the conversion itself. Hence, this particular chimera is ugly as hell, but other chimeras can be beautiful.

As I wrote earlier, from the ESP-IDF perspective, all the drivers are separate components. In this case it is the "ili9488" component. It can not do anything useful in itself, because it needs the generic (Sitronix style) display driver and the "disp_spi". These two things are, therefore, the dependencies. With the appropriate CMakeLists.txt files, I configured the build system in a way that dependencies are automatically pulled in when the user (app developer) specifies the "ili9488" component as a required part of the application. By default, all components (including all ESP-IDF components) are included in the build. If you want to change that (and save a lot of memory), you should list the required components in the CMakeLists.txt file in the "main" (not project root) folder.

In the case of this nasty workaround, it comes really handy. If someone does not have ili9488 in her hardware setup, she can omit that from the list of the required components. The savings on the code portion (flash) is not so big, since most work is done in the generic driver, but the RAM savings can be substantial. "ILI9488" uses a statically allocated buffer for conversion, and this is in this component. No component, no buffer allocation.

zladay commented 4 years ago

Drivers and Demo Application are ready for review

I have managed to complete the reorganization of the code. Support is there for the above listed hardware. All objectives stated earlier are met. I did testing with multiple devices, but not thoroughly. Some portions of the code was just completed today.

Drop me a line if you are interested and I will give you access to the two repositories.

kisvegabor commented 4 years ago

@zladay Where are the repos?

zladay commented 4 years ago

I have invited you as a collaborator for the two repositories. You should receive emails with the details. Pls. confirm if you have access.

zladay commented 4 years ago

"New" ili9488 Driver

I have implemented a "new" driver (ili9488_32). This supports RGB8888 (32 bit) lvgl color model. TFT runs in RGB666 (RGB888 on the wire) mode. Conversion does not need buffer allocation. lvgl RGB8888 buffer is compacted in situ (the idea of @kisvegabor above). It works fine.

Comparison of 16 bit and 32 bit RGB scenarios

lvgl color depth 16 bit 32 bit
actual color depth on TFT 65k 262k
memory required / pixel in bytes 5 4

I think it is not particularly difficult to decide which one is better.

In addition to this, it is a huge advantage that a conversion buffer does not need to be allocated.

I added this "new" driver to the repository. The demo program is updated to use this driver. The other one (the plain ili9488) is kept. You can switch between the two in main.c. You may experiment if you can see the difference on the TFT.

zladay commented 4 years ago

@kisvegabor I do not know if anyone has already noticed the following. In lv_widgets -> Selectors -> Roller, there are "stray" pixels when rolling "Options". I do not think it is a driver problem, because everything else is OK.

embeddedt commented 4 years ago

@zladay I believe it's the same issue as https://github.com/lvgl/lvgl/issues/1602. Have you tried the latest versions of lvgl and lv_examples?

zladay commented 4 years ago

@embeddedt

lvgl @ 416ef9e lv_examples @ be98257

kisvegabor commented 4 years ago

@zladay Ah, it's a private repo. I got the invitation. Thanks. I will check it soon. @C47D would you like to check it as well?