Porting to ESP-IDF with C++ support

[v1993]

Hello, I'm interested in porting this great library to ESP32 without Arduino compatiblity layer. However, for me it seems that there is a problem with cppsrc/U8x8lib.cpp: it use arduino specific stuff.

Is it possible to add support for other platform in it or it isn't required for C-side calls implementations and headers will be enough (i. e. not build this file at all)?

Is it doing only low-level stuff which I can do in any plain C file, or some things which are reqired to get C++ working too?

[olikraus]

U8g2 itself is a pure C library. For Arduino however it calls the C++ Arduino HAL and adds a C++ lib API wrapper. Both, Arduino wrapper and HAL are put together in the same c++/h files. As a consequence: If you want the C++ U8g2 API but not the C++ Arduino HAL, then some code needs to be copied out the existing C++ code.

I am not 100%, but I think this has been done by the nodemcu people already: https://nodemcu.readthedocs.io/en/dev-esp32/modules/u8g2/

In general porting instructions are here: https://github.com/olikraus/u8g2/wiki/Porting-to-new-MCU-platform

[v1993]

Thank you. Now I think about architecture: is it good idea to place ESP32 specific stuff in sys/esp-idf? It will be included from CMakeLists.txt or component.mk. I also plan adding config (for ESP32 menuconfig menu) for 16bit mode and fonts to make u8g2 feel like nice component.

Also, where can I find pure C exmples? I feel like there is a lack of them.

[olikraus]

Now I think about architecture: is it good idea to place ESP32 specific stuff in sys/esp-idf? It will be included from CMakeLists.txt or component.mk

I have no idea about this. CMakeLists.txt or component.mk in the u8g2 are part of a pull request. I can not tell you something about these files.

Also, where can I find pure C exmples?

Have a look here: https://github.com/olikraus/u8g2/tree/master/sys All directories except "arduino" will contain pure c examples.

Here is a "hello world" example for the STM32L031: https://github.com/olikraus/u8g2/blob/master/sys/arm/stm32l031x6/u8x8_test/main.c

Once I created a blog entry also: https://drolliblog.wordpress.com/2017/11/02/stm32l031x6-eval-board/

I feel like there is a lack of them.

I personally feel, that I created more docs and examples than many other Arduino libs...

[v1993]

Thanks you again and sorry for blaming you about no docs. Your Arduino/C++ reference is just great! I just had some problems about pure C documentation.

[olikraus]

:-)

Let me know if I can help you with u8g2 related problems.

[v1993]

Now I think about architecture: is it good idea to place ESP32 specific stuff in sys/esp-idf? It will be included from CMakeLists.txt or component.mk

I have no idea about this. CMakeLists.txt or component.mk in the u8g2 are part of a pull request. I can not tell you something about these files.

There are no problems with changing those files, just wonder is sys/platformname a right place to store platform-specific stuff (I/O frunctions for ESP32 in this case).

[olikraus]

There are no problems with changing those files, just wonder is sys/platformname a right place to store platform-specific stuff (I/O frunctions for ESP32 in this case).

You mean below u8g2? Well sure, it you like to do so and send a corresponding pull request then I can accept this.

[v1993]

Thank you! Another question: how presice should be all the U8X8_MSG_DELAY_* stuff should be at waiting?

There are three options: use vTaskDelay (round time to ticks, pretty rough), ets_delay_us for busywait correct time (much better at timing, but not let anything else run while waiting), or ets_delay_us with disabeling interrupts with portDISABLE_INTERRUPTS/portENABLE_INTERRUPTS (100% correct timing, but extremely bad at long and not that presice waitings due to not be able to handle interrupts).

I suggest using vTaskDelay on ≥ 10ms delays and ets_delay_us on smaller scale. Are there any parts which should be more accurate at timing (so disable interrupts)? If so, it's still better to disable them in actual calling code and not callback.

Thanks.

[v1993]

Ok, I took a look at files.

/* generated code (codebuild), u8g2 project */

Where code generator and template files are? It looks like I'll have to add ESP32 support there.

My current idea it to leave csrc as is and place EPS32 stuff in some other directory, like sys/esp-idf (and auto generated stuff there too).

[olikraus]

Another question: how presice should be all the U8X8_MSGDELAY* stuff should be at waiting?

It always can be more than requested, so: It does not need to be precise.

Are there any parts which should be more accurate at timing (so disable interrupts)?

I think the only sensitive message is the I2C delay. But this is only required for software emulated I2C.

Where code generator and template files are?

You probably do not need to take care on the code generator. "codebuild" only connects the displays with the communication interfaces: It defines which types of interfaces are available for which display.

The hardware specific files are either in the sys//common directory or (an exception) for Arduino in the u8g2/cppsrc directory.

My current idea it to leave csrc as is

Yes please. "csrc" should not contain any hardware specific code.

[v1993]

Are Arduino display C++ binding auto generated? If yes, because I want to support them on ESP-IDF too, it will require tweaking code generator.

Am I right?

My goal is to provide C++ bindings compatible with Arduino interface too, because C++ is commonly used on ESP32 too.

[olikraus]

Are Arduino display C++ binding auto generated?

If you mean the constructor classes here, then yes: https://github.com/olikraus/u8g2/blob/master/cppsrc/U8g2lib.h#L404

[v1993]

I also want to auto generate bindings for ESP-IDF to make migration/usage as comfortable as possible. Therefore, I have to tweak generator, right?

[olikraus]

Or you create a copy of the codebuild Code and add your modifications there.

[v1993]

To be fair, current code generator looks like hell for me ☺.

I'll look at it when get pure C working.

Do you have anything againist re-writing in in something non-C? I'm good (enough to write such thing) with C++ and lua.

[v1993]

Can you take a look an my test implementation for gpio_and_delay, please?

uint8_t u8x8_gpio_and_delay_espidf(u8x8_t *u8x8, uint8_t msg, uint8_t arg_int, U8X8_UNUSED void *arg_ptr)
{
    uint8_t i;
    switch(msg)
    {
    case U8X8_MSG_GPIO_AND_DELAY_INIT:
        for( i = 0; i < U8X8_PIN_CNT; i++ )
            if ( u8x8->pins[i] != U8X8_PIN_NONE )
            {
            bool success;
                if ( i < U8X8_PIN_OUTPUT_CNT )
                {
                    success = u8x8_setpinoutput_espidf(u8x8->pins[i]);
                }
                else
                {
                    success = u8x8_setpininput_espidf(u8x8->pins[i]);
                }
                if (!success) return 0;
            }

        break;

    case U8X8_MSG_DELAY_NANO:
        ets_delay_us(arg_int==0?0:1);
        break;

    case U8X8_MSG_DELAY_10MICRO:
        /* not used at the moment */
        break;

    case U8X8_MSG_DELAY_100NANO:
        /* not used at the moment */
        break;

    case U8X8_MSG_DELAY_MILLI:
        vTaskDelay(arg_int);
        break;
    case U8X8_MSG_DELAY_I2C:
        /* arg_int is 1 or 4: 100KHz (5us) or 400KHz (1.25us) */
        ets_delay_us(arg_int<=2?5:2);
        break;
    case U8X8_MSG_GPIO_I2C_CLOCK:
    case U8X8_MSG_GPIO_I2C_DATA:
        auto pin = u8x8_GetPinValue(u8x8, msg);
        if ( arg_int == 0 )
        {
            if (
            (!u8x8_setpinoutput_espidf(pin)) ||
            (gpio_set_level(pin, 0) != ESP_OK)
            ) return 0;
        }
        else
        {
            if (!u8x8_setpininput_espidf(pin)) return 0;
        }
        break;
    default:
        if ( msg >= U8X8_MSG_GPIO(0) )
        {
            i = u8x8_GetPinValue(u8x8, msg);
            if ( i != U8X8_PIN_NONE )
            {
                if ( u8x8_GetPinIndex(u8x8, msg) < U8X8_PIN_OUTPUT_CNT )
                {
                    gpio_set_level(i, arg_int);
                }
                else
                {
                    u8x8_SetGPIOResult(u8x8, gpio_get_level(i));
                }
            }
            break;
        }

        return 0;
    }
    return 1;
}

Are there any visible flaws? Thanks.

[olikraus]

To be fair, current code generator looks like hell for me

then, it is better to work on a copy... ;-)

Do you have anything againist re-writing in in something non-C?

I do not care if you work on your own repo ;-) For pull requests: New files in new directories should be ok. For the existing code I prefer C over C++

[v1993]

Hmm, seems there is a problem…

Arduino introduce global SPI state, but ESP-IDF does not. Doing so would mean big problems with more than displays running in parallel. It would be really nice to have non-interputtable version where SPI state don't have to be stored between calls. Any ideas?

Possible solution is to add SPI and I2C states to u8x8_t struct on ESP32. How bad is it? In my opinion, that's ok, but you're main developer after all…

[v1993]

BTW, do you have an IRC/Jabber? It probably would be faster to communicate there (nothing personal, feel free to reject with or without reason).

[v1993]

Allright, got it! There IS global state on hardware level (SPI1 and SPI2), so I make one "universal" function which can work with any SPI descriptor and call it from SPI(1/2) handlers.

[v1993]

Well, just got that I was doing HW SPI all wrong… It's better try to get ESP-IDF driver working and NOT try to adapt one from arduino-esp32.

I have some questions now:

1. How big data transfers usually are? There are huge optimisations for < 32 byte ones.
2. Do all SPI stuff have to be syncronized?
3. Do U8G2 read anything? I'm pretty sure it isn't, just want to be 100% clear about it.

[olikraus]

Possible solution is to add SPI and I2C states to u8x8_t struct on ESP32. How bad is it?

Yes, it will break the independence from SPI and I2C

BTW, do you have an IRC/Jabber?

No

How big data transfers usually are? There are huge optimisations for < 32 byte ones.

I2C transfers are limited to 32 bytes. Any other transfers can have any length.

Do all SPI stuff have to be syncronized?

Against what?

Do U8G2 read anything? I'm pretty sure it isn't, just want to be 100% clear about it.

U8g2 does not read from displays (but from GPIOs)

Just a general notes: I do not have much time to work on U8G2. So I can not do more than give comments on your questions. The core part of u8g2 has to be C and it must not depend on any low level, system specific code... at least if you want me to accept pull requests.

[v1993]

Ok, thank you.

Do all SPI stuff have to be syncronized?

Against what?

In this case syncronized means no caching in driver, i.e. use polling (write means waiting until transfer is complete) and not interrupt (things goes into queue and are done in background) SPI mode (ESP-IDF specific stuff).

P.S.: I was talking about HW SPI transfers, or to be presice arg_int for U8X8_MSG_BYTE_SEND message. Sorry for not being clear.

[v1993]

Also, if adding stuff to u8x8_t is bad, can I use U8X8_WITH_USER_PTR and store my stuff by that pointer?

[v1993]

Hi. My porting is going ok yet. There is a question: can I be sure that memory pointed by arg_ptr is HW SPI callback is on stack? Thanks.

[olikraus]

Also, if adding stuff to u8x8_t is bad, can I use U8X8_WITH_USER_PTR and store my stuff by that pointer?

Yes, sure, that is the purpose of the user ptr

There is a question: can I be sure that memory pointed by arg_ptr is HW SPI callback is on stack?

probably not.

[v1993]

Interesting thing: it's possible to use two displays with all same pins except CS (and DC I think) which is handled by driver (even from two cores)!

However, I want to know one thing: is it possible to trace destruction of u8x8? I know that it's rare thing, but it would be nice to delete device in such a case.

Thanks.

[v1993]

And another (more actual) question: can I make few SPI transactions instead of one?

[olikraus]

Interesting thing: it's possible to use two displays with all same pins except CS (and DC I think) which is handled by driver (even from two cores)!

Yes, sure, just create two different u8x8 / u8g2 structures (classes).

However, I want to know one thing: is it possible to trace destruction of u8x8?

What do you want to trace? U8g2/U8x8 do not use dynamic memory.

I know that it's rare thing, but it would be nice to delete device in such a case.

There are many nice features. All of them will cost resources. And people already complain about u8g2 size. Nevertheless, i think this is very much possible.

[v1993]

At first, what about my last question?

Arduino have SPI API like this:

• Begin transaction
• Write byes
• End transaction

ESP-IDF, on other hand, does it in different way:

1. • Add transaction (complete) into queue
   • Request results later
2. • Do synchronous transaction (complete memory required again)

Under complete memory I mean that only bytes available at moment of beginning transaction will be sent in it.

So, is it ok to perform complete transaction on every "write" and ignore transaction begin/end (CS managed by ESP-IDF SPI driver itself)?

[olikraus]

So, is it ok to perform complete transaction on every "write" and ignore transaction begin/end (CS managed by ESP-IDF SPI driver itself)?

Unknown. It probably will depend on the display controller. It is like this: The display controller will receive commands from u8g2. A command sometimes include a command byte and an argument bytes, so it may consist of two (or more) bytes. A controller may allow the reception of these two bytes in two different transactions, but this is not 100% clear to me. You need to study the graphics controller datasheets for this. In u8g2 it is ensured, that command and args are always sent in the same transaction (the CS line does not change between command byte and argument byte of the same full command).

[v1993]

And about "destructor":

U8g2 itself don't use dynamic memory, but some implementations (like mine) may need to or require some hardware descriptors. Possible implemetation is quite simple:

• Add U8X8_MSG_GPIO_AND_DELAY_CLEAN and pass it to main callback.
• Let C users call u8x8_destroy() or something like that.
• For C++, call it from actual destructor.

Souldn't be hard or breaking, right?

A command sometimes include a command byte and an argument bytes, so it may consist of two (or more) bytes.

Hmm, seems pretty bad. Is it accepable to just send them as usually or will I have to use these fields? That would be complicated.

Another solution is to allocate buffer for SPI stuff and do actual transaction on "end transaction".

You need to study the graphics controller datasheets for this.

I'm not ready to do so on all displays supported by u8g2…

Thanks.

[v1993]

And another question: what is max size of SPI transaction? I have to set in bus configuration.

Can I rely on default 4094 bytes?

[olikraus]

Can I rely on default 4094 bytes?

It will probably much shorter. For I2C it is limited to 32 for SPI I assume it will not exceed 512 bytes. The only exception might be e-paper devices, which might require larger blocks.

[v1993]

Ok, thanks. I'll add into config then (with 512 as default).

[v1993]

Should CS line be positive or negative during communication?

Default driver's behaviour is to set it ti negative, but there is a flag SPI_DEVICE_POSITIVE_CS. Should I use it?

P.S.: what about destructor idea? Thanks.

[v1993]

And yet another question: can SPI mode change for device?

[olikraus]

Should CS line be positive or negative during communication?

Depends on the display: https://github.com/olikraus/u8g2/blob/master/csrc/u8x8.h#L224

P.S.: what about destructor idea? Thanks.

U8g2 is fine with the C++ default constructor, because there is no further dynamic memory allocation.

And yet another question: can SPI mode change for device?

Yes of course.

[v1993]

And yet another question: can SPI mode change for device?

Yes of course.

I mean, for one device between different transactions? If still yes, that's pretty bad, because it mean that I have to remove and add device again, reducing speed a lot.

P.S.: what about destructor idea? Thanks.

U8g2 is fine with the C++ default constructor, because there is no further dynamic memory allocation.

As I already said, implementation may need to do internal cleanup. I have to malloc things to store custom stuff.

Should CS line be positive or negative during communication?

Depends on the display: /csrc/u8x8.h@master#L224

If chip_enable_level == 1, should it be negative or positive during activity?

[v1993]

Can say one thing for sure: SW I2C for SSD1306 from pure C is working just fine!

@olikraus Please, help me with remaining questions. Thank you for all help!

[olikraus]

And yet another question: can SPI mode change for device?

Yes of course. I mean, for one device between different transactions? If still yes, that's pretty bad, because it mean > that I have to remove and add device again, reducing speed a lot.

no

P.S.: what about destructor idea? Thanks.

U8g2 is fine with the C++ default constructor, because there is no further dynamic memory allocation. As I already said, implementation may need to do internal cleanup. I have to malloc things to store > custom stuff.

I assume you will create a new interface, so of course, you should provide a proper constructor.

Should CS line be positive or negative during communication?

Depends on the display: /csrc/u8x8.h@master#L224 If chip_enable_level == 1, should it be negative or positive during activity?

chip_enable_level is not a variable, it is a calibration value which returns the level. If your transfer is started, you should set the chip select level like this:

  case U8X8_MSG_BYTE_START_TRANSFER:
      u8x8_gpio_SetCS(u8x8, u8x8->display_info->chip_enable_level);  

Can say one thing for sure: SW I2C for SSD1306 from pure C is working just fine!

nice

[v1993]

I'm having a strange problems when try to get HW I2C working.

Display correctly responds with ACK on data and adress, but nothing works. If I try rebooting from SW to HW implementation (without changing screen), some black lines appears, like this:

Can you guess any reason? Thanks.

[olikraus]

Difficult to say. Speed might be to high. Or the picture just reflects the content from the last correct update with some garbage from the restart.

[v1993]

At first, I started SW I2C version which draw the picture. Then flased HW I2C version, then rebooted few times after finishing (more lines after each restart).

Display changes only when actual sending occours (I use logging to PC terminal).

I tried changing speed, but can't get working display this way.

Tried speeds:

• 400000Hz (from U8G2)
• 100000Hz
• 10000Hz
• 1000Hz

Any other suggestions?

[olikraus]

Your low level code might be wrong

[v1993]

Hmm, I have a theory: probably i2c_master_write(u8x8->user_ptr, arg_ptr, arg_int, true); don't copy memory but only store pointer? In this case it may point to invalid memory at time of transaction.

Only a theory…

[olikraus]

user_ptr just is a pointer value, which you provide. It does not copy anything (not sure if this helps)

[v1993]

Yep, I know. I store transaction pointer there.

Just checked source code of that i2c_master_write, looks like it doesn't copy memory. So I really probably am right…

Is there any limit on how many writes may go in one I2C transaction?

[v1993]

Got HW I2C working :smile:

I copy incoming data into dynamically mallocated memory and then free it after sending. It have pretty big perfomance impact.

Are I2C transactions limited to some size? Not "write" calls to I2C callback, but I2C transactions themselves. This way it would be possible to allocate buffer only once.

Also, how many times per transaction "write" callback can be called?

Thank you.

[olikraus]

Are I2C transactions limited to some size? Not "write" calls to I2C callback, but I2C transactions themselves. This way it would be possible to allocate buffer only once.

The buffer size in Arduino is 32 bytes for I2C.

Also, how many times per transaction "write" callback can be called?

hmmm up to 32