bertmelis / espMqttClient

MQTT 3.1.1 client library for the Espressif devices ESP8266 and ESP32 on the Arduino framework.
https://www.emelis.net/espMqttClient/
MIT License
100 stars 21 forks source link

PSRAM #27

Closed podaen closed 1 year ago

podaen commented 2 years ago

Interested to add support for PSRAM? I am glad to test...

esp32 wrover Arduino IDE 1.8.16 Arduino ESP 2.0.4

bertmelis commented 2 years ago

Which items would you put in psram?

I haven't worked with psram yet but I think it's very doable.

podaen commented 2 years ago

The only thing is that PSRAM can not be allocated in global variables. I did a quick look and got my eye on...

_txBuffer _rxBuffer

bertmelis commented 2 years ago

I'll have a look tomorrow.

Easiest is to have the outgoing packets in psram. Allocation for packets is done on only one line in the code. The queue itself likewise. I have to scan the code to see what else. Buffer might be possible indeed.

Now, I suppose you are fine with a compile time choice to put them in psram.

podaen commented 2 years ago

It could be that you have to make additional functions. First detect if there is a size for the PSRAM and then init it. You can choose an other size of buffer than the DRAM. My esp has a 4MB chip on board. So there is plenty of room.

I use this my shetch.

#include "esp_heap_caps.h

const int limit = 10000;
heap_caps_malloc_extmem_enable(limit);

For this I had to make sure the my objects are allocate with new after executed.

Pablo2048 commented 2 years ago

The only thing is that PSRAM can not be allocated in global variables. I did a quick look and got my eye on...

_txBuffer _rxBuffer

That is not correct - the PSRAM is unacessible for DMA so encryption functions (like those, used in SSL) are not working. There is option in SDK config to enable support for non-DMA operations in PSRAM region, but it's experimental IIRC and You need to rebuild the ARduino libraries - see https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/external-ram.html#restrictions

podaen commented 2 years ago

For the I2S, things can be initialize as zero DMA. For encryption you are right. I see they are improving there designs to handle more encryption, but nothing right now for that memory I think.

There is option in SDK config to enable support for non-DMA operations in PSRAM region, but it's experimental IIRC and You need to rebuild the ARduino libraries

I won't do that, you have to do a lot things to get it working and after that, if you start an other project you maybe have to set everything back to it original state.

bertmelis commented 1 year ago

I actually don't have any modules with PSRAM so I can't even test atm. I've ordered some though. But because of this, working with PSRAM is totally new for me and I don't know anything about the limitations.

Ideas and PRs welcome!

First link on a popular search engine: https://thingpulse.com/esp32-how-to-use-psram/ Is it that easy?

bertmelis commented 1 year ago

Ok, did some reading.

With the latest Arduino core, PSRAM is used automagically. If available, PSRAM is added to the heap on startup and malloc/free is used it as it needs. Hence, closing this issue.

If you want the have explicit PSRAM allocation, create a feature request.

podaen commented 1 year ago

Hi! It's a basic example, but there are a few things you could do more. Like determing the size automaticly...

    if(m_buffer) free(m_buffer);
    m_buffer = NULL;
    if(psramInit() && m_buffSizePSRAM > 0) {
        m_f_psram = true;
        m_buffSize = m_buffSizePSRAM;
        m_buffer = (uint8_t*) ps_calloc(m_buffSize, sizeof(uint8_t));
        m_buffSize = m_buffSizePSRAM - m_resBuffSizePSRAM;
    }
    if(m_buffer == NULL) {
        m_f_psram = false;
        m_buffSize = m_buffSizeRAM;
        m_buffer = (uint8_t*) calloc(m_buffSize, sizeof(uint8_t));
        m_buffSize = m_buffSizeRAM - m_resBuffSizeRAM;
    }

After that you can choose what you want to to do with it.

bertmelis commented 1 year ago

Sorry, I don't understand what you're trying to say. This lib allocates most memory as 'global' except for data in the outgoing queue. These outgoing packets are allocated on the heap with a malloc call and as I understand are allocated in PSRAM when memory usage requires.

podaen commented 1 year ago

I think I saw in the config file that the maximum package size could be set... Unfortualy PSRAM can not be allocated in global variables. But in the member list, it works fine.

I understand are allocated in PSRAM when memory usage requires.

That's exactly what this code is!

bertmelis commented 1 year ago

I think I saw in the config file that the maximum package size could be set...

Good point. You can set the minimum free memory you need to have to allocate memory for a packet. This however does not take psram into account. I'll take care of that.