espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.1k stars 7.15k forks source link

Enabling PSRAM with "Make RAM allocatable using malloc() as well" doesn't seem to use external memory (IDFGH-13095) #14041

Open Omegaki113r opened 1 month ago

Omegaki113r commented 1 month ago

Answers checklist.

IDF version.

v5.3-beta1-108-gae876915ec

Espressif SoC revision.

ESP32-S3 (QFN56) (revision v0.1)

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32-S3-WROOM-1

Power Supply used.

USB

What is the expected behavior?

I am developing a firmware that needs to use BLE, WIFI, Websocket client, websocket server, UART, CAN peripherals at the same time. Therefore i enabled PSRAM on the devkit i am using.

i have enabled following configs,

CONFIG_ESP32S3_SPIRAM_SUPPORT=y CONFIG_BT_ALLOCATION_FROM_SPIRAM_FIRST=y

CONFIG_SPIRAM=y CONFIG_SPIRAM_MODE_QUAD=y CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y CONFIG_SPIRAM_CLK_IO=30 CONFIG_SPIRAM_CS_IO=26 CONFIG_SPIRAM_FETCH_INSTRUCTIONS=y CONFIG_SPIRAM_RODATA=y CONFIG_SPIRAM_SPEED_40M=y CONFIG_SPIRAM_SPEED=40 CONFIG_SPIRAM_BOOT_INIT=y CONFIG_SPIRAM_USE_MALLOC=y CONFIG_SPIRAM_MEMTEST=y CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384 CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768 CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=y

So what my assumption was when i run my code it will use combination of both Internal and PSRAM. But i can only see my internal ram going down.

i am logging both internal and psram availability to terminal using,

printf("RAM INFORMATION %d %d\r\n", xPortGetFreeHeapSize(), heap_caps_get_free_size(MALLOC_CAP_SPIRAM));

What is the actual behavior?

eventhough i have enabled the settings related to asking BLE, WiFi and mbedtls to use external ram, almost all the allocations done by ESPIDF seems to be done on internal memory.

Steps to reproduce.

  1. Enable SPIRAM with following menu config settings

CONFIG_ESP32S3_SPIRAM_SUPPORT=y CONFIG_BT_ALLOCATION_FROM_SPIRAM_FIRST=y

CONFIG_SPIRAM=y CONFIG_SPIRAM_MODE_QUAD=y CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y CONFIG_SPIRAM_CLK_IO=30 CONFIG_SPIRAM_CS_IO=26 CONFIG_SPIRAM_FETCH_INSTRUCTIONS=y CONFIG_SPIRAM_RODATA=y CONFIG_SPIRAM_SPEED_40M=y CONFIG_SPIRAM_SPEED=40 CONFIG_SPIRAM_BOOT_INIT=y CONFIG_SPIRAM_USE_MALLOC=y CONFIG_SPIRAM_MEMTEST=y CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384 CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768 CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=y

  1. enable BLE and WiFi with combining ESPIDF examples
  2. Connect to a websocket server and enable websocket client after the WIFI is initialized
  3. Enable 2 UART peripherals ...

Debug Logs.

I (22) boot: ESP-IDF v5.3-beta1-108-gae876915ec 2nd stage bootloader
I (22) boot: compile time Jun 20 2024 22:37:53
I (23) boot: Multicore bootloader
I (27) boot: chip revision: v0.1
I (31) boot.esp32s3: Boot SPI Speed : 80MHz
I (36) boot.esp32s3: SPI Mode       : DIO
I (41) boot.esp32s3: SPI Flash Size : 8MB
I (45) boot: Enabling RNG early entropy source...
I (51) boot: Partition Table:
I (54) boot: ## Label            Usage          Type ST Offset   Length
I (62) boot:  0 nvs              WiFi data        01 02 00009000 00019000
I (69) boot:  1 otadata          OTA data         01 00 00022000 00002000
I (77) boot:  2 phy_init         RF data          01 01 00024000 00001000
I (84) boot:  3 ota_0            OTA app          00 10 00030000 0023f000
I (92) boot:  4 ota_1            OTA app          00 11 00270000 0023f000
I (99) boot:  5 storage          Unknown data     01 82 004af000 0007d000
I (107) boot: End of partition table
I (111) esp_image: segment 0: paddr=00030020 vaddr=3c190020 size=78400h (492544) map
I (208) esp_image: segment 1: paddr=000a8428 vaddr=3fc9f000 size=05744h ( 22340) load
I (212) esp_image: segment 2: paddr=000adb74 vaddr=40374000 size=024a4h (  9380) load
I (216) esp_image: segment 3: paddr=000b0020 vaddr=42000020 size=1847e4h (1591268) map
I (507) esp_image: segment 4: paddr=0023480c vaddr=403764a4 size=18a80h (100992) load
I (541) boot: Loaded app from partition at offset 0x30000
I (541) boot: Disabling RNG early entropy source...
I (552) cpu_start: Multicore app
I (553) esp_psram: Found 2MB PSRAM device
I (553) esp_psram: Speed: 40MHz
I (972) esp_psram: SPI SRAM memory test OK
I (984) cpu_start: Pro cpu start user code
I (984) cpu_start: cpu freq: 160000000 Hz
I (984) app_init: Application information:
I (987) app_init: Project name:     Demo
I (991) app_init: App version:      85be56b-dirty
I (997) app_init: Compile time:     Jun 20 2024 23:06:50
I (1003) app_init: ELF file SHA256:  c895638cf...
I (1008) app_init: ESP-IDF:          v5.3-beta1-108-gae876915ec
I (1015) efuse_init: Min chip rev:     v0.0
I (1020) efuse_init: Max chip rev:     v0.99 
I (1025) efuse_init: Chip rev:         v0.1
I (1030) heap_init: Initializing. RAM available for dynamic allocation:
I (1037) heap_init: At 3FCA9B90 len 0003FB80 (254 KiB): RAM
I (1043) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (1049) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (1056) heap_init: At 600FE100 len 00001EE8 (7 KiB): RTCRAM
I (1062) esp_psram: Adding pool of 2016K of PSRAM memory to heap allocator
I (1071) spi_flash: detected chip: generic
I (1074) spi_flash: flash io: dio
I (1126) sleep: Configure to isolate all GPIO pins in sleep state
I (1127) sleep: Enable automatic switching of GPIO sleep configuration
I (1129) coexist: coex firmware version: 66616e60c
I (1135) coexist: coexist rom version e7ae62f
I (1140) main_task: Started on CPU0
I (1150) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1150) main_task: Calling app_main()
1)RAM INFORMATION 206311  2062412
I (1190) SPIFFS: Initializing SPIFFS
I (1210) SPIFFS: Partition size: total: 463346, used: 19076
I (1220) app_main: DeviceID: e59fc75a
3)RAM INFORMATION 199775  2062216
I (1240) BLE_INIT: BT controller compile version [c23ab4c]
I (1240) BLE_INIT: Bluetooth MAC: 68:b6:b3:2b:53:b6
I (1240) phy_init: phy_version 670,b7bc9b9,Apr 30 2024,10:54:13
I (1320) gatts_stack_event_handler_task: ESP_GATTS_REG_EVT
I (1320) gatts_stack_event_handler_task: ESP_GATTS_CREATE_EVT
I (1320) gatts_stack_event_handler_task: ESP_GATTS_START_EVT
I (1330) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1330) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1340) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1340) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1350) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1360) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1370) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1370) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1380) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1390) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1390) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1400) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1410) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1410) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1420) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1430) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1430) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1440) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1450) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1450) gatts_stack_event_handler_task: ESP_GATTS_CREATE_EVT
I (1460) gatts_stack_event_handler_task: ESP_GATTS_START_EVT
I (1470) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1470) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1480) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1490) gatts_stack_event_handler_task: ESP_GATTS_REG_EVT
I (1490) gatts_stack_event_handler_task: ESP_GATTS_CREATE_EVT
I (1500) gatts_stack_event_handler_task: ESP_GATTS_START_EVT
I (1510) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1510) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1520) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1530) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1530) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1540) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_DESCR_EVT
I (1550) gatts_stack_event_handler_task: ESP_GATTS_REG_EVT
I (1550) gatts_stack_event_handler_task: ESP_GATTS_CREATE_EVT
I (1560) gatts_stack_event_handler_task: ESP_GATTS_START_EVT
I (1570) gatts_stack_event_handler_task: ESP_GATTS_ADD_CHAR_EVT
I (1580) OmegaLEController_start_advertising: 0x3c2110c4   09 09 4b 4d 53 42 6f 6f  73 74 00 00 00 00 00 00  |..KMSBoost......|
I (1580) OmegaLEController_start_advertising: 0x3c2110d4   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00     |...............|
I (1600) OmegaLEController_start_advertising: 0x3c2110a4   1b 21 51 49 df 63 c2 51  7d 96 d8 40 8f 16 a7 ab  |.!QI.c.Q}..@....|
I (1610) OmegaLEController_start_advertising: 0x3c2110b4   47 86 65 35 39 66 63 37  35 61 2c 32 00 00 00     |G.e59fc75a,2...|
I (1620) gap_stack_event_handler_task: ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT
I (1630) gap_stack_event_handler_task: ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT
I (1640) gap_stack_event_handler_task: ESP_GAP_BLE_ADV_START_COMPLETE_EVT
2)RAM INFORMATION 133479  2031884
I (1650) pp: pp rom version: e7ae62f
I (1650) net80211: net80211 rom version: e7ae62f
I (1670) wifi:wifi driver task: 3fcd314c, prio:23, stack:6656, core=0
I (1680) wifi:wifi firmware version: 6dbdc26
I (1680) wifi:wifi certification version: v7.0
I (1680) wifi:config NVS flash: enabled
I (1680) wifi:config nano formating: disabled
I (1680) wifi:Init data frame dynamic rx buffer num: 32
I (1690) wifi:Init static rx mgmt buffer num: 5
I (1690) wifi:Init management short buffer num: 32
I (1700) wifi:Init static tx buffer num: 16
I (1700) wifi:Init static tx FG buffer num: 2
I (1710) wifi:Init static rx buffer size: 1600
I (1710) wifi:Init static rx buffer num: 10
I (1710) wifi:Init dynamic rx buffer num: 32
I (1720) wifi_init: rx ba win: 6
I (1720) wifi_init: accept mbox: 6
I (1730) wifi_init: tcpip mbox: 32
I (1730) wifi_init: udp mbox: 6
I (1730) wifi_init: tcp mbox: 6
I (1740) wifi_init: tcp tx win: 5760
I (1740) wifi_init: tcp rx win: 5760
I (1750) wifi_init: tcp mss: 1440
I (1750) wifi_init: WiFi/LWIP prefer SPIRAM
I (1760) wifi_init: WiFi IRAM OP enabled
I (1760) wifi_init: WiFi RX IRAM OP enabled
I (1770) wifi:mode : sta (68:b6:b3:2b:53:b4)
I (1770) wifi:enable tsf
W (1770) wifi_stack_event_handler_task: UNHANDLED EVENT: 41
I (1780) wifi_stack_event_handler_task: LINE(196) [OMEGA_WIFI_LOG] WIFI_EVENT_STA_START
I (1790) sta_queue_handler_task: LINE(124) [OMEGA_WIFI_LOG] attempting to connect to ssid: xxxxx
I (1800) user_sta_event_handler_task: LINE(417) [OMEGA_WIFI_LOG] STA_PROVISIONING_EVT
4)RAM INFORMATION 48631  2023624
I (1800) wifi:flush txq
I (1810) wifi:stop sw txq
I (1810) wifi:lmac stop hw txq
I (1810) wifi_stack_event_handler_task: LINE(205) [OMEGA_WIFI_LOG] WIFI_EVENT_STA_STOP
I (1830) wifi:mode : sta (68:b6:b3:2b:53:b4)
I (1830) wifi:enable tsf
I (1830) wifi_stack_event_handler_task: LINE(196) [OMEGA_WIFI_LOG] WIFI_EVENT_STA_START
I (1840) ua.�I (1850) uart: queue free spaces: 5
I (1850) gpio: GPIO[17]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (1850) gpio: GPIO[18]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1860) TWAI Listen Only: Driver installed
I (1870) TWAI Listen Only: Driver started
5)RAM INFORMATION 44375  2013296

++++++++ AFTER ESP32S3 CONNECT TO STATION +++++++++++++++

6)RAM INFORMATION 35239  2012304
I (63520) esp_https_server: Starting server
I (63520) esp_https_server: Server listening on port 80
I (63530) [OMEGA_WSS_LOG]: Registering URI handlers
7)RAM INFORMATION 13667  2011640
I (63530) main_task: Returned from app_main()
I (63540) wifi:<ba-add>idx:1 (ifx:0, 04:20:84:43:78:b0), tid:6, ssn:2, winSize:64
I (65700) [OMEGA_WEBSOCKET_CLIENT_LOG]: WEBSOCKET_EVENT_CONNECTED
I (65700) [MAIN]: WEBSOCKET_EVENT_CONNECTED
I (66320) [MAIN]: WEBSOCKET_EVENT_DATA
I (66320) [MAIN]: Incoming data: {"topic":"DEVICE_INFO","payload":{"localIp":"192.168.1.2"}} Size: 59      
I (76760) [MAIN]: WEBSOCKET_EVENT_DATA
I (76760) [MAIN]: Incoming data:  Size: 0
E (76760) ws_client_callback: this is not valid json
I (87210) [MAIN]: WEBSOCKET_EVENT_DATA
I (87210) [MAIN]: Incoming data:  Size: 0

More Information.

No response

atanisoft commented 1 month ago

From your config:

CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384

Anything 16kb or smaller will always attempt internal allocation first. You can reduce this further to shift some allocations over or you can specifically call the heap APIs to allocate with caps for PSRAM.