espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
12.99k stars 7.12k forks source link

assertion "pbuf_free: p->ref > 0" failed: (IDFGH-5213) #6986

Closed iloop2020 closed 2 years ago

iloop2020 commented 3 years ago

Hi,

We see assertion at firmware/esp-idf/components/lwip/lwip/src/core/pbuf.c line 757, function: pbuf_free

Info: git commit 9778b163b174b1e4702cf354e9a26d98c511fd6c

I happen randomly , How can we fix it ?

Thank you.

igrr commented 3 years ago

Hi @iloop2020, this looks similar to https://github.com/espressif/esp-idf/issues/5423#issuecomment-644560431. What is the version of the toolchain that you are using? You can check it by running xtensa-esp32-elf-gcc --version. From the IDF commit ID you have provided, it seems that you might be using esp-2020r2 toolchain. If this is the case, then you will need to:

iloop2020 commented 3 years ago

Hi @igrr

Thank you for your support !!

I am using xtensa-esp32-elf-gcc (crosstool-NG esp-2019r2) 8.2.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Should I update it ? Thank you

igrr commented 3 years ago

Yes, I recommend following the instructions above to update ESP-IDF to v4.1.1 and the toolchain to esp-2020r3.

iloop2020 commented 3 years ago

But after we update to stable v4.1.1, xtensa-esp32-elf-gcc (crosstool-NG esp-2020r3) 8.4.0,

can we still compare idf at 9778b16 ?

iloop2020 commented 3 years ago

The reason we want to keep idf version, because

  1. v4.1.1 is a bit old ,
  2. but if we use to v4.2, there is error "ld: region `iram0_0_seg' overflowed by 1976 bytes".

Please help, thank you.

igrr commented 3 years ago

Do I understand correctly that you want to keep IDF version 9778b16, rather than v4.1.1? I'm not sure why you say that v4.1.1 is a bit old. It has been released on 29 January 2021. Whereas commit 9778b16 is dated 5 May 2020. So the commit you are using right now is actually older than v4.1.1.

iloop2020 commented 3 years ago

Yes, you are right, v.4.1.1 is newer.

And how can we fix "iram0_0_seg' overflowed " issue ?

Thank you.

igrr commented 3 years ago

Regarding IRAM overflow, could you please attach your project map file (found in the build directory) and your sdkconfig file? Most likely there are some options which can be changed in menuconfig to reduce IRAM usage.

iloop2020 commented 3 years ago

Hi @igrr

Thank you for your suggestion, and we change Compiler options -> Assertion level to Silent, then can bypass the IRAM overflow issue.

But the pub_free issue seems still there, and I cannot 100% sure because the crash log does not show line number .

Question: Would you please help to check which config increase the IRAM usage , between 9778b16 to v4.1.1, It is because I want to change back the Assertion level to Enable for debugging.

Thank you very much.

iloop2020 commented 3 years ago

Hi @igrr

Seems libbtdm_app.a consume a lot of IRAM(Is it the 3rd column, which is 27410 bytes ?

image

Thank you.

iloop2020 commented 3 years ago

Hi @igrr

Any good news ?

Thank you

igrr commented 3 years ago

@iloop2020 sorry for not giving you any feedback on the libbtdm_app issue. I'll have to pass the issue to my colleagues working on Bluetooth. @Alvin1Zhang please help follow.

wmy-espressif commented 3 years ago

Hi @iloop2020 , I looked into the difference made to the Bluetooth library file libbtdm_app.a between the two IDF version 9778b16 and v4.1.1, and found that the difference is resulted from a bunch of bug fixes. In fact, the year 2020 observes a good amount of bug fixes for Bluetooth Controller on ESP32. The use of IRAM is contributed by the patches for Bluetooth Controller ROM code. The use of IRAM increases gradually as more bugs are found and patches are applied.

I don't find an easy way to shrink these IRAM cost, we need to go through each updates in Bluetooth Controller whether the patches can be eliminated or there can be other ways of fixing the issues. This take a time, and according to my evaluation, not much memory can be saved. So can can you try reducing IRAM usage in other ways?

iloop2020 commented 3 years ago

Hi @wmy-espressif ,

Thank you for your support.

  1. We tried to reduce IRAM on different way, but no one work on my side, do you have any suggestion ?
  2. And we are facing another issue, on Bluetooth AVRCP feature, after we upgrade to v4.1.1, the AVRCP is not working with following error message image We confirmed 9778b16 works fine, without any changes on our code, we just rollback the IDF, it works again. Please help to fix it.

Thank you

wmy-espressif commented 3 years ago

Hi @iloop2020 , For Issue 2 about the AVRCP initialization, you need to change the initialization order of A2DP and AVRCP: AVRCP first and then A2DP. Inside Bluedroid, AVRCP is dependent on A2DP and cannot be used in separate from A2DP. When doing A2DP lower stack initialization, AVRCP requirement is checked.

For Issue 1, As far as I know there are some Wi-Fi Kconfig options that can be disabled to save some IRAM, e.g. CONFIG_ESP32_WIFI_IRAM_OPT, CONFIG_ESP32_WIFI_RX_IRAM_OPT, but will lead to some performance degradations. You can check their usage.

iloop2020 commented 3 years ago

Hi @wmy-espressif ,

  1. For AVRCP, we are using ADF's blueotooth_service library (v2.2), how can we re-order the A2DP adn AVRCP ?
  2. We already disable both CONFIG_ESP32_WIFI_IRAM_OPT and CONFIG_ESP32_WIFI_RX_IRAM_OPT, do you have another suggestion for us ?

Thank you.

wmy-espressif commented 3 years ago

@iloop2020 , For the initialization A2DP and AVRCP in ADF's bluetooth service library, I have to pass the issue to my collegues.

My responsibility concerntrates on Bluetooth Part, and for current stage the IRAM use in the patches of libbtdm_app.a is not able to be reduced. You need to try to save IRAM from other modules and I am not sure how much I can help. I'd like to ask what hardware or software modules are being used in your project?

iloop2020 commented 3 years ago

Hi @wmy-espressif ,

Do we have any update from your colleges ?

New question: How can we stop/disconnect or disable the blueooth if we want to switch to local SD playback ? Currently, it we are not able to stop the AD2P, so actually playing MP3 in SD card and receiving bluetooth streaming at the same time, the result is overload the ESP32.

Any suggestion ? Thank you,

wmy-espressif commented 3 years ago

Hi @iloop2020, to re-order the initialization of A2DP adn AVRCP in ADF's bluetooth service module, you have do modifiocations on the source file in ADF: components/bluetooth_service/bluetooth_service.c. My collegue doesn't tell their plan to fix the issue, so you may need to do the modification yourself if in a hurry.

For your new question, you can disconnect A2DP connection using the API esp_a2d_sink_disconnect

iloop2020 commented 3 years ago

Hi @wmy-espressif

Thank you for your suggestion !

For esp_a2d_sink_disconnect() API, if our application call them directly, will it confuse the ADF bluetooth_service ? And how can we resume the Bluetooth A2DP ?

Thank you

boblane1 commented 3 years ago

Hi @iloop2020,

In ADF bluetooth_service, there are periph_bluetooth_xxx APIs, which use AVRC commands to control the A2DP stream. You can learn about these APIs in bluetooth_service.h, and the example examples/player/pipeline_bt_sink show the demo of some APIs.

Thanks!

Alvin1Zhang commented 2 years ago

@iloop2020 Thanks for reporting, would you please help share if any updates for the issue? Thanks.

Alvin1Zhang commented 2 years ago

Thanks for reporting, will close due to short of feedback, feel free to reopen.