manuelbl / ttn-esp32

The Things Network device library for ESP32 (ESP-IDF) and SX127x based devices
MIT License
301 stars 64 forks source link

deep_sleep not saving data #68

Closed mdede439 closed 1 year ago

mdede439 commented 1 year ago

I was able to get deep_sleep running on my ESP32S3 but when I observe its operation I see that it doesn't seem to restore any of the RTC memory and has to do another join when it wakes. I also see that counter_in_rtc_mem does not increment, it stays at 1. To me it looks like it never saves any of the required data to RTC Memory. I see this line in main.cpp if (ttn.resumeAfterDeepSleep()) { printf("Resumed from deep sleep.\n"); } else {........

This if statement is never true so it never prints Resumed from deep sleep and never restores anything from RTC. Where ttn.resumeAfterDeepSleep() code? I cannot seem to find it.

manuelbl commented 1 year ago

Can you run the below program to check if RTC memory and NVS are working correctly?

#include <string.h>
#include "sdkconfig.h"
#include "freertos/FreeRTOS.h"
#include "nvs_flash.h"
#include "esp_sleep.h"
#include "esp_log.h"

#define TTN_RTC_FLAG_VALUE 0xf30b84ce
#define NVS_FLASH_PARTITION "ttn"
#define NVS_FLASH_KEY "key"
#define TTN_NVS_VALUE 1002
#define TAG "deepsleep_test"

void ttn_nvs_save();
bool ttn_nvs_restore();

RTC_DATA_ATTR int counter_in_rtc_mem;
RTC_DATA_ATTR uint32_t ttn_rtc_flag;
int nvs_value = 0;

void app_main(void)
{
    esp_err_t err = nvs_flash_init();
    ESP_ERROR_CHECK(err);

    if (ttn_rtc_flag != TTN_RTC_FLAG_VALUE) {
        printf("Woke up from reset\n");
        counter_in_rtc_mem = 0;
        ttn_rtc_flag = TTN_RTC_FLAG_VALUE;
        nvs_value = TTN_NVS_VALUE;
        ttn_nvs_save();
        printf("Saved value to NVS: %d\n", nvs_value);

    } else {
        counter_in_rtc_mem++;
        printf("Woke up from deep sleep: counter = %d\n", counter_in_rtc_mem);
        if (ttn_nvs_restore()) {
            printf("Successfully read value from NVS: %d\n", nvs_value);
        } else {
            printf("Failed to read value from NVS\n");
        }
    }

    esp_sleep_enable_timer_wakeup(10 * 1000000LL);

    printf("Going to deep sleep...\n");
    esp_deep_sleep_start();
}

void ttn_nvs_save()
{
    nvs_handle handle = 0;
    esp_err_t res = nvs_open(NVS_FLASH_PARTITION, NVS_READWRITE, &handle);
    if (res != ESP_OK)
        goto done;

    res = nvs_set_blob(handle, NVS_FLASH_KEY, &nvs_value, sizeof(nvs_value));
    if (res != ESP_OK)
        goto done;

    res = nvs_commit(handle);
    if (res != ESP_OK)
        goto done;

done:
    ESP_ERROR_CHECK(res);
    nvs_close(handle);
}

bool ttn_nvs_restore()
{
    nvs_handle handle = 0;
    esp_err_t res = nvs_open(NVS_FLASH_PARTITION, NVS_READWRITE, &handle);
    if (res != ESP_OK)
        goto done;

    size_t len = sizeof(nvs_value);
    res = nvs_get_blob(handle, NVS_FLASH_KEY, &nvs_value, &len);
    if (res != ESP_OK)
        goto done;

done:
    ESP_ERROR_CHECK(res);
    nvs_close(handle);

    return res == ESP_OK;
}
mdede439 commented 1 year ago

Hi, and thanks for the assistance. I am using a Linux UBUNTU environment.

"Which code are you running? The deep_sleep example? The C or the C++ version?"
 I am using the deep_sleep example in c++.  I am more familiar with the Arduino IDE and since it uses c++ I thought that would be better for me.  Also I have never used the ESP-IDF nor VSCODE so all this is very new to me and was a steep learning curve to even get the code flashed to my board. 

"What ESP-IDF version are you using?"
I am using IDF Version 5.1.0, the most recent version.

"What hardware are you using?"

I am using Adafruit hardware, the ESP32S3 (8MB of flash with no PSRAM product code 5323) and the LoRa Radio FeatherWing (product code 3231).

I am trying to use the code you supplied to test the RTC memory but I am having issues. I am new this environment so I may be making some errors, but I did get it to compile in c and flash the board, but once I hit reset to get the code running I get a message on the ESP-IDF Terminal of "Waiting for the device to reconnect....." I tried an ESP32S2 but get the same message. It does not seem to output any data to the terminal.

Mike

manuelbl commented 1 year ago

The problem with the output doesn't seem to be related to my code. It's a more general setup problem.

I had a look at the Adafruit Feather ESP32-S3 description and could figure out what is actually connected to the USB (where you likely expect the output). It doesn't look like there is a USB-to-serial converter chip on the board (like on mine). So the remaining options are that the USB port is either connected to the USB OTG port or the USB Serial/JTAG port.

My guess is you have no configure correctly where the log output is going. In menuconfig > Component config > ESP System Settings > Channel for console output, either try USB CDC (for the USB OTG port) or USB Serial/JTAG Controller.

And yes, my code is C code. I forgot to mention it.

mdede439 commented 1 year ago

OK, got it to work. I understand what you are saying about USB controller configuration with this device but if I select anything other than the default UART in menuconfig, the code does not compile. However on this board there is a DB pin. If I connect a USB to TTL Serial Cable - Debug / Console Cable, as in Adafruit product 954, to the DB pin and use a serial terminal program I can see the output from your code. Here is the output:

Woke up from deep sleep: counter = 3 Successfully read value from NVS: 1002 Going to deep sleep... ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x5 (DSLEEP),boot:0x8 (SPI_FAST_FLASH_BOOT) pro cpu reset by JTAG SPIWP:0xee mode:DIO, clock div:1 load:0x3fce3818,len:0x16e8 load:0x403c9700,len:0x4 load:0x403c9704,len:0xc00 load:0x403cc700,len:0x2eb0 SHA-256 comparison failed: Calculated: 83fc2cf1b8e1bad2afb91b53d7420cbc163d85cf8cf2c39c37c92d89264fa649 Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff Attempting to boot anyway... entry 0x403c9908 [0;32mI (47) boot: ESP-IDF v5.1-rc1-dirty 2nd stage bootloader[0m [0;32mI (47) boot: compile time Jun 10 2023 21:59:35[0m [0;32mI (47) boot: Multicore bootloader[0m [0;32mI (51) boot: chip revision: v0.1[0m [0;32mI (55) boot.esp32s3: Boot SPI Speed : 80MHz[0m [0;32mI (60) boot.esp32s3: SPI Mode : DIO[0m [0;32mI (64) boot.esp32s3: SPI Flash Size : 8MB[0m [0;32mI (69) boot: Enabling RNG early entropy source...[0m [0;32mI (74) boot: Partition Table:[0m [0;32mI (78) boot: ## Label Usage Type ST Offset Length[0m [0;32mI (85) boot: 0 nvs WiFi data 01 02 00009000 00006000[0m [0;32mI (93) boot: 1 phy_init RF data 01 01 0000f000 00001000[0m [0;32mI (100) boot: 2 factory factory app 00 00 00010000 00100000[0m [0;32mI (108) boot: End of partition table[0m [0;32mI (112) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=0a900h ( 43264) map[0m [0;32mI (125) esp_image: segment 1: paddr=0001a928 vaddr=3fc92b00 size=028e0h ( 10464) load[0m [0;32mI (131) esp_image: segment 2: paddr=0001d210 vaddr=40374000 size=02e08h ( 11784) load[0m [0;32mI (139) esp_image: segment 3: paddr=00020020 vaddr=42000020 size=1c9c0h (117184) map[0m [0;32mI (159) esp_image: segment 4: paddr=0003c9e8 vaddr=40376e08 size=0bcc8h ( 48328) load[0m [0;32mI (167) esp_image: segment 5: paddr=000486b8 vaddr=50000000 size=00008h ( 8) [0m [0;32mI (167) esp_image: segment 6: paddr=000486c8 vaddr=600fe000 size=00070h ( 112) [0m [0;32mI (179) boot: Loaded app from partition at offset 0x10000[0m [0;32mI (179) boot: Disabling RNG early entropy source...[0m [0;32mI (195) cpu_start: Multicore app[0m [0;32mI (196) cpu_start: Pro cpu up.[0m [0;32mI (196) cpu_start: Starting app cpu, entry point is 0x40375504[0m [0;32mI (0) cpu_start: App cpu up.[0m [0;32mI (214) cpu_start: Pro cpu start user code[0m [0;32mI (214) cpu_start: cpu freq: 160000000 Hz[0m [0;32mI (219) cpu_start: Application information:[0m [0;32mI (219) cpu_start: Project name: esp32s3_memtest[0m [0;32mI (223) cpu_start: App version: 1[0m [0;32mI (228) cpu_start: Compile time: Jun 10 2023 21:59:28[0m [0;32mI (234) cpu_start: ELF file SHA256: 6b255ed69ab9bb32...[0m [0;32mI (240) cpu_start: ESP-IDF: v5.1-rc1-dirty[0m [0;32mI (245) cpu_start: Min chip rev: v0.0[0m [0;32mI (250) cpu_start: Max chip rev: v0.99 [0m [0;32mI (255) cpu_start: Chip rev: v0.1[0m [0;32mI (259) heap_init: Initializing. RAM available for dynamic allocation:[0m [0;32mI (267) heap_init: At 3FC95C90 len 00053A80 (334 KiB): DRAM[0m [0;32mI (273) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM[0m [0;32mI (280) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM[0m [0;32mI (286) heap_init: At 600FE070 len 00001F90 (7 KiB): RTCRAM[0m [0;32mI (293) spi_flash: detected chip: gd[0m [0;32mI (296) spi_flash: flash io: dio[0m [0;32mI (300) sleep: Configure to isolate all GPIO pins in sleep state[0m [0;32mI (307) sleep: Enable automatic switching of GPIO sleep configuration[0m [0;32mI (314) app_start: Starting scheduler on CPU0[0m [0;32mI (319) app_start: Starting scheduler on CPU1[0m [0;32mI (319) main_task: Started on CPU0[0m [0;32mI (329) main_task: Calling app_main()[0m Woke up from deep sleep: counter = 4 Successfully read value from NVS: 1002 Going to deep sleep...

Looking at the output I see that counter is incrementing so it looks like it is saving to RTC memory successfully. and it also does seem to read from NVS.

I will try and use the deep_sleep_in_c code and see if that is any different than the c++ version.

Mike

mdede439 commented 1 year ago

I tried the deep_sleep_in_c and it also appears to not increment the counter_in_rtc_mem variable, it stays at 1.

Mike

mdede439 commented 1 year ago

I forgot to ask you what your hardware is. Maybe I should get your code working on the same hardware you are using. Did you develop this code with an ESP32? And if so what model? And which LoRa hardware? I have many versions so hopefully I already have what you used.

manuelbl commented 1 year ago

I've tested the code on basically all of the boards mentioned on Boards and Pins. Additionally, I've tested it with LilyGo ESP32-S2 connected to a RFM95 board. The latest ESP-IDF version I've used is 5.0.1. All my tests have been conducted with the European settings.

Based on the information in this issue and issue #67 I would propose:

Can you provide the full output of the deep_sleep sample across at least 2 sleep periods?

mdede439 commented 1 year ago

It is working now!!
I see what you mean about my version being "dirty". No idea how how that happened but when I went to install a clean version of the ESP-IDF extension I could only find 5.0.2 so that is what I used. I also modified your code to add a couple print statements for the counter_in_rtc_memory variable, one at the beginning of main and one right before it enters deep_sleep, just to monitor it's behaviour. Now I can see that it is incrementing and the code does not need to rejoin everytime it wakes up. Thanks! I am including the output from the DeBug pin as it does include more information than the USB port. Thanks again! Learning more and more!

deep_sleep_output.txt

manuelbl commented 1 year ago

It's difficult to say what the issue is in this case. But I note some suspicious error message:

On every boot it says:

SHA-256 comparison failed:
Calculated: 3dc8f48d6f8df6c5c146cc4734449bde9366d9a9e3a4c05639a29fc2a1ceaaa0
Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Attempting to boot anyway...

On starting with the third boot, it says:

W (344) ttn_prov: NVS storage is not initialized. Call 'nvs_flash_init()' first.
W (354) ttn: DevEUI, AppEUI/JoinEUI and/or AppKey have not been provided

I don't think you have remove nvs_flash_init() as it wouldn't have worked initially. But there seems to be a problem with NVS. The root cause could be either an invalid flash partitioning, a pin conflict (some pins are used to communicate with the flash memory, some are used to control where the chip boots from, some control flash parameters on ESP32-S2 and ESP32-S3).

What tools do you use to build and flash the code? The command line tools? The Visual Studio Code extension? PlatformIO?

mdede439 commented 1 year ago

I am running all this under Ubuntu 22.04.2 LTS with Visual Studio Code with the ESP-IDF extension. Currently, however, I am running a dirty version again. I was trying some code from another person and it was wriiten for a version 4 release of ESP-IDF so I tried changing mine to that version but the code would not work. I noticed that it was also a dirty version so I spent the last few days trying everything I can to install a clean ESP-IDF of any version but to no avail. My last attempt was a clean wipe of the OS and starting again, BUT now I had lost the deep_sleep code I had that worked and currently trying to recreate it. It does work but it rejoins every time it awakens from sleep. Not sure what is different except for my version of the IDF is dirty. Any ideas on how to get a clean version installed?

I will look into the GPIO pins and look into the NVS setting in the idf.py menuconfig.

Michael

manuelbl commented 1 year ago

I'm not sure if it helps. But erase the entire flash to reset NVS. With idf.py it's:

idf.py -p /dev/yourdevice erase-flash

After that, reupload the firmware.

mdede439 commented 1 year ago

Excellent! That did the trick.

I also do not see the ttn_prov: NVS storage is not initialized. Call 'nvs_flash_init()' first. message either. All I see is [0;32mI (354) ttn_prov: DevEUI, AppEUI/JoinEUI and AppKey saved in NVS storage[0m

It looks good! Now I just need to add some code to read some sensor data and transmit that instead of the test message.

I am still going to look at fixing the dirty part of my install.

mdede439 commented 1 year ago

FYI, I was able to fix this: SHA-256 comparison failed: Calculated: 3dc8f48d6f8df6c5c146cc4734449bde9366d9a9e3a4c05639a29fc2a1ceaaa0 Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff Attempting to boot anyway...

In the menuconfig I usually select 8MB of Flash AND also select "Detect size of flash size when flashing bootloader". I deselected this second option and only select the correct flash size for my module. Now I do not get that error anymore: