Lost data on power failure despite calling fflush

[Blueknife-w]

I recently ported esp_littlefs to run on esp32, and tried to refer to Easylogger for a program based on file storage logs to record device log information. During the process, I encountered the following problems:

After opening the file, every time a log is written, fflush is called to refresh the cache, the file pointer moves, but the file size does not change; and after the device is powered off, the file is still rewritten from the position when it was opened last time data. However, I continue to test and call close to close the file immediately after writing a log, and there is no such problem.

I am not very familiar with the operation of the file system API, so I am not sure whether this problem is my operation problem or the problem of esp_littlefs or littlefs, hoping to be corrected.

log write function

void elog_file_write(const char *log, size_t size)
{
    size_t file_size = 0;

    ELOG_ASSERT(init_ok);
    ELOG_ASSERT(log);
    if(fp == NULL) {
        return;
    }

    elog_file_port_lock();

    fseek(fp, 0L, SEEK_END);
    file_size = ftell(fp);

    printf("offset:%d\n", file_size);

    if (unlikely(file_size > local_cfg.max_size)) {
#if ELOG_FILE_MAX_ROTATE > 0
        if (!elog_file_rotate()) {
            goto __exit;
        }
#else
        goto __exit;
#endif
    }

    fwrite(log, size, 1, fp);

#ifdef ELOG_FILE_FLUSH_CACHE_ENABLE
    fflush(fp);
#endif

__exit:
    elog_file_port_unlock();
}

Traverse directory file information function

int list_dir_file(void)
{
    DIR *dir;
    struct dirent *ent;
    struct stat statbuf = {0};

    FILE *fp = NULL;
    long int size;
    char buf[100] = {0};

    dir = opendir("/littlefs");
    if (dir != NULL) {
        while ((ent = readdir(dir)) != NULL) {
            ESP_LOGI(TAG, "%s", ent->d_name);
            stat(ent->d_name, &statbuf);

            // ESP_LOGW(TAG, "name:%s size:%d", ent->d_name, statbuf.st_size);

            memset(buf, 0, 100);
            snprintf(buf, 100, "/littlefs/%s", ent->d_name);
            //ESP_LOGI(TAG, "%s", buf);

            fp = fopen(buf, "r");
            if (fp == NULL) {
                ESP_LOGE(TAG, "fopen");
                continue;
            }
            fseek(fp, 0, SEEK_END);
            size = ftell(fp);
            fclose(fp);

            ESP_LOGI(TAG, "name: %s\tsize:%ld %d", ent->d_name, size, statbuf.st_size);
        }
        closedir(dir);
    } else {
        ESP_LOGI(TAG, "Failed to open directory.\n");
    }
    return 0;
}

Do the following in the main loop

    while (1) {
        list_dir_file();
        elog_file_write("abcabcabc", sizeof("abcabcabc"));
        vTaskDelay(500);
// The function of the following masked code is to reopen the file after closing the file
// elog_file_deinit();
// elog_file_init();
    }

The running log is as follows，and you can see the file pointer is offset, but file size isn't change.

Please input password:W (1784) main: esp_get_free_heap_size : 173 k

I (2284) cmd: software_version: 2.5.1
I (2285) cmd: hardware_version: 1.0
I (2291) lettlefs_cfg: elog_file.log
I (2307) lettlefs_cfg: name: elog_file.log  size:320 0
I (2310) lettlefs_cfg: foo.txt
I (2327) lettlefs_cfg: name: foo.txt    size:16 0
offset:320
I (3333) lettlefs_cfg: elog_file.log
I (3350) lettlefs_cfg: name: elog_file.log  size:320 0
I (3352) lettlefs_cfg: foo.txt
I (3370) lettlefs_cfg: name: foo.txt    size:16 0
offset:330
I (4199) wifi_conn: wifi disconnect! reconnect...
I (4376) lettlefs_cfg: elog_file.log
I (4393) lettlefs_cfg: name: elog_file.log  size:320 0
I (4396) lettlefs_cfg: foo.txt
I (4413) lettlefs_cfg: name: foo.txt    size:16 0
offset:340
I (5419) lettlefs_cfg: elog_file.log
I (5436) lettlefs_cfg: name: elog_file.log  size:320 0
I (5439) lettlefs_cfg: foo.txt
I (5458) lettlefs_cfg: name: foo.txt    size:16 0
offset:350
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x16 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6992
load:0x40078000,len:15532
load:0x40080400,len:3840

Reset device, the log is as follows, the file is written again from offset 320!

I (1734) wifi:enable Please input password:W (1740) main: esp_get_free_heap_size : 173 k

I (2240) cmd: software_version: 2.5.1
I (2241) cmd: hardware_version: 1.0
I (2248) lettlefs_cfg: elog_file.log
I (2267) lettlefs_cfg: name: elog_file.log  size:320 0
I (2271) lettlefs_cfg: foo.txt
I (2293) lettlefs_cfg: name: foo.txt    size:16 0
offset:320
I (3092) control: Run warning, lackwater: 1 sewage:0 motor abnormal:0 sn flag:0 tend:0
I (3300) lettlefs_cfg: elog_file.log
I (3319) lettlefs_cfg: name: elog_file.log  size:320 0
I (3323) lettlefs_cfg: foo.txt
I (3345) lettlefs_cfg: name: foo.txt    size:16 0
offset:330
I (4155) wifi_conn: wifi disconnect! reconnect...
I (4352) lettlefs_cfg: elog_file.log
I (4372) lettlefs_cfg: name: elog_file.log  size:320 0
I (4375) lettlefs_cfg: foo.txt
I (4400) lettlefs_cfg: name: foo.txt    size:16 0
offset:340
I (5407) lettlefs_cfg: elog_file.log
I (5427) lettlefs_cfg: name: elog_file.log  size:320 0
I (5431) lettlefs_cfg: foo.txt
I (5453) lettlefs_cfg: name: foo.txt    size:16 0

[Blueknife-w]

Note, the subtype I used for the littlefs partition is FAT, not SPIFFS, this should not affect it

[Blueknife-w]

I solved this problem using the following two methods:

1. use fsync command after fflush
2. Enable Flush file to flash after each write operation in menucofnig.