unable to create and flash littlefs image at compile time

[narangmayank]

Problem

I was trying to create the littlefs partition image at compile time using littlefs_create_partition_image(littlefs config_data) but couldn't make it to work. can someone please help me out?

Code Snippet

void app_main(void)
{
        printf("Demo LittleFs implementation by esp_littlefs!\n");
        printf("   https://github.com/joltwallet/esp_littlefs\n");

        /* Print chip information */
        esp_chip_info_t chip_info;
        esp_chip_info(&chip_info);
        printf("This is %s chip with %d CPU cores, WiFi%s%s, ",
               CONFIG_IDF_TARGET,
               chip_info.cores,
               (chip_info.features & CHIP_FEATURE_BT) ? "/BT" : "",
               (chip_info.features & CHIP_FEATURE_BLE) ? "/BLE" : "");

        printf("silicon revision %d, ", chip_info.revision);

        uint32_t size_flash_chip = 0;
        esp_flash_get_size(NULL, &size_flash_chip);
        printf("%uMB %s flash\n", (unsigned int)size_flash_chip >> 20,
               (chip_info.features & CHIP_FEATURE_EMB_FLASH) ? "embedded" : "external");

        printf("Free heap: %u\n", (unsigned int) esp_get_free_heap_size());

        printf("Now we are starting the LittleFs Demo ...\n");

        ESP_LOGI(TAG, "Initializing LittleFS");

        esp_vfs_littlefs_conf_t conf = {
            .base_path = "/littlefs",
            .partition_label = "littlefs",
            .format_if_mount_failed = true,
            .dont_mount = false,
        };

        // Use settings defined above to initialize and mount LittleFS filesystem.
        // Note: esp_vfs_littlefs_register is an all-in-one convenience function.
        esp_err_t ret = esp_vfs_littlefs_register(&conf);

        if (ret != ESP_OK)
        {
                if (ret == ESP_FAIL)
                {
                        ESP_LOGE(TAG, "Failed to mount or format filesystem");
                }
                else if (ret == ESP_ERR_NOT_FOUND)
                {
                        ESP_LOGE(TAG, "Failed to find LittleFS partition");
                }
                else
                {
                        ESP_LOGE(TAG, "Failed to initialize LittleFS (%s)", esp_err_to_name(ret));
                }
                return;
        }

        size_t total = 0, used = 0;
        ret = esp_littlefs_info(conf.partition_label, &total, &used);
        if (ret != ESP_OK)
        {
                ESP_LOGE(TAG, "Failed to get LittleFS partition information (%s)", esp_err_to_name(ret));
        }
        else
        {
                ESP_LOGI(TAG, "Partition size: total: %d, used: %d", total, used);
        }

        // Open file for reading
        ESP_LOGI(TAG, "Reading file");
        FILE* f = fopen("/littlefs/hello.txt", "r");
        if (f == NULL)
        {
                ESP_LOGE(TAG, "Failed to open file for reading");
                return;
        }
        char line[64];
        fgets(line, sizeof(line), f);
        fclose(f);
        // strip newline
        char *pos = strchr(line, '\n');
        if (pos)
        {
                *pos = '\0';
        }
        ESP_LOGI(TAG, "Read from file: '%s'", line);

        // All done, unmount partition and disable LittleFS
        esp_vfs_littlefs_unregister(conf.partition_label);
        ESP_LOGI(TAG, "LittleFS unmounted");
}

Serial Logs

Demo LittleFs implementation by esp_littlefs!
   https://github.com/joltwallet/esp_littlefs
This is esp32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 100, 4MB external flash
Free heap: 295852
Now we are starting the LittleFs Demo ...
I (396) demo_esp_littlefs: Initializing LittleFS
I (406) demo_esp_littlefs: Partition size: total: 524288, used: 8192
I (406) demo_esp_littlefs: Reading file
E (416) demo_esp_littlefs: Failed to open file for reading

Thanks in Advance.

[BrianPugh]

I just published v1.5.4, which has several fixes involving mklittlefs, especially for windows users (if you're a windows user). Please give it a try, as well as giving more details about your environment if you're still running into issues. I've recently compiled/ran the example code and it works fine on my machine ™️ .

[narangmayank]

Hi @BrianPugh , I tried the lastest version but it's not working either i.e (same behaviour)

Please see the app https://github.com/mayankbytebeam/littlefs_provisioning_example

Environment:

• Windows
• ESP-IDF 5.0
• Building with bare idf.py command line

Observation:

• Getting compilation error with idf.py build logs can be seen here
• If I do directly idf.py -p PORT flash monitor then I'm not getting any build error instead not able to read the file.

I (354) demo_esp_littlefs: Initializing LittleFS
I (364) demo_esp_littlefs: Partition size: total: 204800, used: 8192
I (364) demo_esp_littlefs: Reading file
E (374) demo_esp_littlefs: Failed to open file for reading

I'm not understanding this behaviour, can you please look into it?

[BrianPugh]

in your CMakeLists.txt you need to add a line like:

littlefs_create_partition_image(storage config_data)

Here, the first argument storage matches the name of the partition defined in your partitions_example.csv (listed below for completeness). The second argument is the path the local folder that you want to pack into the flashed filesystem-image.

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you have increased the bootloader size, make sure to update the offsets to avoid overlap,,,,
nvs,      data, nvs,     0x9000,  16K,
otadata,data,ota,0xd000,8K,
phy_init, data, phy,     0xf000,  4K,
factory,  app,  factory, 0x10000, 1M,
ota_0,app,ota_0,0x110000,1M,
ota_1,app,ota_1,0x210000,1M,
storage,  data, spiffs,0x310000,  200K,

[narangmayank]

Yes, I knew this one. I'm not sure if you have notice or not but I've added this in project level CMakeLists.txt

did you tried the example app in your machine by any chance? If yes, then what's the status.

[BrianPugh]

Moving that line (and updating it to below) from main/CMakeList.txt -> CMakeList.txt fixes it.

littlefs_create_partition_image(storage config_data)

Which is how it is in the example of this repo. I can try and look through the esp-idf docs to see if this is the expected location.

EDIT: actually hold on its still not quite working; messing with it more now

[BrianPugh]

Ok I think something is certainly wrong on my side, will update you with stuff as I figure it out.

[BrianPugh]

Alright, so the answer was pretty silly. A lot of this project is modeled off of the SPIFFS esp-idf port, and the solution was in the SPIFFS docs. Basically, we just had to specify FLASH_IN_PROJECT.

littlefs_create_partition_image(storage ../config_data FLASH_IN_PROJECT)

And as a side note, it doesn't matter which CMakeList.txt file you do it in, I think it's more appropriate to put it in the main/CMakeList.txt as you did, so I updated the example.

Since there was no actual implementation change, v1.5.4 remains the latest and should work 👍

[narangmayank]

Hi @BrianPugh,

sorry for the delay in the response, I just tested it out and actually it's not working. Again I'm getting error in the build time.

Logs

[18/240] Generating C:/Users/mayan/testing/littlefs_provis...aged_components/joltwallet__littlefs/mklittlefs/mklittlefsFAILED: C:/Users/mayan/testing/littlefs_provisioning_example/managed_components/joltwallet__littlefs/mklittlefs/mklittlefs
cmd.exe /C "cd /D C:\Users\mayan\testing\littlefs_provisioning_example\managed_components\joltwallet__littlefs\mklittlefs && make clean && make dist LFS_NAME_MAX=64"
fatal: No names found, cannot describe anything.
Could Not Find C:\Users\mayan\testing\littlefs_provisioning_example\managed_components\joltwallet__littlefs\mklittlefs\mklittlefs.exe
Could Not Find C:\Users\mayan\testing\littlefs_provisioning_example\managed_components\joltwallet__littlefs\src\littlefs\lfs.o
fatal: No names found, cannot describe anything.
g++ -std=gnu++11 -Os -Wall   -Itclap -Iinclude -Ilittlefs -I. -I ../src -D VERSION=\"8b5eb09\" -D LITTLEFS_VERSION=\""unknown"\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__ -D LFS_NAME_MAX=64   -c -o main.o main.cpp
process_begin: CreateProcess(NULL, g++ -std=gnu++11 -Os -Wall -Itclap -Iinclude -Ilittlefs -I. -I ../src -D VERSION=\"8b5eb09\" -D LITTLEFS_VERSION=\"unknown\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__ -D LFS_NAME_MAX=64 -c -o main.o main.cpp, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [<builtin>: main.o] Error 2
[100/101] Generating binary image from built executableesptool.py v4.5.1
Creating esp32 image...
Merged 1 ELF section
Successfully created esp32 image.
Generated C:/Users/mayan/testing/littlefs_provisioning_example/build/bootloader/bootloader.bin
[101/101] cmd.exe /C "cd /D C:\Users\mayan\testing\littlef...lefs_provisioning_example/build/bootloader/bootloader.bin"Bootloader binary size 0x6710 bytes. 0x8f0 bytes (8%) free.
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the c:\users\mayan\testing\littlefs_provisioning_example\build\log\idf_py_stderr_output_22644 and c:\users\mayan\testing\littlefs_provisioning_example\build\log\idf_py_stdout_output_22644

[BrianPugh]

I just cloned your repo on a windows 11 machine, made the one line change mentioned above, and successfully built/flashed it to an esp32 using esp-idf 4.4.

From your error log, it appears its trying to compile mklittlefs with g++, which is not on the system. See eldendiss's solution here: https://github.com/joltwallet/esp_littlefs/issues/102#issue-1353190169

If you are able to find a more generic solution, let me know.

[narangmayank]

I tried on idf 4.4.3 and Windows 11 ( ESP IDF 4.4 CMD ) but it didn't work for me. I'm sharing the error screenshot below. Could be please share the build process, exact command lines will be helpful.

Basically, I did the standard idf.py build and in the background that all is happening so there is no room for me to choose gcc or g++. Also, for better understanding of ours prospective please see our official sdk here. We just want the user to use standard idf.py build and that's all.

[narangmayank]

Just want to tell you that If I didn't use littlefs_create_partition_image(storage ../config_data FLASH_IN_PROJECT) and instead use the code to write and read back the file that way is working fine as expected. So we need to look an work around for this guy :)

[BrianPugh]

If you externally compile mklittlefs, what compiler does it use? Is something in idf.py modifying CXX? I don't use Windows, so I'm a bit out of my element on this.

[BrianPugh]

This issue should be rendered moot in v1.9.0 with the move from mklittlefs to littlefs-python.