Crash on file write or read

hitecSmartHome commented 3 months ago

Hi!

I have a crash on file write every time at the same spot.

I'm using arduino as a component of IDF.

IDF Version: 4.4.6
Arduino Version: 2.0.14

Here is my coredump

==================== CURRENT THREAD STACK =====================
#0  0x400839c0 in panic_abort (details=0x3ffd09c0 "assert failed: 0x401c7251") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\panic.c:408
#1  0x4008a134 in esp_system_abort (details=0x3ffd09c0 "assert failed: 0x401c7251") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\esp_system.c:137
#2  0x4008f1a8 in __assert_func (file=<optimized out>, line=0, func=0x0, expr=0x0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components
ewlib\assert.c:47
#3  0x401c7254 in lfs_file_write (lfs=0x3ffbe9d8, file=0x0, buffer=0x3f8186c0, size=128) at managed_components\esp_littlefs\src\littlefs\lfs.c:6144
#4  0x401c30e5 in vfs_littlefs_write (ctx=0x3ffbe8f4, fd=<optimized out>, data=0x3f8186c0, size=128) at managed_components\esp_littlefs\src\esp_littlefs.c:1509
#5  0x401f60f1 in esp_vfs_write (r=<optimized out>, fd=0, data=0x3f8186c0, size=128) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\vfs\vfs.c:431
#6  0x4000bd86 in ?? ()
#7  0x40001180 in ?? ()
#8  0x40059301 in ?? ()
#9  0x4005937d in ?? ()
#10 0x40058ad6 in ?? ()
#11 0x402699c2 in _fwrite_r (ptr=0x3ffbf02c, buf=<optimized out>, size=1, count=1, fp=0x3ffaec1c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fwrite.c:137
#12 0x40269a24 in fwrite (buf=0x3ffd0b90, size=1, count=1, fp=0x3ffaec1c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fwrite.c:177
#13 0x401c080c in VFSFileImpl::write (this=0x3f816a50, buf=0x3ffd0b90 "0", size=1) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/vfs_api.cpp:399
#14 0x401a78be in fs::File::write (this=<optimized out>, c=<optimized out>) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.cpp:32
#15 0x40272324 in ArduinoJson::V700PB2::detail::Writer<fs::File, void>::write (c=48 '0', this=0x3ffd0cb0) at lib/ArduinoJson-7.x/src/ArduinoJson/Serialization/Writers/PrintWriter.hpp:19
#16 ArduinoJson::V700PB2::detail::CountingDecorator<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::write (this=0x3ffd0cb0, c=48 '0') at lib/ArduinoJson-7.x/src/ArduinoJson/Serialization/CountingDecorator.hpp:17
#17 0x40138446 in ArduinoJson::V700PB2::detail::TextFormatter<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::writeRaw (c=<optimized out>, this=<optimized out>) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/TextFormatter.hpp:166
#18 ArduinoJson::V700PB2::detail::TextFormatter<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::writeChar (this=0x3ffd0cb0, c=<optimized out>) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/TextFormatter.hpp:59
#19 0x40138475 in ArduinoJson::V700PB2::detail::TextFormatter<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::writeString (this=0x3ffd0cb0, value=0x3ffe0934 <error: Cannot access memory at address 0x3ffe0934>, n=<optimized out>) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/TextFormatter.hpp:51
#20 0x40138650 in ArduinoJson::V700PB2::detail::JsonSerializer<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::visit (this=<optimized out>, value=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonSerializer.hpp:73
#21 ArduinoJson::V700PB2::detail::VariantData::accept<ArduinoJson::V700PB2::detail::JsonSerializer<ArduinoJson::V700PB2::detail::Writer<fs::File, void> > > (this=<optimized out>, visit=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Variant/VariantData.hpp:45
#22 0x401384ff in ArduinoJson::V700PB2::detail::JsonSerializer<ArduinoJson::V700PB2::detail::Writer<fs::File, void> >::visit (this=0x3ffd0cb0, object=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Collection/CollectionData.hpp:59
#23 0x40138630 in ArduinoJson::V700PB2::detail::VariantData::accept<ArduinoJson::V700PB2::detail::JsonSerializer<ArduinoJson::V700PB2::detail::Writer<fs::File, void> > > (this=0x3ffd0d88, visit=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Variant/VariantData.hpp:37
#24 0x40138690 in ArduinoJson::V700PB2::detail::VariantData::accept<ArduinoJson::V700PB2::detail::JsonSerializer<ArduinoJson::V700PB2::detail::Writer<fs::File, void> > > (visit=..., var=0x3ffd0d88) at lib/ArduinoJson-7.x/src/ArduinoJson/Variant/VariantData.hpp:69
#25 ArduinoJson::V700PB2::detail::doSerialize<ArduinoJson::V700PB2::detail::JsonSerializer, ArduinoJson::V700PB2::detail::Writer<fs::File, void> > (writer=..., source=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Serialization/serialize.hpp:15
#26 ArduinoJson::V700PB2::detail::serialize<ArduinoJson::V700PB2::detail::JsonSerializer, fs::File> (source=..., destination=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Serialization/serialize.hpp:22
#27 0x40138794 in ArduinoJson::V700PB2::serializeJson<fs::File> (destination=..., source=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonSerializer.hpp:131
#28 FileSystem::write (this=0x3ffb43c0 <db>, path=0x3f40b0dc "/config.json", doc=...) at src/FileSystem/FileSystemWrites.cpp:22
#29 0x4013334a in Config::setNew (this=0x3ffb4058 <config>, doc=...) at src/Config/config.cpp:214
#30 0x401339de in Config::save (this=0x3ffb4058 <config>) at src/Config/config.cpp:74
#31 0x40169775 in <lambda(TmParams*)>::operator() (__closure=0x3ffd0ea0, info=<optimized out>) at src/NewSunCycle/NewSunCycle.cpp:75
#32 std::_Function_handler<void(TmParams*), NewSunCycle::calculate()::<lambda(TmParams*)> >::_M_invoke(const std::_Any_data &, TmParams *&&) (__functor=..., __args#0=@0x3ffd0ecc: 0x3ffd0ec0) at c:\users\pc\.platformio\packages oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
#33 0x4019373c in std::function<void (TmParams*)>::operator()(TmParams*) const (__args#0=<optimized out>, this=0x3ffd0ea0) at c:\users\pc\.platformio\packages  oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
#34 Sys::handleTimeouts (this=0x3ffb3b10 <hshSystem>) at src/System/SystemTimeout.cpp:61
#35 0x4018af0a in Sys::loop (this=0x3ffb3b10 <hshSystem>) at src/System/System.cpp:551
#36 0x4018af2c in systemTask (pvParameters=<optimized out>) at src/System/System.cpp:559

I have a method which calls file write that looks like this

void Config::setNew(JsonDocument& doc){
    if( !db.write(CONFIG_PATH, doc) ){
        #if CONFIG_DEBUG
            logger.write("[Config] - Configuration file saved...\n");
        #endif
    }
}

Which calls this method

boolean FileSystem::write(const char* path, JsonDocument& doc){
    if( !createPath(path) ){ return false; }
    File file = userFS.open(path, FILE_WRITE);
    if(!file){
        #if DB_DEBUG
            logger.write("[FileSystem] - (writeJson) File not found: %s\n",path);
        #endif
        return false;
    }
    size_t result = serializeJson(doc, file);
    file.flush();
    file.close();
    return result != 0;
}

It is using ArduinoJson V7. I have asked arduinojson about this problem but he pointed me to here. createPath() method:

boolean FileSystem::createPath(const char* path){
    if(userFS.exists(path)){
        #if DB_DEBUG
            logger.write("[FileSystem] - (createPath) Path already exists: %s\n",path);
        #endif
        return true;
    }

    if (strchr(path, '/')) {
        #if DB_DEBUG
            logger.write("[FileSystem] - Creating path: %s\n",path);
        #endif
        char *pathStr = strdup(path);
        if (pathStr) {
            char *ptr = strchr(pathStr, '/');
            while (ptr) {
                *ptr = 0;
                userFS.mkdir(pathStr);
                *ptr = '/';
                ptr = strchr(ptr+1, '/');
            }
        }
        free(pathStr);
        return true;
    }
    return false;
}

Is this because of multiple core wants to write at once? Thank you for your help!

hitecSmartHome commented 3 months ago

It can also happen on other file writes like this one

==================== CURRENT THREAD STACK =====================
#0  0x400839c0 in panic_abort (details=0x3fff0f30 "assert failed: 0x401c71b1") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\panic.c:408
#1  0x4008a134 in esp_system_abort (details=0x3fff0f30 "assert failed: 0x401c71b1") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\esp_system.c:137
#2  0x4008f1a8 in __assert_func (file=<optimized out>, line=0, func=0x0, expr=0x0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components
ewlib\assert.c:47
#3  0x401c71b4 in lfs_file_close (lfs=0x3ffbea48, file=0x0) at managed_components\esp_littlefs\src\littlefs\lfs.c:6092
#4  0x401c3008 in vfs_littlefs_close (ctx=0x3ffbe964, fd=0) at managed_components\esp_littlefs\src\esp_littlefs.c:1698
#5  0x401f61ce in esp_vfs_close (r=<optimized out>, fd=0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\vfs\vfs.c:498
#6  0x4000bd4e in ?? ()
#7  0x40001784 in ?? ()
#8  0x401a52ac in WiFiClientSecure::stop (this=0x3ffb44cc <espFetch>) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.cpp:93
#9  0x401a111d in ESP_Fetch::fetch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned short, ArduinoJson::V700PB2::JsonDocument&, std::function<void (ArduinoJson::V700PB2::JsonDocument)>) (this=0x3ffb44cc <espFetch>, url=..., port=<optimized out>, options=..., cb=...) at src/espFetch/espFetch.cpp:205
#10 0x401a1377 in ESP_Fetch::fetch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned short, ArduinoJson::V700PB2::JsonDocument&, std::function<void (ArduinoJson::V700PB2::JsonDocument)>) (this=0x3ffb44cc <espFetch>, url=..., port=443, options=..., cb=...) at src/espFetch/espFetch.cpp:235
#11 0x401a13c6 in ESP_Fetch::fetch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ArduinoJson::V700PB2::JsonDocument&, std::function<void (ArduinoJson::V700PB2::JsonDocument)>) (this=0x3ffb44cc <espFetch>, url=..., options=..., cb=...) at src/espFetch/espFetch.cpp:255
#12 0x401a1499 in ESP_Fetch::fetch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::function<void (ArduinoJson::V700PB2::JsonDocument)>) (this=0x3ffb44cc <espFetch>, url=..., cb=...) at src/espFetch/espFetch.cpp:265
#13 0x4019e615 in VersionHandler::fetch (this=0x3ffb5168 <versionHandler>) at src/VersionHandler/VersionHandler.cpp:34
#14 0x4019e671 in versionHandlerTask (pvParameters=0x0) at src/VersionHandler/VersionHandler.cpp:16

This called the same LittleFS method

void VersionHandler::saveInfo(JsonDocument doc){
    if( !db.write(VERSION_INFO_PATH,doc) ){
        #if VERSION_DEBUG
        logger.write("[VersionHandler] - Failed to save version info\n");
        #endif
    }
}

Can it be because of the flush() call?

It also happened on file read

==================== CURRENT THREAD STACK =====================
#0  0x400839c0 in panic_abort (details=0x3ffcc3c0 "assert failed: 0x401c720b") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\panic.c:408
#1  0x4008a134 in esp_system_abort (details=0x3ffcc3c0 "assert failed: 0x401c720b") at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\esp_system\esp_system.c:137
#2  0x4008f1a8 in __assert_func (file=<optimized out>, line=0, func=0x0, expr=0x0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components
ewlib\assert.c:47
#3  0x401c720e in lfs_file_read (lfs=0x3ffbe870, file=0x0, buffer=0x3f82688c, size=128) at managed_components\esp_littlefs\src\littlefs\lfs.c:6126
#4  0x401c313d in vfs_littlefs_read (ctx=0x3ffbe78c, fd=<optimized out>, dst=0x3f82688c, size=128) at managed_components\esp_littlefs\src\esp_littlefs.c:1545
#5  0x401f6189 in esp_vfs_read (r=<optimized out>, fd=0, dst=0x3f82688c, size=128) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\vfs\vfs.c:457
#6  0x4000bdbe in ?? ()
#7  0x40001128 in ?? ()
#8  0x400594ec in ?? ()
#9  0x40269409 in _fread_r (ptr=0x3ffccaf4, buf=<optimized out>, size=1, count=1, fp=0x3ffd1a8c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fread.c:228
#10 0x4026949c in fread (buf=0x3ffcc580, size=1, count=1, fp=0x3ffd1a8c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fread.c:266
#11 0x401c083a in VFSFileImpl::read (this=0x3f8260b4, buf=0x3ffcc580 "\214\020\034\200\300\305\374?\034\306\374?$\306\374?", size=1) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/vfs_api.cpp:408
#12 0x401a797e in fs::File::read (this=<optimized out>, buf=0x3ffcc580 "\214\020\034\200\300\305\374?\034\306\374?$\306\374?", size=1) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.cpp:82
#13 0x401a7990 in fs::File::readBytes (this=0x3ffcc690, buffer=0x3ffcc580 "\214\020\034\200\300\305\374?\034\306\374?$\306\374?", length=1) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.h:63
#14 0x402722a5 in ArduinoJson::V700PB2::detail::Reader<fs::File, void>::read (this=0x3ffcc610) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/Readers/ArduinoStreamReader.hpp:17
#15 ArduinoJson::V700PB2::detail::Latch<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::load (this=0x3ffcc610) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/Latch.hpp:38
#16 ArduinoJson::V700PB2::detail::Latch<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::current (this=0x3ffcc610) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/Latch.hpp:30
#17 ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::current (this=0x3ffcc600) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:47
#18 0x40137959 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::skipSpacesAndComments (this=0x3ffcc600) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:581
#19 0x40137ca1 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::skipSpacesAndComments (this=0x3ffcc600) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:579
#20 ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::parseVariant<ArduinoJson::V700PB2::detail::AllowAllFilter> (this=0x3ffcc600, variant=..., filter=..., nestingLimit=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:67
#21 0x40137f49 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::parse<ArduinoJson::V700PB2::detail::AllowAllFilter> (nestingLimit=..., variant=..., this=0x3ffcc600, filter=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:31
#22 ArduinoJson::V700PB2::detail::doDeserialize<ArduinoJson::V700PB2::detail::JsonDeserializer, ArduinoJson::V700PB2::JsonDocument&, ArduinoJson::V700PB2::detail::Reader<fs::File, void>, ArduinoJson::V700PB2::detail::DeserializationOptions<ArduinoJson::V700PB2::detail::AllowAllFilter> > (dst=..., reader=..., options=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/deserialize.hpp:57
#23 0x401380f9 in ArduinoJson::V700PB2::detail::deserialize<ArduinoJson::V700PB2::detail::JsonDeserializer, ArduinoJson::V700PB2::JsonDocument&, fs::File&, , void>(ArduinoJson::V700PB2::JsonDocument&, fs::File&) (input=..., dst=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/deserialize.hpp:69
#24 ArduinoJson::V700PB2::deserializeJson<ArduinoJson::V700PB2::JsonDocument&, fs::File&> (dst=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:679
#25 FileSystem::read (this=0x3ffb43c0 <db>, path=0x3f40228c "/graphs.json", doc=...) at src/FileSystem/FileSystemReads.cpp:138
#26 0x400e697e in Graphs::get (this=<optimized out>, type=0x3f406918 "shader", id=..., graphData=...) at src/Components/Graph/Graph.cpp:8
#27 0x4010c4d2 in Shader::handleGraph (this=0x3f8515ec, detail=...) at src/Components/Shader/GraphHandler.cpp:22
#28 0x4010c670 in Shader::checkGraph (this=0x3f8515ec) at src/Components/Shader/GraphHandler.cpp:13
#29 0x40113a5d in <lambda(int, int)>::operator() (__closure=0x3ffcc950, prev=<optimized out>, now=<optimized out>) at src/Components/Shader/Shader.cpp:14
#30 std::_Function_handler<void(int, int), Shader::Shader(ArduinoJson::V700PB2::JsonObject&)::<lambda(int, int)> >::_M_invoke(const std::_Any_data &, int &&, int &&) (__functor=..., __args#0=@0x3ffcc920: 59, __args#1=@0x3ffcc924: 0) at c:\users\pc\.platformio\packages       oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
#31 0x401977c0 in std::function<void (int, int)>::operator()(int, int) const (this=0x3ffcc950, __args#0=<optimized out>, __args#1=<optimized out>) at c:\users\pc\.platformio\packages     oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
#32 0x40197888 in Time::emitMinChange (now=0, prev=59, this=<optimized out>) at src/Time/TimeEvents.cpp:45
#33 Time::emitMinChange (this=<optimized out>, prev=59, now=0) at src/Time/TimeEvents.cpp:39
#34 0x40197a0a in Time::monitorMinChange (this=0x3ffb3d7c <hshTime>) at src/Time/TimeEvents.cpp:154
#35 0x40197a33 in Time::handleEvents (this=0x3ffb3d7c <hshTime>) at src/Time/TimeEvents.cpp:205
#36 0x4019676d in Time::loop (this=0x3ffb3d7c <hshTime>) at src/Time/Time.cpp:211
#37 0x401a1754 in loop () at src/main.cpp:91
#38 0x401bf404 in loopTask (pvParameters=<optimized out>) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:50

This crash happened in here

int FileSystem::read(const char* path, JsonDocument& doc){
    File file = userFS.open(path);
    if(!file){
        #if DB_DEBUG
            logger.write("[FileSystem] - (getJson) File not found: %s\n",path);
        #endif
        return -1;
    }
    DeserializationError error = deserializeJson(doc, file);
    file.close();
    if( error == DeserializationError::Ok ){
        #if DB_DEBUG
            logger.write("[FileSystem] - (getJson) Message read from file: %s\n",path);
        #endif
        return doc.size();
    }
    return -1;
}

BrianPugh commented 3 months ago

It seems like in your cases, the following LFS assertion is failing:

    LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file));

Can you produce a minimal example reproducing the bug?

hitecSmartHome commented 3 months ago

Well, I try.

hitecSmartHome commented 3 months ago

Meanwhile I have removed the code I suspected and the crash happened again on file append when closed the filw

==================== THREAD 2 (TCB: 0x3ffcca88, name: 'loopTask') =====================
#0  0x4000bff0 in ?? ()
#1  0x4008ab49 in vPortClearInterruptMaskFromISR (prev_level=<optimized out>) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\freertos\port\xtensa\include/freertos/portmacro.h:571
#2  vPortExitCritical (mux=<optimized out>) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\freertos\port\xtensa\port.c:332
#3  0x4027b562 in xQueueGenericSend (xQueue=0x3ffbeabc, pvItemToQueue=<optimized out>, xTicksToWait=<optimized out>, xCopyPosition=0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\freertos\queue.c:946
#4  0x4027b6ba in xQueueGiveMutexRecursive (xMutex=0x3ffbeabc) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\freertos\queue.c:685
#5  0x401c2b00 in sem_give (efs=<optimized out>) at managed_components\esp_littlefs\src\esp_littlefs.c:1149
#6  0x401c352a in vfs_littlefs_close (ctx=0x3ffbea38, fd=0) at managed_components\esp_littlefs\src\esp_littlefs.c:1718
#7  0x401f5f46 in esp_vfs_close (r=<optimized out>, fd=0) at C:\Users\Pc\.platformio\packages\framework-espidf@3.40406.240122\components\vfs\vfs.c:498
#8  0x4000bd4e in ?? ()
#9  0x400011c4 in ?? ()
#10 0x40002058 in ?? ()
#11 0x400020b8 in ?? ()
#12 0x401c0650 in VFSFileImpl::close (this=0x3f813688) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/vfs_api.cpp:359
#13 0x401a7c12 in fs::File::close (this=0x3ffcc810) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.cpp:145
#14 0x401387e8 in FileSystem::append (this=0x3ffb43c0 <db>, path=<optimized out>, doc=..., terminator=10 '
') at src/FileSystem/FileSystemWrites.cpp:188
#15 0x4015386c in Logger::log (this=0x3ffb43f0 <logger>, type=2 '\002', tag=<optimized out>, message=<optimized out>) at src/Logger/Log.cpp:42
#16 0x4012e380 in <lambda(int, int)>::operator() (__closure=0x3ffcc950, prev=<optimized out>, now=<optimized out>) at src/Components/WateringSystem/WateringSystem.cpp:19
#17 std::_Function_handler<void(int, int), WateringSystem::WateringSystem(ArduinoJson::V700PB2::JsonObject&)::<lambda(int, int)> >::_M_invoke(const std::_Any_data &, int &&, int &&) (__functor=..., __args#0=<optimized out>, __args#1=<optimized out>) at c:\users\pc\.platformio\packages oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
#18 0x401977cc in std::function<void (int, int)>::operator()(int, int) const (this=0x3ffcc950, __args#0=<optimized out>, __args#1=<optimized out>) at c:\users\pc\.platformio\packages        oolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
#19 0x40197894 in Time::emitMinChange (now=0, prev=59, this=<optimized out>) at src/Time/TimeEvents.cpp:45
#20 Time::emitMinChange (this=<optimized out>, prev=59, now=0) at src/Time/TimeEvents.cpp:39
#21 0x40197a16 in Time::monitorMinChange (this=0x3ffb3d7c <hshTime>) at src/Time/TimeEvents.cpp:154
#22 0x40197a3f in Time::handleEvents (this=0x3ffb3d7c <hshTime>) at src/Time/TimeEvents.cpp:205
#23 0x40196779 in Time::loop (this=0x3ffb3d7c <hshTime>) at src/Time/Time.cpp:211
#24 0x401a17ff in loop () at src/main.cpp:90
#25 0x401bf2a4 in loopTask (pvParameters=<optimized out>) at C:/Users/Pc/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:50

boolean FileSystem::append(const char* path, JsonDocument& doc, char terminator){
    if( !createPath(path) ){ return false; }
    File file = userFS.open(path, FILE_APPEND);
    if(!file){
        #if DB_DEBUG
            logger.write("[FileSystem] - (append) File not found: %s\n",path);
        #endif
        return false;
    }
    boolean result = serializeJson(doc,file);
    if( result ){
        file.print(terminator);
        #if DB_DEBUG
            logger.write("[FileSystem] - (append) Json Message written to file: %s\n",path);
        #endif
    }
    #if DB_DEBUG
        logger.write("[FileSystem] - (append) Failed to append json to file: %s\n",path);
    #endif

    file.close();
    return result;
}

BrianPugh commented 3 months ago

try and reduce it to a super minimal project. Some of these symptoms are indicative that you may have a buffer-overflow rewriting some of these objects, resulting in odd errors.

hitecSmartHome commented 3 months ago

I see. I will try to reproduce. Thank you very much!

hitecSmartHome commented 2 months ago

Well, I can't easily reproduce this bug. I have five esp32 running 12 days now with a test sketch and no crash occoured so far. In my main application hovewer this crash happened again. Soooo frustrating.... This crash happened when the ESP32 tried to READ from a file. It seems to me that it is an invalid File pointer reference somewhere in the library.

My code:

int FileSystem::read(const char* path, JsonDocument& doc){
    File file = userFS.open(path);
    if(!file){
        #if DB_DEBUG
            logger.write("[FileSystem] - (getJson) File not found: %s\n",path);
        #endif
        return -1;
    }
    DeserializationError error = deserializeJson(doc, file);
    file.close();
    if( error == DeserializationError::Ok ){
        #if DB_DEBUG
            logger.write("[FileSystem] - (getJson) Message read from file: %s\n",path);
        #endif
        return doc.size();
    }
    return -1;
}

I'm sure it checks for invalid file.

Crash:

==================== THREAD 1 (TCB: 0x3ffcd194, name: 'loopTask') =====================
#0  0x40083cc0 in panic_abort (details=0x3ffccac0 "assert failed: 0x401adb0b") at C:\Users\DrRandom\.platformio\packages\framework-espidf\components\esp_system\panic.c:408
#1  0x401dad54 in esp_system_abort (details=0x3ffccac0 "assert failed: 0x401adb0b") at C:\Users\DrRandom\.platformio\packages\framework-espidf\components\esp_system\esp_system.c:137
#2  0x400906cc in __assert_func (file=<optimized out>, line=0, func=0x0, expr=0x0) at C:\Users\DrRandom\.platformio\packages\framework-espidf\components
ewlib\assert.c:47
#3  0x401adb0e in lfs_file_read (lfs=0x3ffbf228, file=0x0, buffer=0x3ffe99b0, size=128) at .pio/libdeps/esp-wrover-kit/esp_littlefs/src/littlefs/lfs.c:6126
#4  0x401a9a99 in vfs_littlefs_read (ctx=0x3ffbf144, fd=<optimized out>, dst=0x3ffe99b0, size=128) at .pio/libdeps/esp-wrover-kit/esp_littlefs/src/esp_littlefs.c:1545
#5  0x401fdbf9 in esp_vfs_read (r=<optimized out>, fd=0, dst=0x3ffe99b0, size=128) at C:\Users\DrRandom\.platformio\packages\framework-espidf\components\vfs\vfs.c:457
#6  0x4000bdbe in ?? ()
#7  0x40001128 in ?? ()
#8  0x400594ec in ?? ()
#9  0x4027186d in _fread_r (ptr=0x3ffcd200, buf=<optimized out>, size=1, count=1, fp=0x3ffaec1c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fread.c:228
#10 0x40271900 in fread (buf=0x3ffccc80, size=1, count=1, fp=0x3ffaec1c) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fread.c:266
#11 0x401cbaa2 in VFSFileImpl::read (this=0x3fff1488, buf=0x3ffccc80 "\364\302\034\200\300\314\374?\034\315\374?$\315\374?", size=1) at C:/Users/DrRandom/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/vfs_api.cpp:408
#12 0x401b3012 in fs::File::read (this=<optimized out>, buf=0x3ffccc80 "\364\302\034\200\300\314\374?\034\315\374?$\315\374?", size=1) at C:/Users/DrRandom/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.cpp:82
#13 0x401b3024 in fs::File::readBytes (this=0x3ffccd90, buffer=0x3ffccc80 "\364\302\034\200\300\314\374?\034\315\374?$\315\374?", length=1) at C:/Users/DrRandom/.platformio/packages/framework-arduinoespressif32/libraries/FS/src/FS.h:63
#14 0x4027a951 in ArduinoJson::V700PB2::detail::Reader<fs::File, void>::read (this=0x3ffccd10) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/Readers/ArduinoStreamReader.hpp:17
#15 ArduinoJson::V700PB2::detail::Latch<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::load (this=0x3ffccd10) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/Latch.hpp:38
#16 ArduinoJson::V700PB2::detail::Latch<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::current (this=0x3ffccd10) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/Latch.hpp:30
#17 ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::current (this=0x3ffccd00) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:47
#18 0x4013c6f9 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::skipSpacesAndComments (this=0x3ffccd00) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:581
#19 0x4013ca41 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::skipSpacesAndComments (this=0x3ffccd00) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:579
#20 ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::parseVariant<ArduinoJson::V700PB2::detail::AllowAllFilter> (this=0x3ffccd00, variant=..., filter=..., nestingLimit=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:67
#21 0x4013cce9 in ArduinoJson::V700PB2::detail::JsonDeserializer<ArduinoJson::V700PB2::detail::Reader<fs::File, void> >::parse<ArduinoJson::V700PB2::detail::AllowAllFilter> (nestingLimit=..., variant=..., this=0x3ffccd00, filter=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:31
#22 ArduinoJson::V700PB2::detail::doDeserialize<ArduinoJson::V700PB2::detail::JsonDeserializer, ArduinoJson::V700PB2::JsonDocument&, ArduinoJson::V700PB2::detail::Reader<fs::File, void>, ArduinoJson::V700PB2::detail::DeserializationOptions<ArduinoJson::V700PB2::detail::AllowAllFilter> > (dst=..., reader=..., options=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/deserialize.hpp:57
#23 0x4013ce99 in ArduinoJson::V700PB2::detail::deserialize<ArduinoJson::V700PB2::detail::JsonDeserializer, ArduinoJson::V700PB2::JsonDocument&, fs::File&, , void>(ArduinoJson::V700PB2::JsonDocument&, fs::File&) (input=..., dst=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Deserialization/deserialize.hpp:69
#24 ArduinoJson::V700PB2::deserializeJson<ArduinoJson::V700PB2::JsonDocument&, fs::File&> (dst=...) at lib/ArduinoJson-7.x/src/ArduinoJson/Json/JsonDeserializer.hpp:679
#25 FileSystem::read (this=0x3ffb4880 <db>, path=0x3f402270 "/graphs.json", doc=...) at src/FileSystem/FileSystemReads.cpp:138
#26 0x400e74ba in Graphs::get (this=<optimized out>, type=0x3f406c58 "shader", id=..., graphData=...) at src/Components/Graph/Graph.cpp:8
#27 0x4010ed3e in Shader::handleGraph (this=0x3f8629a0, detail=...) at src/Components/Shader/GraphHandler.cpp:22
#28 0x4010eedc in Shader::checkGraph (this=0x3f8629a0) at src/Components/Shader/GraphHandler.cpp:13
#29 0x40116311 in <lambda(int, int)>::operator() (__closure=0x3ffcd050, prev=<optimized out>, now=<optimized out>) at src/Components/Shader/Shader.cpp:14
#30 std::_Function_handler<void(int, int), Shader::Shader(ArduinoJson::V700PB2::JsonObject&)::<lambda(int, int)> >::_M_invoke(const std::_Any_data &, int &&, int &&) (__functor=..., __args#0=@0x3ffcd020: 59, __args#1=@0x3ffcd024: 0) at c:\users\drrandom\.platformio\packages  oolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
#31 0x4019e600 in std::function<void (int, int)>::operator()(int, int) const (this=0x3ffcd050, __args#0=<optimized out>, __args#1=<optimized out>) at c:\users\drrandom\.platformio\packages        oolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
#32 0x4019e6d4 in Time::emitMinChange (now=0, prev=59, this=<optimized out>) at src/Time/TimeEvents.cpp:54
#33 Time::emitMinChange (this=<optimized out>, prev=59, now=0) at src/Time/TimeEvents.cpp:47
#34 0x4019e85e in Time::monitorMinChange (this=0x3ffb423c <hshTime>) at src/Time/TimeEvents.cpp:168
#35 0x4019e887 in Time::handleEvents (this=0x3ffb423c <hshTime>) at src/Time/TimeEvents.cpp:219
#36 0x4019d5b5 in Time::loop (this=0x3ffb423c <hshTime>) at src/Time/Time.cpp:211
#37 0x401a736f in loop () at src/main.cpp:91
#38 0x401ca668 in loopTask (pvParameters=<optimized out>) at C:/Users/DrRandom/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:50

This function returns File() if the _impl pointer is invalid in FS.cpp

File FS::open(const char* path, const char* mode, const bool create)
{
    if (!_impl) {
        return File();
    }

    return File(_impl->open(path, mode, create));
}

Is this correct? What if that happens? What is an empty File()?

hitecSmartHome commented 2 months ago

Meanwhile one of my other thread called vfs_littlefs_close

==================== THREAD 2 (TCB: 0x3fff9bac, name: 'versionHandlerT') =====================
#0  0x401a995e in vfs_littlefs_close (ctx=0x3ffbf144, fd=0) at .pio/libdeps/esp-wrover-kit/esp_littlefs/src/esp_littlefs.c:1698

BrianPugh commented 2 months ago

@hitecSmartHome I think these issues you are seeing are outside the scope of the repo. Couple possibilities (unless there's concrete evidence it's an issue in this repo):

Somewhere else there's a buffer overflow stomping on the file pointer.
In your logging/reading code there's a race condition. Possibly opening/closing the same file handle in different threads?
There's an issue in the file abstraction layer. I'm not sure what File() refers to in this situation (it's outside this repo).

hitecSmartHome commented 2 months ago

Thank you very much for the help. Still searching for buffer overflows

joltwallet / esp_littlefs

Crash on file write or read #186