search

esp-rs / rust

Rust for the xtensa architecture. Built in targets for the ESP32 and ESP8266
https://www.rust-lang.org
Other
744 stars 39 forks source link

Can't read files using `fs::read` and `fs::read_to_string` #96

Closed bugadani closed 2 years ago

bugadani commented 2 years ago

I tried reading files using the fs helper functions read and read_to_string. I was lazy to verify the actual return value, but the length of the returned data was considerably shorter (< 4 bytes) than the contents of the files.

Reading the files using File::open().read() worked as expected. Writing the files using either io::Write or std::fs::write works.

bugadani commented 2 years ago

Example:

use c_str_macro::c_str;

fn init_partition(path: &CStr, label: &CStr, max_files: usize) {
    let storage_conf = esp_vfs_spiffs_conf_t {
        base_path: path.as_ptr(),
        partition_label: label.as_ptr(),
        max_files,
        format_if_mount_failed: true,
    };

    unsafe { esp_vfs_spiffs_register(&storage_conf) };
}

fn test_fs() {
    init_partition(c_str!("/storage"), c_str!("storage"), 3);
    std::fs::write("/storage/test_file", "hello");

    let str = std::fs::read_to_string("/storage/test_file").unwrap();

    println!("String is: {:?}", str);
    assert_eq!("hello", str);
}

Output:

String is: ""
Guru Meditation Error: Core  0 panic'ed (Unhandled debug exception). 
Debug exception reason: BREAK instr
Core  0 register dump:
PC      : 0x401a6d0b  PS      : 0x00060636  A0      : 0x800f9ada  A1      : 0x3ffb3450  
0x401a6d0b: core::panicking::assert_failed_inner at C:\Users\bugad\.rustup\toolchains\esp\lib\rustlib\src\rust\library\core\src/panicking.rs:?

A2      : 0x00000000  A3      : 0x3ffb3474  A4      : 0x3f4151a0  A5      : 0x3ffb3478
A6      : 0x3f4151b0  A7      : 0x3ffb347c  A8      : 0x00000000  A9      : 0x3ffb33d0
A10     : 0x3ffc48f8  A11     : 0x3f40e150  A12     : 0x00000004  A13     : 0x50000028
A14     : 0x000000ff  A15     : 0x00000018  SAR     : 0x00000018  EXCCAUSE: 0x00000001  
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000

Backtrace:0x401a6d08:0x3ffb3450 0x400f9ad7:0x3ffb3470 0x400deafa:0x3ffb34c0 0x401b27f5:0x3ffb3a50 0x400907c9:0x3ffb3a70
0x401a6d08: core::panicking::assert_failed_inner at C:\Users\bugad\.rustup\toolchains\esp\lib\rustlib\src\rust\library\core\src/panicking.rs:?

0x400f9ad7: core::panicking::assert_failed at C:\Users\bugad\.rustup\toolchains\esp\lib\rustlib\src\rust\library\core\src/panicking.rs:138
MabezDev commented 2 years ago

Hello @bugadani,

I've been looking into the file system related issues, starting with this one. It seems in idf v4.4 (native) I can't reproduce your error, and with idf 4.3 (platformio) doesn't compile.

Here is my test repo, https://github.com/MabezDev/esp-fs-test-suite, please point out anything I might have missed.

bugadani commented 2 years ago

Interesting, I still have this issue with idf v4.4. My project was set up based on https://github.com/ivmarkov/rust-esp32-example so it's possible something went wrong here.

MabezDev commented 2 years ago

So interestingly, compiling for Xtensa in this case the esp32, I can reproduce your error on v4.4.

String is: "lo"
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"hello"`,
 right: `"lo"`', src/main.rs:28:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

In this case the first three bytes seem to be missing (seems this has also been reported in https://github.com/esp-rs/esp-idf-sys/issues/65)

MabezDev commented 2 years ago

Looks like a miscompilation issue. If I set opt level = 0, it works fine, but opt level = 1 or opt level = 'z' either fails to compile or produces the buggy code.

MabezDev commented 2 years ago

Ran this test case on Xtensa (esp32), with all opt-levels, here are the results.

opt-level = 1

Compilation failure:

fix up value out of range

opt-level = 2

Runtime failure:

String is: ""
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"hello"`,
 right: `""`', src/main.rs:42:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

abort() was called at PC 0x400f282e on core 0

Backtrace:0x40081a5a:0x3ffb63900x40085a31:0x3ffb63b0 0x4008aeb2:0x3ffb63d0 0x400f282e:0x3ffb6440 0x400f2822:0x3ffb6460 0x400dda31:0x3ffb6480 0x400dda12:0x3ffb64f0 0x400d4fe7:0x3ffb6550 0x400d4bc5:0x3ffb6580 0x400dd611:0x3ffb65b0 0x4010063f:0x3ffb65e0 0x401006fe:0x3ffb6610 0x400d35b1:0x3ffb66a0 0x400d34ba:0x3ffb66f0 0x4011a99f:0x3ffb6750 0x400d3210:0x3ffb6770 0x400d5805:0x3ffb6790 0x400d3575:0x3ffb67c0 0x400d38df:0x3ffb67f0 0x4011bb01:0x3ffb6810 0x40088399:0x3ffb6830

opt-level = 3

Runtime failure:

String is: ""
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"hello"`,
 right: `""`', src/main.rs:42:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

abort() was called at PC 0x400f5172 on core 0

Backtrace:0x40081a5a:0x3ffb63800x40085a31:0x3ffb63a0 0x4008aeb2:0x3ffb63c0 0x400f5172:0x3ffb6430 0x400f5166:0x3ffb6450 0x400de2bd:0x3ffb6470 0x400de298:0x3ffb64f0 0x400d4f71:0x3ffb6560 0x400d4a48:0x3ffb6590 0x400ddd85:0x3ffb65b0 0x40105ff7:0x3ffb65e0 0x401060bd:0x3ffb6610 0x400d305d:0x3ffb66a0 0x400d3341:0x3ffb66f0 0x4012084b:0x3ffb6750 0x400d30b8:0x3ffb6770 0x400d5949:0x3ffb6790 0x400d3415:0x3ffb67c0 0x400d372b:0x3ffb67f0 0x40121aa1:0x3ffb6810 0x40088399:0x3ffb6830

opt-level = 's'

Runtime failure:

String is: ""
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"hello"`,
 right: `""`', src/main.rs:42:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

abort() was called at PC 0x400f3162 on core 0

Backtrace:0x40081a5e:0x3ffb61800x40085305:0x3ffb61a0 0x40089b52:0x3ffb61c0 0x400f3162:0x3ffb6230 0x400f3156:0x3ffb6250 0x400e27d1:0x3ffb6270 0x400e2769:0x3ffb62f0 0x400d9507:0x3ffb6360 0x400d9225:0x3ffb6390 0x400e24e5:0x3ffb63c0 0x401009b3:0x3ffb63f0 0x40100af8:0x3ffb6420 0x400d3e13:0x3ffb64b0 0x400d3d8e:0x3ffb6500 0x401195ff:0x3ffb6570 0x400d3de0:0x3ffb6590 0x400f0859:0x3ffb65b0 0x400d3dc1:0x3ffb65d0 0x400d4123:0x3ffb6600 0x4011a5a4:0x3ffb6620

opt-level = 'z'

Runtime failure:

String is: "lo"
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"hello"`,
 right: `"lo"`', src/main.rs:42:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

abort() was called at PC 0x400f3826 on core 0

Backtrace:0x40081a5e:0x3ffb61700x40085305:0x3ffb6190 0x40089b52:0x3ffb61b0 0x400f3826:0x3ffb6220 0x400f381a:0x3ffb6240 0x400e32a1:0x3ffb6260 0x400e328b:0x3ffb62e0 0x400d9db3:0x3ffb6340 0x400d9a0d:0x3ffb6370 0x400e30b8:0x3ffb63a0 0x4010194b:0x3ffb63d0 0x40101a90:0x3ffb6400 0x400d4bc7:0x3ffb6490 0x400d4afe:0x3ffb64e0 0x4011a50b:0x3ffb6550 0x400d4c04:0x3ffb6570 0x400f13f5:0x3ffb6590 0x400d4bf2:0x3ffb65b0 0x400d4b55:0x3ffb65e0 0x400d4f5f:0x3ffb6600 0x4011b7b0:0x3ffb6620
jeffutter commented 2 years ago

I’ve run into this problem too. Writing seems to work. The files get created, but reading them, they are empty (not just missing 3 characters).

In my case, I can’t seem to work around it with opt-level = 0. I think that makes my firmware too large to fit into ram or something and it locks up while booting.

MabezDev commented 2 years ago

In my case, I can’t seem to work around it with opt-level = 0. I think that makes my firmware too large to fit into ram or something and it locks up while booting.

If you're using a chip with 2MB of flash, this might be the case. When testing I was using an esp32 with 4MB of flash.

MabezDev commented 2 years ago

Would you guys mind trying the 1.60.0.0 branch of the compiler? It seems this has been fixed by updating libc, which is sort of surprising. Would be great if either of you could confirm :).

bugadani commented 2 years ago

String is: "hello" this seems good, indeed!

axos88 commented 2 years ago

This is still an issue with the esp32c3, with the built-in compiler, I am getting a core dump:

I (452) doorlock: Dir / = [Ok(DirEntry("/st/foo.txt"))]
Guru Meditation Error: Core  0 panic'ed (Instruction access fault). Exception was unhandled.
Core  0 register dump:
MEPC    : 0x00000000  RA      : 0x00000000  SP      : 0x3fc99c70  GP      : 0x3fc8f600  
TP      : 0x3fc669b4  T0      : 0x000081ff  T1      : 0x00000000  T2      : 0x00000000  
S0/FP   : 0x00000000  S1      : 0x00000000  A0      : 0x3fc99c84  A1      : 0x3fc99c64  
A2      : 0x00000000  A3      : 0x3fc99cc0  A4      : 0x3fc99cc0  A5      : 0x3fc99cc0  
A6      : 0x00000000  A7      : 0x00000000  S2      : 0x3fc99cec  S3      : 0x3fc9ba1c  
S4      : 0x00000001  S5      : 0x00000000  S6      : 0x00000000  S7      : 0x00000000  
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x00000000  T4      : 0x00000003  T5      : 0x00000000  T6      : 0x00000000  
MSTATUS : 0x00001881  MTVEC   : 0x40380001  MCAUSE  : 0x00000001  MTVAL   : 0x00000000  
0x40380001 - _vector_table
    at ??:??
MHARTID : 0x00000000  
Stack memory:
3fc99c70: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x000081ff 0x00000000
3fc99c90: 0x00000000 0x00000003 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc99cb0: 0x00000000 0x00000000 0x00000000 0x00000000 0x3fc99d00 0x3fc99cec 0x3fc99cf0 0x42028a9e
0x42028a9e - _ZN47_$LT$std..fs..File$u20$as$u20$std..io..Read$GT$14read_to_string17h85bd660d9a67a604E
    at ??:??
3fc99cd0: 0x3fc99d18 0x00000003 0x3fc99ed8 0x42028938 0x00000069 0x00000000 0x00000000 0x00000003
0x42028938 - _ZN3std2fs14read_to_string5inner17h8a38a1c9e96c254bE
    at ??:??
3fc99cf0: 0x00000001 0x00000000 0x00000000 0x3fc9ba1c 0x3fc99d18 0x00000003 0x3fc99ed8 0x4200833e
0x4200833e - _ZN8doorlock4main17h44ef55067e871a3cE
    at ??:??
3fc99d10: 0xa5a5a5a5 0xa5a5a5a5 0x3fc9cb40 0x00000004 0x00000001 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99d30: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99d50: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99d70: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0x3c0f1f18 0x00000001 0x00000000 0xa5a5a501
3fc99d90: 0x3fc99ed8 0x00000001 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99db0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99dd0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99df0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99e10: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99e30: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99e50: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99e70: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99e90: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
3fc99eb0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0x4038ae26 0xa5a5a5a5 0xa5a5a5a5 0x00000000 0xc0380000
0x4038ae26 - mapping_insert
    at /home/akos/projects/rust/esp-rs/esp-idf/components/heap/tlsf/tlsf.c:281
3fc99ed0: 0x00000010 0x0000000c 0x3fc99d18 0x42002364 0x3fc9b79c 0x00000008 0x00000000 0xc0380000
0x42002364 - _ZN65_$LT$alloc..vec..Vec$LT$T$C$A$GT$$u20$as$u20$core..fmt..Debug$GT$3fmt17h0216278f2c131a03E
    at ??:??
3fc99ef0: 0x00000008 0x00000004 0x3fc961c0 0x4038ae26 0xa5a5a5a5 0xa5a5a5a5 0x00000000 0xc0380000
0x4038ae26 - mapping_insert
    at /home/akos/projects/rust/esp-rs/esp-idf/components/heap/tlsf/tlsf.c:281
3fc99f10: 0x00000010 0x0000000c 0x3fc961c0 0x4038a6b2 0x00000000 0x00000000 0x00000000 0x0000000c
0x4038a6b2 - multi_heap_malloc_impl
    at /home/akos/projects/rust/esp-rs/esp-idf/components/heap/multi_heap.c:217
3fc99f30: 0x00000000 0x0000000c 0x00001800 0x40380aec 0x00000000 0x00000000 0x00000000 0x00000000
0x40380aec - heap_caps_malloc_base
    at /home/akos/projects/rust/esp-rs/esp-idf/components/heap/heap_caps.c:147
3fc99f50: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000003
3fc99f70: 0x00001800 0x3fc9a0a8 0x3fc9aac0 0x42035e5a 0x00000000 0x00000002 0x00000000 0x00000000
0x42035e5a - _ZN3std6thread5local4lazy21LazyKeyInner$LT$T$GT$10initialize17hdb59896fa4f310ebE
    at ??:??
3fc99f90: 0x00000000 0x0000001c 0x00000000 0x3fc961d4 0x00000000 0x4207ee4a 0x3fc9a9c8 0x3fc9aa10
0x4207ee4a - esp_register_freertos_idle_hook_for_cpu
    at /home/akos/projects/rust/esp-rs/esp-idf/components/esp_system/freertos_hooks.c:71
3fc99fb0: 0x00000000 0x3fc9a0a8 0x3fc9a068 0x4202868a 0x00001800 0x00000005 0x00000000 0x4207ee4a
0x4202868a - _ZN3std6thread5local17LocalKey$LT$T$GT$4with17h0d1dc80d17364dc4E
    at ??:??
0x4207ee4a - esp_register_freertos_idle_hook_for_cpu
    at /home/akos/projects/rust/esp-rs/esp-idf/components/esp_system/freertos_hooks.c:71
3fc99fd0: 0x3fc9a9c8 0x3fc9aa10 0x00000001 0x4203f9e6 0x00000005 0x00000004 0x3c0fdacc 0x00000000
0x4203f9e6 - _ZN3std6thread6Thread3new17h0c436ff9245578fdE
    at ??:??
3fc99ff0: 0x00000000 0x00000000 0x00000004 0x40380b06 0x3fc9aa00 0x00000005 0x00000005 0x00000000
0x40380b06 - heap_caps_malloc
    at /home/akos/projects/rust/esp-rs/esp-idf/components/heap/heap_caps.c:166
3fc9a010: 0x00000000 0x3fc9a0a8 0x3c0f1550 0x42041708 0xa5a5a5a5 0xa5a5a5a5 0x3fc9a9ec 0x00000000
0x42041708 - _ZN3std2rt4init17h7f4a1c8aff67e15aE
    at ??:??
3fc9a030: 0x3fc9aa00 0x00000005 0x00000002 0x01000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc9a050: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x3fc9a0a8 0x3c0f1550 0x42009880
0x42009880 - _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h2cd253b5e8d6c706E
    at ??:??
ELF file SHA256: 0000000000000000
Rebooting...
ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0xc (SPI_FAST_FLASH_BOOT)
Saved PC:0x40048b82
0x40048b82 - chip726_phyrom_version_num
    at ??:??
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x16fc
load:0x403cc710,len:0x95c
0x403cc710 - _iram_data_end
    at ??:??
load:0x403ce710,len:0x2dbc
0x403ce710 - _iram_data_end
    at ??:??
entry 0x403cc710
0x403cc710 - _iram_data_end
    at ??:??

Code is: 

    let files: Vec<_> = fs::read_dir("/st").unwrap().into_iter().collect();

    info!("Dir / = {:?}", files);

    info!("foo = {:?}", fs::read_to_string("/st/foo.txt"));
axos88 commented 2 years ago

I am using an esp32c3 with 4MB of flash and esp-idf master (5.1)

MabezDev commented 2 years ago

@axos88 Sorry to hear you're having issues. esp-idf v5 is not yet officially supported, so there is still some work to do. With that said, this should be fixable by upgrading libc in the standard library to a version which includes this PR: https://github.com/rust-lang/libc/pull/2913. Then you will also need to pass --cfg espidf_time64 in your RUSTFLAGS.

Feel free to open a new issues if it persists :).

axos88 commented 2 years ago

Ah gotcha. Does that mean if I update to latest nightly, and include the RUSTFLAG, it should work?

axos88 commented 2 years ago

Shouldn't adding a dependency and patch to libc = { git = "https://github.com/rust-lang/libc.git" } fix the issue?

My rustflags are: rustflags = "--cfg espidf_time64 -C default-linker-libraries "

MabezDev commented 2 years ago

Shouldn't adding a dependency and patch to libc = { git = "https://github.com/rust-lang/libc.git" } fix the issue?

My rustflags are: rustflags = "--cfg espidf_time64 -C default-linker-libraries "

Unfortunately not, because cargo patch does not patch std dependencies.

axos88 commented 2 years ago

@MabezDev, so do you have any pointers on how to do what you are suggesting? upgrading libc in the standard library to a version which includes https://github.com/rust-lang/libc/pull/2913?