Linker multiple definition issue with FATFS and VFS (IDFGH-13239)

AntoineSX commented 1 month ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.3-rc1

Operating System used.

Linux

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Calling opendir, closedir, mkdir works in v5.2.2. I use std::filesystem to call std::filesystem::exists, std::filesystem::remove and std::filesystem::file_size with no issues.

What is the actual behavior?

With v5.3 you get a multiple definition error at link time with newlib's implementations of opendir, closedir, mkdir and so on.

Steps to reproduce.

Make a component with fatfs in the component's REQUIRES
call any of the vfs io functions.
build ...

Build or installation Logs.

/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_ops.o):(.literal._ZNSt10filesystem12current_pathB5cxx11ERSt10error_code+0x8): warning: pathconf is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_ops.o):(.literal._ZNSt10filesystem11permissionsERKNS_7__cxx114pathENS_5permsENS_12perm_optionsERSt10error_code+0xc): warning: chmod is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_dir.o):(.literal._ZNKSt10filesystem7__cxx114_Dir7currentEv[_ZNKSt10filesystem7__cxx114_Dir7currentEv]+0x4): warning: dirfd is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: esp-idf/mbedtls/mbedtls/library/libmbedx509.a(x509_crt.c.obj):(.literal.mbedtls_x509_crt_parse_path+0x38): warning: readdir is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_ops.o):(.literal._ZNSt10filesystem12current_pathERKNS_7__cxx114pathERSt10error_code+0x0): warning: chdir is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_ops.o):(.literal._ZNSt10filesystem12current_pathB5cxx11ERSt10error_code+0x14): warning: getcwd is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libstdc++.a(fs_ops.o):(.literal._ZN12_GLOBAL__N_110create_dirERKNSt10filesystem7__cxx114pathENS0_5permsERSt10error_code+0x0): warning: mkdir is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: esp-idf/mbedtls/mbedtls/library/libmbedx509.a(x509_crt.c.obj):(.literal.mbedtls_x509_crt_parse_path+0x20): warning: closedir is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: esp-idf/mbedtls/mbedtls/library/libmbedx509.a(x509_crt.c.obj):(.literal.mbedtls_x509_crt_parse_path+0x1c): warning: opendir is not implemented and will always fail
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `opendir':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:16: multiple definition of `opendir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:733: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `closedir':
/builds/idf/crosstool-NG/.build/src/newlib-git-7cdcb0b0/newlib/libc/include/dirent.h:76: multiple definition of `closedir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:805: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `mkdir':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:30: multiple definition of `mkdir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:818: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `getcwd':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:37: multiple definition of `getcwd'; esp-idf/newlib/libnewlib.a(realpath.c.obj):/opt/esp/idf/components/newlib/realpath.c:111: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `chdir':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:42: multiple definition of `chdir'; esp-idf/newlib/libnewlib.a(realpath.c.obj):/opt/esp/idf/components/newlib/realpath.c:120: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `readdir':
/builds/idf/crosstool-NG/.build/src/newlib-git-7cdcb0b0/newlib/libc/include/dirent.h:56: multiple definition of `readdir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:750: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `telldir':
/builds/idf/crosstool-NG/.build/src/newlib-git-7cdcb0b0/newlib/libc/include/dirent.h:73: multiple definition of `telldir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:776: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `seekdir':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:73: multiple definition of `seekdir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:789: first defined here
/opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/bin/ld: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/13.2.0/../../../../xtensa-esp-elf/lib/esp32s3/no-rtti/libnosys.a(dir.o): in function `rewinddir':
/builds/idf/crosstool-NG/.build/xtensa-esp-elf/src/newlib/libgloss/libnosys/dir.c:101: multiple definition of `rewinddir'; esp-idf/vfs/libvfs.a(vfs.c.obj):/opt/esp/idf/components/vfs/vfs.c:800: first defined here
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

More Information.

No response

igrr commented 1 month ago

I think the key point here is not the usage of fatfs component, but the usage of std::filesystem. I also got the same issue with the latest toolchain when trying std::filesystem.

We didn't have any test cases which used std::filesystem, so we didn't notice this issue when doing the last toolchain upgrade. We'll try to fix this soon. In the meantime, please try this patch which can help you temporarily bypass this issue: https://github.com/espressif/esp-idf/pull/13736#issuecomment-2102443764

AntoineSX commented 1 month ago

Thank you ! The patch works. What's the timeframe for having a fix on the release branch ? I just need a ballpark figure.

espressif / esp-idf