juicedata / juicefs

JuiceFS is a distributed POSIX file system built on top of Redis and S3.
https://juicefs.com
Apache License 2.0
10.93k stars 965 forks source link

Bus Error #4678

Open recall704 opened 7 months ago

recall704 commented 7 months ago

一、出现的情况

使用 python 程序 往 juicefs 中写入大量 jpg 文件失败,出现 bus error

Bus error (core dumped) 

由于出现了 coredump,使用 gdb python3 core.321111 后内容如下

Core was generated by `/usr/bin/python3 /usr/local/bin/data_extract --output_folder test_mao /clever/p'.
Program terminated with signal SIGBUS, Bus error.
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:353
353     ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
[Current thread is 1 (Thread 0x7f43b7feb700 (LWP 1165))]
(gdb) bt full
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:353
No locals.
#1  0x00007f4401ff8c88 in int pcl::PCDWriter::writeBinaryCompressed<file_writer::PointXYZITR>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, pcl::PointCloud<file_writer::PointXYZITR> const&) () from /usr/local/lib/python3.8/dist-packages/datafusion/libfilewriter.so
No symbol table info available.
#2  0x00007f4401fec3b7 in save_pcd_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<double []>, unsigned long, unsigned char, unsigned char) () from /usr/local/lib/python3.8/dist-packages/datafusion/libfilewriter.so
No symbol table info available.
#3  0x00007f4401ff018e in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<double []>, unsigned long, unsigned char, unsigned char), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<double []>, unsigned long, unsigned char, unsigned char> > >::_M_run() () from /usr/local/lib/python3.8/dist-packages/datafusion/libfilewriter.so
No symbol table info available.
#4  0x00007f44b71cfde4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#5  0x00007f455a53a609 in start_thread (arg=<optimized out>) at pthread_create.c:477
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139928826459904, 3175586999392010751, 140720883313038, 140720883313039, 140720883313040, 139928826458048, -3272256073010249217, -3270942457440894465}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = 0
#6  0x00007f455a674353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

二、期望状态

正常写入文件

三、怎样复现

由于数据涉及到敏感信息,无法提供

四、正常情况

正常情况 1

往本地磁盘写入数据正常

正常情况 2

juicefs 挂载时开启 内核回写模式 时写入正常

五、环境信息

操作系统: centos7 系统内核版本: 5.4.269 meta组件: redis:6.2.14 csi 版本: 0.19.6 JuiceFS version: 5.0.6 (2023-12-25 04eaeb2)

SandyXSD commented 7 months ago

JuiceFS version: 5.0.6 (2023-12-25 04eaeb2) 这个是企业版的版本号,但你又提到了 meta组件: redis:6.2.14,因此无法确认具体使用的是哪个版本。另外请尽量提供一些非涉密的复现步骤,不然我们难以入手,谢谢!

Waynnnnnne commented 4 months ago

我也遇到了同样的问题, 使用C++的pcl库向juicefs写入点云pcd文件提示BUS ERROR core dump, 向本地磁盘或nfs均正常 代码: pcl::io::savePCDFileBinaryCompressed(pcd_file, cloud); 操作系统: ubuntu20.04 pcl version: 1.10.0+dfsg-5ubuntu1 redis_version:5.0.3 minio version RELEASE.2024-06-29T01-20-47Z juicefs version 1.2.0+2024-06-18.873c47b img_v3_02cf_4d7240a1-3196-43b5-b7b1-e73a593fba7g