多进程encode无法正常运行

[imempty]

在centos7、python3.10.2（python3.8也试过）环境下程序无法运行，而同样的程序在win10、anconada3.10.9及相关依赖包装好后能正常运行。报错内容参考：https://github.com/conda/conda/issues/9589

[shibing624]

不懂conda环境问题，我解决不了。

[imempty]

不是conda问题，是在centos7、python3.10.2（python3.8也试过）环境下程序无法运行。只是报错文字内容参考conda那个链接。 我贴下文字信息吧

[jupyter@jupyter-6fbbb0fa-8ae8-4c62-87bb-c79e7cbeed59-d868889b8-nbx2j data_lineage]$ python build_index_with_BGE.py 2023-10-12 11:20:24.970 | DEBUG | text2vec.sentence_model:init:80 - Use device: cpu load sentence embedding model success Sentences size: 2, model: <SentenceModel: ./BGE_model/bge-base-en-v1.5, encoder_type: MEAN, max_seq_length: 384, emb_dim: 768> 2023-10-12 11:20:24.974 | INFO | text2vec.sentence_model:start_multi_process_pool:327 - Start multi-process pool on devices: cpu, cpu, cpu, cpu, cpu, cpu /usr/local/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' Bus error (core dumped) [jupyter@jupyter-6fbbb0fa-8ae8-4c62-87bb-c79e7cbeed59-d868889b8-nbx2j data_lineage]$ gdb python core.2339 GNU gdb (GDB) EulerOS 9.2-1.oe1 Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-openEuler-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.

For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from python... [New LWP 2363] [New LWP 2345] [New LWP 2339] [New LWP 2365] [New LWP 2364] [New LWP 2343] [New LWP 2341] [New LWP 2344] [New LWP 2352] [New LWP 2348] [New LWP 2346] [New LWP 2340] [New LWP 2355] [New LWP 2353] [New LWP 2347] [New LWP 2362] [New LWP 2354] [New LWP 2357] [New LWP 2356] [New LWP 2358] [New LWP 2361] [New LWP 2360] [New LWP 2351] [New LWP 2342] [New LWP 2350] [New LWP 2359] [New LWP 2349]

warning: File "/usr/lib64/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load". To enable execution of this file add add-auto-load-safe-path /usr/lib64/libthread_db-1.0.so line to your configuration file "/home/jupyter/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/home/jupyter/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path"

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

warning: File "/usr/lib64/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Core was generated by `python build_index_with_BGE.py'. Program terminated with signal SIGBUS, Bus error.

0 0x00007fb8cd0b06c3 in void c10::function_ref<void (char*, long const, long, long)>::callback_fn<at::native::AVX2::VectorizedLoop2d<at::native::AVX2::direct_copy_kernel(at::TensorIteratorBase&)::{lambda()#3}::operator()() const::{lambda()#1}::operator()() const::{lambda(unsigned char)#1}, at::native::AVX2::direct_copy_kernel(at::TensorIteratorBase&)::{lambda()#3}::operator()() const::{lambda()#3}::operator()() const::{lambda(at::vec::AVX2::Vectorized)#2}> >(long, char**, --Type for more, q to quit, c to continue without paging--

long const*, long, long) () from /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so [Current thread is 1 (LWP 2363)] (gdb) bt

0 0x00007fb8cd0b06c3 in void c10::function_ref<void (char*, long const, long, long)>::callback_fn<at::native::AVX2::VectorizedLoop2d<at::native::AVX2::direct_copy_kernel(at::TensorIteratorBase&)::{lambda()#3}::operator()() const::{lambda()#1}::operator()() const::{lambda(unsigned char)#1}, at::native::AVX2::direct_copy_kernel(at::TensorIteratorBase&)::{lambda()#3}::operator()() const::{lambda()#3}::operator()() const::{lambda(at::vec::AVX2::Vectorized)#2}> >(long, char*, long const, long, long) ()

from /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so

1 0x00007fb8c8d77a2e in at::TensorIteratorBase::serial_for_each(c10::function_ref<void (char*, long const, long, long)>, at::Range) const () from /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so

2 0x00007fb8c8d77b62 in void at::internal::invoke_parallel<at::TensorIteratorBase::for_each(c10::function_ref<void (char*, long const, long, long)>, long)::{lambda(long, long)#1}>(long, long, long, at::TensorIteratorBase::for_each(c10::function_ref<void (char*, long const, long, long)>, long)::{lambda(long, long)#1} const&) [clone ._omp_fn.0] () from /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so

3 0x00007fb928c59405 in ?? () from /usr/local/lib/python3.10/site-packages/torch/lib/libgomp-a34b3233.so.1

4 0x00007fb95ac11f2b in ?? () from /lib64/libpthread.so.0

5 0x00007fb95a9bc70f in clone () from /lib64/libc.so.6

(gdb)

下半部分是生成.core文件后，用gdb看了下具体内容，报错信息看不懂啊，搜索也基本找不到相关可用信息。另外我申了2次微信群，你都没通过。。