Ubuntu 22,04: core dumped when translating

[dlippold]

I have an updated system Ubuntu 22.04. and have compiled translateLocally as described on the project page. When I execute ./translateLocally -v I get:

translateLocally v0.0.2+a210037

But when I try to translate somethin by the CLI, e.g. echo "Me gustaria comprar la casa verde" | ./translateLocally -m es-en-tiny, I get:

Aborted (core dumped)

This happens also when I type the first charavter in the GUI.

What can I do to investigate the problem?

[jelmervdl]

What's your output when you run cat /proc/cpuinfo?

I'm suspecting you're either running into a marian model loading bug, or your CPU not supporting the matrix operation implementations that are selected at runtime.

[XapaJIaMnu]

~You can try to build it from scratch. You can then produce a .deb using cpack so you can still have your package nicely installed and managed.~

Scratch that, the issue is that MKL was not found, same as #154 .

[dlippold]

I use a virtual root server (Root vServer).I attached the output of cat /proc/cpuinfo as file cpuinfo.txt

I installed the MKL by the command apt-get install intel-mkl

The output of dpkg-query -l | grep intel-mkl is:

ii intel-mkl:amd64 2020.4.304-2ubuntu3 amd64 Intel® Math Kernel Library (Intel® MKL)

I attached the output of the command ls /usr/lib/x86_64-linux-gnu/libmkl_* as file libmkl-files.txt

And I attached the output of the command cmake .. as file cmake-output.txt In that file there is the following line:

-- Could NOT find MKL (missing: MKL_LIBRARIES MKL_INCLUDE_DIRS MKL_INTERFACE_LIB RARY MKL_SEQUENTIAL_LAYER_LIBRARY MKL_CORE_LIBRARY)

but also the line

-- Found BLAS: /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.a;/usr/lib/x86_64-linux-gnu/libmkl_sequential.a;/usr/lib/x86_64-linux-gnu/libmkl_core.a;-lm;-ldl

What shoud I do?

cpuinfo.txt libmkl-files.txt cmake-output.txt

[XapaJIaMnu]

So you can compile with BLAS. (Refer to the readme, compiling with OpenBLAS works but it is slow).

The reason why the MKL libraries are not detected is that we link to the static libraries, not the shared ones.

After compiling by yourself, do you still get the core dump?

[XapaJIaMnu]

Resolved as per #156

[dlippold]

The compilation was not successful.

First I installed some libraries:

apt-get install libblas-dev liblapack-dev libatlas3-base libatlas-base-dev

Then I executed cmake:

cmake .. -DBLAS_LIBRARIES=-lblas -DCBLAS_LIBRARIES=-lcblas

I get the following output:

-- Not Found Tcmalloc CMake Warning at 3rd_party/bergamot-translator/3rd_party/marian-dev/CMakeLists.txt:608 (message): Cannot find TCMalloc library. Continuing.

I ignored that as a warning, executed the make and get the following output:

/home/testuser/TransLocal/translateLocally/src/inventory/ModelManager.cpp: In constructor ‘ModelManager::ModelManager(QObject, Settings)’: /home/testuser/TransLocal/translateLocally/src/inventory/ModelManager.cpp:82:83: warning: ‘QDir& QDir::operator=(const QString&)’ is deprecated: Use QDir::setPath() instead [-Wdeprecated-declarations] 82	appDataDir_ = QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);	^ In file included from /usr/include/x86_64-linux-gnu/qt5/QtCore/QDir:1, from /home/testuser/TransLocal/translateLocally/src/inventory/ModelManager.h:3, from /home/testuser/TransLocal/translateLocally/src/inventory/ModelManager.cpp:1: /usr/include/x86_64-linux-gnu/qt5/QtCore/qdir.h:110:11: note: declared here 110	QDir &operator=(const QString &path);

A some lines lated maybe more important:

/usr/bin/ld: libmarian.a(VectorTransform.cpp.o): in function (anonymous namespace)::eig(unsigned long, double*, double*, int)': VectorTransform.cpp:(.text+0x1fc): undefined reference todsyev' /usr/bin/ld: VectorTransform.cpp:(.text+0x26a): undefined reference to `dsyev' /usr/bin/ld: libmarian.a(VectorTransform.cpp.o): in function matrix_qr(int, int, float*)': VectorTransform.cpp:(.text+0x25c1): undefined reference tosgeqrf' /usr/bin/ld: VectorTransform.cpp:(.text+0x266a): undefined reference to `sgeqrf' /usr/bin/ld: VectorTransform.cpp:(.text+0x2694): undefined reference to sorgqr_' /usr/bin/ld: libmarian.a(lsh.cpp.o): in functionstd::_Function_handler<void (IntrusivePtr<marian::Chainable<IntrusivePtr > >, std::vector<IntrusivePtr<marian::Chainable<IntrusivePtr > >, std::allocator<IntrusivePtr<marian::Chainable<IntrusivePtr > > > > const&), marian::LSH::affine(IntrusivePtr<marian::Chainable<IntrusivePtr > >, IntrusivePtr<marian::Chainable<IntrusivePtr > >, IntrusivePtr<marian::Chainable<IntrusivePtr > >, IntrusivePtr<marian::Chainable<IntrusivePtr > >)::{lambda(IntrusivePtr<marian::Chainable<IntrusivePtr > >, std::vector<IntrusivePtr<marian::Chainable<IntrusivePtr > >, std::allocator<IntrusivePtr<marian::Chainable<IntrusivePtr > > > > const&)#1}>::_M_invoke(std::_Any_data const&, IntrusivePtr<marian::Chainable<IntrusivePtr > >&&, std::vector<IntrusivePtr<marian::Chainable<IntrusivePtr > >, std::allocator<IntrusivePtr<marian::Chainable<IntrusivePtr > > > > const&)': lsh.cpp:(.text+0x33a4): undefined reference to cblas_sgemm(CBLAS_ORDER, CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const*, int, float const*, int, float, float*, int)' /usr/bin/ld: lsh.cpp:(.text+0x343c): undefined reference tocblas_sgemm(CBLAS_ORDER, CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const, int, float const, int, float, float, int)' /usr/bin/ld: libmarian.a(prod.cpp.o): in function marian::cpu::Prod(IntrusivePtr<marian::TensorBase>, IntrusivePtr<marian::TensorBase> const&, IntrusivePtr<marian::TensorBase> const&, bool, bool, float, float)': prod.cpp:(.text+0x14ef): undefined reference tocblas_sgemm(CBLAS_ORDER, CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const, int, float const, int, float, float, int)' /usr/bin/ld: libmarian.a(prod.cpp.o): in function marian::cpu::ProdBatchedOld(IntrusivePtr<marian::TensorBase>, std::shared_ptr<marian::Allocator>, IntrusivePtr<marian::TensorBase>, IntrusivePtr<marian::TensorBase>, bool, bool, float, float)': prod.cpp:(.text+0x20bf): undefined reference tocblas_sgemm(CBLAS_ORDER, CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const, int, float const, int, float, float, int)' /usr/bin/ld: prod.cpp:(.text+0x2162): undefined reference to `cblas_sgemm(CBLAS_ORDER, CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const, int, float const, int, float, float, int)' collect2: error: ld returned 1 exit status make[2]: [CMakeFiles/translateLocally-bin.dir/build.make:384: translateLocally] Error 1 make[1]: [CMakeFiles/Makefile2:476: CMakeFiles/translateLocally-bin.dir/all] Error 2 make: *** [Makefile:156: all] Error 2

What can I do?

BTW: I was successful to install translateLocally for Ubuntu 20.04 by using the package from the download page (https://translatelocally.com/downloads/) by installing the packages for qt5 in addition to the already available packages for qt6.

[XapaJIaMnu]

Basically none of the libraries you are linking to provide the cblas_sgemm symbol. Try installing cblas and cblas-dev (just trying to guess ubuntu package naming scheme).

Also, if you have shared MKL installed you can try adding -DBUILD_SHARED_LIBS=ON and see if you can find them in that way.

[dlippold]

The packages libatlas3-base and libatlas-base-dev contain cblas as far as I understand.

The output of ls /usr/lib/x86_64-linux-gnu/libcblas.* is:

/usr/lib/x86_64-linux-gnu/libcblas.a /usr/lib/x86_64-linux-gnu/libcblas.so /usr/lib/x86_64-linux-gnu/libcblas.so.3 /usr/lib/x86_64-linux-gnu/libcblas.so.3.10.3

And the output of ar -t /usr/lib/x86_64-linux-gnu/libcblas.a | grep cblas_sgemm is:

cblas_sgemm.o

And the output of grep cblas_sgemm /usr/include/x86_64-linux-gnu/cblas-atlas.h is:

void cblas_sgemm(const enum CBLAS_ORDER Order, const enum CBLAS_TRANSPOSE TransA,

I also executed cmake .. -DBUILD_SHARED_LIBS=ON and then make -j4, but still get the following output when I try to translate:

Aborted (core dumped)

Interestingly the output of ldd translateLocally | grep -i mkl is nothing, i.e. the created binary seems not to contain a shared MKL library.

[dlippold]

For your information: I tested the creating of the program by compilation after I installed the MKL similar as described on the project page in the section Install MKL for Ubuntu (Any) (but installed the new intel-mkl-core-2020.0-166 of type amd64 instead of intel-mkl-64bit-2020.0-088).

The execution of cmake and make run in the same way as after installation of the MKL package from Ubuntu (after apt-get install intel-mkl). And also the result was the same. When I try to translate something I get the following output:

Aborted (core dumped)

[dlippold]

To all who read this ticket because they want to use translateLocally on Ubuntu 22.04: As I happened to see, there is already a build for Ubuntu 22.04. You can find it on the releases page https://github.com/XapaJIaMnu/translateLocally/releases when you unfold the section Assets.