deephealthproject / eddl

European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
https://deephealthproject.github.io/eddl/
MIT License
34 stars 10 forks source link

Core dump in EDDL 0.8a using pyEDDL 0.10.1 #224

Closed diegobenedicto closed 3 years ago

diegobenedicto commented 3 years ago

Describe the bug When I execute the python code attached (based on https://github.com/deephealthproject/use_case_pipeline/tree/3rd_hackathon/python) I get a core dump

To Reproduce DeepHealth libraries

Use the file: code.zip

conda env create -f environment.yml

If I execute the python code using gdb (https://wiki.python.org/moin/DebuggingWithGdb) the error is: gdb -ex r --args python msseg08a.py

Thread 1 "python" received signal SIGILL, Illegal instruction. 0x00007fffd93d07a0 in Tensor::updateData(float, void, bool) () from /usr/local/lib/libeddl.so

This error comes from line in python code: in_ = eddl.Input([n_channels, size[0], size[1]])

Screenshots image

simleo commented 3 years ago

Hi Diego,

I've seen the Illegal instruction problem several times recently while testing the latest Docker images. Every time the problem was that EDDL was being compiled with the new BUILD_HPC CMake flag, which is ON by default. That enables extra compiler flags like -march=native -mtune=native, which break the library if it's compiled on a machine and run on another. That's typically the case with Docker, where you build the image once and then run it somewhere else.

To avoid this problem, compile EDDL with -D BUILD_HPC=OFF.

This worked for me and hopefully should fix your issue as well.

diegobenedicto commented 3 years ago

I have compiled EDDL 0.8a with -D BUILD_HPC=OFF and now it works. Thanks Simone!

jonandergomez commented 3 years ago

Thanks Diego and Simone,

I understand it is not necessary yet I check it with C++.

Anyway, do not hesitate to ask Roberto, Salva or me any other problem you find.

Thanks and regards,

Jon

salvacarrion commented 3 years ago

Since the EDDL is intended for C++ users, we've set the -D BUILD_HPC flag to ON by default (this is due to the substantial increase in performance we get). Nevertheless, if you intend to compile the library for a different CPU or docker, you should disable it. To aid with the troubleshooting I've added a warning:

  [WARNING] The 'BUILD_HPC' flag is enabled and might not work on a different
  CPU.

  If you intend to compile the library for different CPUs (or for a docker
  image), you should disable it: '-D BUILD_HPC=OFF'.  This happens because it
  makes use of native compiling flags.

Ps.: If it gives users a lot of headaches, we'll disable it by default