search

UCL / pet-rd-tools

Command line tools for PET-MR (pre-)processing
Apache License 2.0
13 stars 4 forks source link

Segmentation fault when running nm_validate and nm_extract #45

Closed skarrea closed 3 years ago

skarrea commented 4 years ago

I get the following error messages when attempting to convert raw PET from a .dcm and .bf pair to the required interfile format for SIRF. I don't have any idea what is causing the issue.

nm_validate -i

I20200804 15:52:31.933540 34091 NMValidate.cpp:101] Started: Tue Aug 4 15:52:31 2020 I20200804 15:52:31.933631 34091 NMValidate.cpp:102] Running 'nm_validate' version: 2.0.0 I20200804 15:52:31.934140 34091 Common.hpp:190] Manufacturer: SIEMENS I20200804 15:52:31.934155 34091 Common.hpp:198] Model name: Biograph_mMR I20200804 15:52:31.934165 34091 MMR.hpp:135] Image type: ORIGINAL\PRIMARY\PET_LISTMODE Segmentation fault

nm_extract -i

I20200804 15:54:50.730965 34102 NMExtract.cpp:105] Started: Tue Aug 4 15:54:50 2020 I20200804 15:54:50.731042 34102 NMExtract.cpp:106] Running 'nm_extract' version: 2.0.0 I20200804 15:54:50.731570 34102 Common.hpp:190] Manufacturer: SIEMENS I20200804 15:54:50.731580 34102 Common.hpp:198] Model name: Biograph_mMR I20200804 15:54:50.731588 34102 MMR.hpp:135] Image type: ORIGINAL\PRIMARY\PET_LISTMODE I20200804 15:54:50.732039 34102 NMExtract.cpp:151] No output directory specified. Placing output in same directory as input. I20200804 15:54:50.732087 34102 NMExtract.cpp:182] Writing raw data to: Segmentation fault

KrisThielemans commented 4 years ago

no idea... can you provide some info on OS and compiler etc? In any case, it looks like you will need to have a go at providing more information yourself.

would you be able to build in Debug mode? (for most "generators", including make, this means setting CMAKE_BUILD_TYPE=Debug). You might get a little bit more information then. If not, and if you have a debugger, ideally you'd run it from there. For instance,

gdb --args nm_validate -i
$gdb> run
<segmentation fault>
$gdb > info stack

That should tell us where it crashed. Hopefully that'll help.

skarrea commented 4 years ago

System information: ubuntu 18.04 LTS cmake 3.10.2 gcc 7.5.0

Boost 1.61.1 GLOG 0.4.0 ITK 5.1

$gdb> run yields:

Starting program: /usr/local/bin/nm_validate -i [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". I20200804 16:45:55.361763 35848 NMValidate.cpp:101] Started: Tue Aug 4 16:45:55 2020 I20200804 16:45:55.361856 35848 NMValidate.cpp:102] Running 'nm_validate' version: 2.0.0 I20200804 16:45:55.362444 35848 Common.hpp:190] Manufacturer: SIEMENS I20200804 16:45:55.362459 35848 Common.hpp:198] Model name: Biograph_mMR I20200804 16:45:55.362468 35848 MMR.hpp:126] Manufacturer = SIEMENS I20200804 16:45:55.362476 35848 MMR.hpp:135] Image type: ORIGINAL\PRIMARY\PET_PHYSIO I20200804 16:45:55.362483 35848 MMR.hpp:138] Scanner = MMR

Program received signal SIGSEGV, Segmentation fault. 0x0000555555694102 in gdcm::SmartPointer::operator* (this=0x8) at /usr/local/include/ITK-5.1/gdcmSmartPointer.h:63 63 assert( Pointer );

$gdb > info stack yields:

multi-thre Thread 0x7ffff7fcae In: gdcm::SmartPointer<g* L63 PC: 0x555555694102

gdcmSmartPointer.h is appended for reference:

gdcmSmartPointer.txt

Seems the error lies in ITK. I'll try to rebuild using ITK 4.x.x.

KrisThielemans commented 4 years ago

interesting. obviously the assert could have thrown an error when the pointer isn't initialised, but it should cause a segfault.

Could you potentially still show all of the stack to see from where this is called?

skarrea commented 4 years ago

(gdb) info stack

0 0x0000555555694102 in gdcm::SmartPointer::operator* (this=0x8)

at /usr/local/include/ITK-5.1/gdcmSmartPointer.h:63

1 0x0000555555691056 in gdcm::Reader::GetFile (this=0x0)

at /usr/local/include/ITK-5.1/gdcmReader.h:75

2 0x000055555568941f in nmtools::IMMR::ReadHeader (this=0x555555f8b520)

at /home/mrcancer/PET_Tools/pet-rd-tools/lib/nmtools/MMR.hpp:181

3 0x000055555568be16 in nmtools::MMRSino::IsValid (this=0x555555f8b520)

at /home/mrcancer/PET_Tools/pet-rd-tools/lib/nmtools/MMR.hpp:534

4 0x000055555568f70a in main (argc=3, argv=0x7fffffffdb58)

at /home/mrcancer/PET_Tools/pet-rd-tools/src/NMValidate.cpp:143
skarrea commented 4 years ago

The following docker files should be able to reproduce the error. At least it does so on my end.

docker_files.zip

szho42 commented 3 years ago

This happens on my side as well, for both nm_validate and nm_extract. The error happends when the overloaded '*' is called.

Error log:

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". I20201019 04:56:05.504776 322 NMValidate.cpp:101] Started: Mon Oct 19 04:56:05 2020 I20201019 04:56:05.504864 322 NMValidate.cpp:102] Running 'nm_validate' version: 2.0.0 I20201019 04:56:05.505218 322 Common.hpp:190] Manufacturer: SIEMENS I20201019 04:56:05.505246 322 Common.hpp:198] Model name: Biograph_mMR I20201019 04:56:05.505270 322 MMR.hpp:135] Image type: ORIGINAL\PRIMARY\PET_LISTMODE

Program received signal SIGSEGV, Segmentation fault. 0x0000555c038aeda8 in gdcm::SmartPointer::operator*() const () (gdb) info stack

0 0x0000555c038aeda8 in gdcm::SmartPointer::operator*() const ()

1 0x0000555c038ac6de in gdcm::Reader::GetFile() ()

2 0x0000555c038a5a8e in nmtools::IMMR::ReadHeader() ()

3 0x0000555c038a80d2 in nmtools::MMRSino::IsValid() ()

4 0x0000555c038ab0bd in main ()

KrisThielemans commented 3 years ago

Thanks for the extra info. I do now see that in https://github.com/UCL/pet-rd-tools/issues/45#issuecomment-669107744 it says that gdcm::Reader::GetFile is called with this=0x0, which is obviously wrong. However, I cannot immediately spot anything wrong in the code.

The reader calls gdcm::GetFile https://github.com/UCL/pet-rd-tools/blob/2a18303761fb105f0aa7d7204904135ffd68f444/lib/nmtools/MMR.hpp#L179-L181 The gdcm object is initialised https://github.com/UCL/pet-rd-tools/blob/2a18303761fb105f0aa7d7204904135ffd68f444/lib/nmtools/Common.hpp#L127-L146

I do see some weaknesses in the above.

The strange thing of course is that you've got some sensible output first. And that the code works for us... Maybe there's something wrong with the data? Would it be possible to post data where it fails somewhere? That'd make things a lot easier.

szho42 commented 3 years ago

thanks for the reply. Sorry that, we are not able to share the data at the moment.

We installed the latest version of all the dependencies (within in docker container): gcc - 8.3.0 ITK - 5.1.1 cmake - 3.18.4 boost - 1.61 glog - not relevant in the error

What is the versions of the dependencies in your working setup?

Maybe a shared docker image with working setup might be valuable?

KrisThielemans commented 3 years ago

version info is here. Main difference seems the ITK version. Note sure if @skarrea tried with ITK 4.13.

It seems that our default VM and docker container aren't built with the pet_rd_tools unfortunately. You could add it to the existing docker or VM of course, essentially

cd ~/devel/build
cmake -DBUILD_pet_rd_tools=ON .
make pet_rd_tools

These use Ubuntu 18.04 (gcc7) but it works on CentOS 7 as well.

Or you build using all of our versions like

SIRF_INSTALL_PATH=<somethinghere>
git clone https://github.com/SyneRBI/SIRF-SuperBuild.git
cd ..
mkdir -p build 
cd build
cmake ../SIRF-SuperBuild \
        -DCMAKE_INSTALL_PREFIX=${SIRF_INSTALL_PATH} \
        -DUSE_SYSTEM_SWIG=On \
        -DUSE_SYSTEM_Boost=On \
        -DUSE_SYSTEM_Armadillo=On \
        -DUSE_SYSTEM_FFTW3=On \
        -DUSE_SYSTEM_HDF5=ON \
        -DBUILD_pet_rd_tools=On \
        -DUSE_ITK=ON
make pet_rd_tools

I might try to build with more recent ITK, but am a bit pressed in time at the moment.

KrisThielemans commented 3 years ago

Please try #47 if you can. it should prevent a crash but I still don't know why the underlying error happened.

skarrea commented 3 years ago

Yes, I tried to build it with ITK 4.13, but it does not seem to make a difference on my end. The last thing I tried was essentially building the pet-rd-tools inside the sirf superbuild docker image by essentially running the following in the docker image to build pet-rd-tools:

# glog
cd ~
git clone --branch v0.3.5 https://github.com/google/glog.git glog
cd glog

cmake . -DCMAKE_BUILD_TYPE=Release
make
sudo make install

# pet-rd-tools
cd ~
sudo git clone https://github.com/UCL/pet-rd-tools.git
cd pet-rd-tools
git fetch origin pull/47/head:read_safety
git checkout read_safety
mkdir build && cd build
cmake .. && make && sudo make install

cd ~

The build log of the pet-rd-tools, appended below, give some warnings with c++ version used in the pet-rd-tool compared to ITK 4.13.

build_log.txt

With #47 this resulted in the following: 

I1021 11:13:48.966899  2590 NMValidate.cpp:101] Started: Wed Oct 21 11:13:48 2020
I1021 11:13:48.967110  2590 NMValidate.cpp:102] Running 'nm_validate' version: 2.0.0
I1021 11:13:48.997905  2590 Common.hpp:192] Manufacturer: SIEMENS
I1021 11:13:48.997963  2590 Common.hpp:200] Model name: Biograph_mMR
I1021 11:13:48.997982  2590 MMR.hpp:126] Manufacturer = SIEMENS
I1021 11:13:48.998010  2590 MMR.hpp:135] Image type: ORIGINAL\PRIMARY\PET_LISTMODE
I1021 11:13:48.998020  2590 MMR.hpp:138] Scanner = MMR
E1021 11:13:48.999905  2590 MMR.hpp:182] DICOM reader not initialised. Internal error.
E1021 11:13:49.000061  2590 MMR.hpp:540] Unable to read header!
E1021 11:13:49.000078  2590 NMValidate.cpp:144] File appears to be INVALID

The reconstruction of the tested files seems to be working fine on siemens e7 reconstruction tool so I don't think the files are invalid.

Please let me know if #47 fixed the issues on your end @szho42

szho42 commented 3 years ago

@KrisThielemans I will test with #47 and see how it goes from my side. thanks for all the information. We might be able to do some debugging into the CPP code and see where this error happens.

@skarrea : are you using the raw data as one dcm file, or in paired form (one dcm header + a binary file, e.g. .bf file). In our case, we are using the second form (two paired files).

skarrea commented 3 years ago

@szho42 the latter: dcm + binary pair.

KrisThielemans commented 3 years ago

Maybe you could try it on some of our phantom data (link will expire end of the year or sooner). Data should be for instance in \20180822_CCP_CALIB_FFS.zip\ccp_calib_FFS\Biograph_mMR\PT\2018-08-22\194530_562000\Head_MRAC_PET_2bp_Raw_Data_18

KrisThielemans commented 3 years ago

It appears that I was testing our own Siemens data with an older version of the tools. Sorry about that.

Please confirm that this all works for you now, then I'll create a new release

skarrea commented 3 years ago

@KrisThielemans I ran a quick test on my own data now. It seems to be working fine for both sinogram raw data and listmode raw data (.dcm, .bf pair). I have not tested reconstructing the converted files yet, but I suspect it will work. Thanks!

KrisThielemans commented 3 years ago

I found another problem of undefined return, which could mean it refused to read a file. I've fixed it directly on master. Let me know.

szho42 commented 3 years ago

sorry for the late reply. Yes. I can confirm it works on my side as well, for extracting sinogram, listmode and normalization file. It does not support physio file (although that is not important). Thanks for the quick fix.

KrisThielemans commented 3 years ago

correct. contributions to handle the physio file welcome 😄

szho42 commented 3 years ago

yeah, that's something we can contribute definitely. We will commit a PR accordingly. Currently, we are testing with the converted interfiles with STIR, however we experience some issues. I will post the our errors in the STIR repo.