search

IMSY-DKFZ / simpa

The Simulation and Image Processing for Photonics and Acoustics (SIMPA) toolkit.
https://simpa.readthedocs.io/en/main/
Other
86 stars 17 forks source link

[New user] [Manual tests] 'time series data' error #389

Closed jeremieglt closed 1 month ago

jeremieglt commented 1 month ago

My problem: Hi, I need to simulate images with SIMPA for my thesis and I am stuck at the manual tests : I have all the modules installed correctly and the paths seem to be all set (Matlab seems to be opening correctly during the general overview). However, the images requiring the use of k wave are not produced, and I get the following error : "time series data" each time. It seems like the problem comes from the Matlab part or the transmission of the outptus from the Matlab scripts.

Environment:

What seems to be the the first test (Kwave acoustic convenience function):

2024-10-22 17:46:59,964 - DEBUG - Created C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\figures/ directory 2024-10-22 17:46:59,969 - DEBUG - Download folder with reference figures from nextcloud... 2024-10-22 17:47:03,149 - DEBUG - File downloaded successfully and stored at C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\downloaded.zip. 2024-10-22 17:47:03,184 - DEBUG - Files extracted to C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests 2024-10-22 17:47:03,184 - DEBUG - C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\downloaded.zip removed successfully. 2024-10-22 17:47:03,202 - DEBUG - Your simpa version does not match with the simpa version used for generating the reference figures 2024-10-22 17:47:03,202 - DEBUG - Neglect the following files: [] 2024-10-22 17:47:03,202 - DEBUG - Enter dir: acoustic_forward_models 2024-10-22 17:47:03,202 - DEBUG - Enter file: KWaveAcousticForwardConvenienceFunction.py 2024-10-22 17:47:03,202 - DEBUG - import module simpa_tests.manual_tests.acoustic_forward_models.KWaveAcousticForwardConvenienceFunction 2024-10-22 17:47:03,219 - DEBUG - Run KWaveAcousticForwardConvenienceFunction Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 1 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 551 ms retrieving fields ... transfer complete: 554 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 574 ms saving data to file ... compressing data [zlib] ...compression ratio: 89.1% after encoding: 118.8% saving data complete : 836 ms

Thanks in advance for your reading !

kdreher commented 1 month ago

Hi Jeremie, could you please share the whole Traceback (output) that you get? From what I can see, the one you posted above is only until MCX.

jeremieglt commented 1 month ago

Is there a way to output the full log ? Because in my data I don't have all the log files

jeremieglt commented 1 month ago

Because the complete log is too long and the terminal cuts it

jeremieglt commented 1 month ago

PS C:\Users\jeremie.gillet\simpa> & C:/Users/jeremie.gillet/AppData/Local/Microsoft/WindowsApps/python3.12.exe c:/Users/jeremie.gillet/simpa/simpa_tests/manual_tests/generate_overview.py 2024-10-23 11:02:38,222 - DEBUG - Created C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\figures/ directory 2024-10-23 11:02:38,238 - DEBUG - Download folder with reference figures from nextcloud... 2024-10-23 11:02:39,308 - DEBUG - File downloaded successfully and stored at C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\downloaded.zip. 2024-10-23 11:02:39,339 - DEBUG - Files extracted to C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests 2024-10-23 11:02:39,340 - DEBUG - C:\Users\jeremie.gillet\simpa\simpa_tests\manual_tests\downloaded.zip removed successfully. 2024-10-23 11:02:39,341 - DEBUG - Your simpa version does not match with the simpa version used for generating the reference figures 2024-10-23 11:02:39,341 - DEBUG - Neglect the following files: [] 2024-10-23 11:02:39,342 - DEBUG - Enter dir: acoustic_forward_models 2024-10-23 11:02:39,342 - DEBUG - Enter file: KWaveAcousticForwardConvenienceFunction.py 2024-10-23 11:02:39,342 - DEBUG - import module simpa_tests.manual_tests.acoustic_forward_models.KWaveAcousticForwardConvenienceFunction 2024-10-23 11:02:39,344 - DEBUG - Run KWaveAcousticForwardConvenienceFunction Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 1 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 541 ms retrieving fields ... transfer complete: 545 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 564 ms saving data to file ... compressing data [zlib] ...compression ratio: 89.1% after encoding: 118.8% saving data complete : 820 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 19011.41 photon/ms total simulated energy: 10000000.00 absorbed: 23.31475% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 2 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 475 ms retrieving fields ... transfer complete: 479 ms normalizing raw data ... source 1, normalization factor alpha=0.000000 data normalization complete : 499 ms saving data to file ... compressing data [zlib] ...compression ratio: 88.8% after encoding: 118.4% saving data complete : 780 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 21739.13 photon/ms total simulated energy: 10000000.00 absorbed: 17.78358% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined Environment variable +v not defined 2024-10-23 11:03:19,286 - DEBUG - Enter dir: executables 2024-10-23 11:03:19,286 - DEBUG - Enter file: MATLABAdditionalFlags.py 2024-10-23 11:03:19,286 - DEBUG - import module simpa_tests.manual_tests.executables.MATLABAdditionalFlags 2024-10-23 11:03:19,286 - DEBUG - Run MATLABAdditionalFlags Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 1 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 1455 ms retrieving fields ... transfer complete: 1457 ms normalizing raw data ... source 1, normalization factor alpha=0.000000 data normalization complete : 1466 ms saving data to file ... compressing data [zlib] ...compression ratio: 91.8% after encoding: 122.4% saving data complete : 1567 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 6939.63 photon/ms total simulated energy: 10000000.00 absorbed: 25.33510% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined Environment variable +v not defined 2024-10-23 11:03:32,613 - WARNING - No reference image found 2024-10-23 11:03:32,613 - DEBUG - Enter dir: image_reconstruction 2024-10-23 11:03:32,613 - DEBUG - Enter file: DelayAndSumReconstruction.py 2024-10-23 11:03:32,613 - DEBUG - import module simpa_tests.manual_tests.image_reconstruction.DelayAndSumReconstruction 2024-10-23 11:03:32,613 - DEBUG - Run DelayAndSumReconstruction Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 3 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 977 ms retrieving fields ... transfer complete: 985 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 1034 ms saving data to file ... compressing data [zlib] ...compression ratio: 90.0% after encoding: 120.1% saving data complete : 1730 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 10405.83 photon/ms total simulated energy: 10000000.00 absorbed: 48.93413% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined ############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 3 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 995 ms retrieving fields ... transfer complete: 1002 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 1053 ms saving data to file ... compressing data [zlib] ...compression ratio: 90.0% after encoding: 120.1% saving data complete : 1755 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 10214.50 photon/ms total simulated energy: 10000000.00 absorbed: 48.93413% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 2 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 244 ms retrieving fields ... transfer complete: 247 ms normalizing raw data ... source 1, normalization factor alpha=0.000001 data normalization complete : 268 ms saving data to file ... compressing data [zlib] ...compression ratio: 82.3% after encoding: 109.7% saving data complete : 541 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 43668.12 photon/ms total simulated energy: 10000000.00 absorbed: 0.43668% (loss due to initial specular reflection is excluded in the total) Environment variable -v not defined .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 3 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 977 ms retrieving fields ... transfer complete: 985 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 1036 ms saving data to file ... compressing data [zlib] ...compression ratio: 90.0% after encoding: 120.1% saving data complete : 1740 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 10405.83 photon/ms total simulated energy: 10000000.00 absorbed: 48.93413% (loss due to initial specular reflection is excluded in the total)

kdreher commented 1 month ago

Per default, the log is also dumped in a simpa.log file which is most likely located in your home directory, where your path_config.env is or in your SIMPA_SAVE_DIRECTORY

jeremieglt commented 1 month ago

Ok, you have the beginning there, and I'll send it to you when the simulation is finished

jeremieglt commented 1 month ago

I don't have any log file appearing. The only one that I have is in the data folder, and it's "AdditionalFlagsTest_output-DE-JEG-NB.log"

jeremieglt commented 1 month ago

I only have the .md and .html synthesis

jeremieglt commented 1 month ago

I don't have any log file appearing. The only one that I have is in the data folder, and it's "AdditionalFlagsTest_output-DE-JEG-NB.log"

Sorry, there is also "AdditionalFlagsTest_output.log" and "AdditionalFlagsTest.log"

jeremieglt commented 1 month ago

manual_tests_overview.md

Here is the overview file by the way

kdreher commented 1 month ago

Okay, maybe try running the optical and acoustic example. This should definitely create the simpa.log file somewhere, most likely in your home directory

jeremieglt commented 1 month ago

Nope, but I get this log :

2024-10-23 11:19:09,494 - DEBUG - Using $HOME$ path to search for config file: C:\Users\jeremie.gillet\path_config.env 2024-10-23 11:19:09,494 - DEBUG - Did not find path config in $HOME$: C:\Users\jeremie.gillet\path_config.env 2024-10-23 11:19:09,494 - DEBUG - Searching for path config in current working directory... 2024-10-23 11:19:09,494 - DEBUG - Found path_config.env in current working directory: C:\Users\jeremie.gillet\simpa\path_config.env 2024-10-23 11:19:09,495 - DEBUG - Retrieved SIMPA_SAVE_DIRECTORY=.\data 2024-10-23 11:19:09,498 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('Background', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'Background' has been given the value {'molecule_composition': [<simpa.utils.libraries.molecule_library.Molecule object at 0x00000172E14A70B0>], 'structure_type': 'Background'} 2024-10-23 11:19:09,501 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('muscle', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'muscle' has been given the value {('structure_start', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [0, 0, 0], ('structure_end', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [0, 0, 100], ('priority', <class 'numbers.Number'>): 1, ('molecule_composition', <class 'list'>): [<simpa.utils.libraries.molecule_library.Molecule object at 0x00000172DD0884D0>], ('consider_partial_volume', <class 'bool'>): True, ('adhere_to_deformation', <class 'bool'>): True, ('structure_type', <class 'str'>): 'HorizontalLayerStructure'} 2024-10-23 11:19:09,508 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('epidermis', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'epidermis' has been given the value {('structure_start', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [0, 0, 1], ('structure_end', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [0, 0, 1.1], ('priority', <class 'numbers.Number'>): 8, ('molecule_composition', <class 'list'>): [<simpa.utils.libraries.molecule_library.Molecule object at 0x00000172E1434C20>, <simpa.utils.libraries.molecule_library.Molecule object at 0x00000172DB638080>], ('consider_partial_volume', <class 'bool'>): True, ('adhere_to_deformation', <class 'bool'>): True, ('structure_type', <class 'str'>): 'HorizontalLayerStructure'} 2024-10-23 11:19:09,516 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('vessel_1', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'vessel_1' has been given the value {('structure_start', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [27.5, 0, 5], ('structure_end', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [27.5, 20, 5], ('structure_radius', (<class 'numbers.Number'>, <class 'numpy.ndarray'>)): 2, ('priority', <class 'numbers.Number'>): 3, ('molecule_composition', <class 'list'>): [<simpa.utils.libraries.molecule_library.Molecule object at 0x00000172DC933800>, <simpa.utils.libraries.molecule_library.Molecule object at 0x00000172DCDE4320>], ('consider_partial_volume', <class 'bool'>): True, ('adhere_to_deformation', <class 'bool'>): False, ('structure_type', <class 'str'>): 'CircularTubularStructure'} 2024-10-23 11:19:09,526 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('vessel_2', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'vessel_2' has been given the value {('structure_start', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [37.5, 0, 10], ('structure_end', (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>)): [37.5, 20, 10], ('structure_radius', (<class 'numbers.Number'>, <class 'numpy.ndarray'>)): 3, ('priority', <class 'numbers.Number'>): 3, ('molecule_composition', <class 'list'>): [<simpa.utils.libraries.molecule_library.Molecule object at 0x00000172E153A9F0>, <simpa.utils.libraries.molecule_library.Molecule object at 0x00000172E150EC00>], ('consider_partial_volume', <class 'bool'>): True, ('adhere_to_deformation', <class 'bool'>): False, ('structure_type', <class 'str'>): 'CircularTubularStructure'} 2024-10-23 11:19:09,527 - DEBUG - Retrieved MCX_BINARY_PATH=.\mcx\bin\mcx.exe 2024-10-23 11:19:09,527 - DEBUG - Retrieved MATLAB_BINARY_PATH=C:\Program Files\MATLAB\R2023b\bin\matlab.exe 2024-10-23 11:19:09,527 - DEBUG - Retrieved MATLAB_BINARY_PATH=C:\Program Files\MATLAB\R2023b\bin\matlab.exe 2024-10-23 11:19:09,527 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('sos', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'sos' has been given the value 1540 2024-10-23 11:19:09,527 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('alpha_coeff', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'alpha_coeff' has been given the value 0.01 2024-10-23 11:19:09,527 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('density', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'density' has been given the value 1000 2024-10-23 11:19:09,527 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('noise_initial_pressure', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'noise_initial_pressure' has been given the value {'noise_mean': 1, 'noise_std': 0.01, 'noise_mode': 'noise_mode_multiplicative', 'data_field': 'initial_pressure', 'noise_non_negativity_constraint': True} 2024-10-23 11:19:09,527 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('noise_time_series', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'noise_time_series' has been given the value {'noise_std': 1, 'noise_mode': 'noise_mode_additive', 'data_field': 'time_series_data'} 2024-10-23 11:19:09,542 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,542 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('data_field', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'data_field' has been given the value ['mua', 'mus', 'g', 'gamma', 'seg', 'oxy', 'bvf', 'density', 'sos', 'alpha_coeff', 'sensor_mask', 'directivity_angle', 'fluence', 'initial_pressure'] 2024-10-23 11:19:09,543 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('FieldOfViewCropping', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'FieldOfViewCropping' has been given the value {'data_field': ['mua', 'mus', 'g', 'gamma', 'seg', 'oxy', 'bvf', 'density', 'sos', 'alpha_coeff', 'sensor_mask', 'directivity_angle', 'fluence', 'initial_pressure']} 2024-10-23 11:19:09,543 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,543 - DEBUG - Saving settings dictionary... 2024-10-23 11:19:09,575 - DEBUG - Saving settings dictionary...[Done] 2024-10-23 11:19:09,575 - DEBUG - Running pipeline for wavelength 700nm... 2024-10-23 11:19:09,575 - DEBUG - Running <class 'simpa.core.simulation_modules.volume_creation_module.model_based_adapter.ModelBasedAdapter'> 2024-10-23 11:19:09,575 - INFO - VOLUME CREATION 2024-10-23 11:19:09,575 - DEBUG - Tags.SIMULATE_DEFORMED_LAYERS in self.component_settings is TRUE 2024-10-23 11:19:09,665 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,665 - DEBUG - This structure will simulate deformations: True 2024-10-23 11:19:09,666 - DEBUG - This structure's deformation functional: <scipy.interpolate._rgi.RegularGridInterpolator object at 0x00000172DCE33EC0> 2024-10-23 11:19:09,767 - DEBUG - <class 'simpa.utils.libraries.structure_library.HorizontalLayerStructure.HorizontalLayerStructure'> 2024-10-23 11:19:09,786 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,786 - DEBUG - This structure will simulate deformations: False 2024-10-23 11:19:09,786 - DEBUG - This structure's deformation functional: None 2024-10-23 11:19:09,837 - DEBUG - <class 'simpa.utils.libraries.structure_library.CircularTubularStructure.CircularTubularStructure'> 2024-10-23 11:19:09,868 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,869 - DEBUG - This structure will simulate deformations: False 2024-10-23 11:19:09,869 - DEBUG - This structure's deformation functional: None 2024-10-23 11:19:09,888 - DEBUG - <class 'simpa.utils.libraries.structure_library.CircularTubularStructure.CircularTubularStructure'> 2024-10-23 11:19:09,903 - DEBUG - Processing is done on cuda 2024-10-23 11:19:09,905 - DEBUG - This structure will simulate deformations: True 2024-10-23 11:19:09,905 - DEBUG - This structure's deformation functional: <scipy.interpolate._rgi.RegularGridInterpolator object at 0x00000172DCFA3DA0> 2024-10-23 11:19:09,942 - DEBUG - <class 'simpa.utils.libraries.structure_library.HorizontalLayerStructure.HorizontalLayerStructure'> 2024-10-23 11:19:10,003 - DEBUG - Processing is done on cuda 2024-10-23 11:19:10,003 - DEBUG - This structure will simulate deformations: True 2024-10-23 11:19:10,003 - DEBUG - This structure's deformation functional: <scipy.interpolate._rgi.RegularGridInterpolator object at 0x00000172DCDB1F40> 2024-10-23 11:19:10,025 - DEBUG - <class 'simpa.utils.libraries.structure_library.BackgroundStructure.Background'> 2024-10-23 11:19:10,383 - DEBUG - Running <class 'simpa.core.simulation_modules.optical_module.mcx_adapter.MCXAdapter'> 2024-10-23 11:19:10,383 - INFO - Simulating the optical forward process... 2024-10-23 11:19:10,581 - DEBUG - [-62. 50.5 0.5] {'Session': {'ID': '.\data/CompletePipelineExample_4711_output', 'DoAutoThread': 1, 'Photons': 10000000.0, 'DoMismatch': 0, 'RNGSeed': 4711}, 'Forward': {'T0': 0, 'T1': 5e-09, 'Dt': 5e-09}, 'Optode': {'Source': {'Type': 'slit', 'Pos': [-62.0, 50.5, 0.5], 'Dir': [0.0, 0.0, 1.0], 'Param1': [500.0, 0.0, 0.0, 0], 'Param2': [0, 0, 0, 0]}}, 'Domain': {'OriginType': 0, 'LengthUnit': 0.2, 'Media': [{'mua': 0, 'mus': 0, 'g': 1, 'n': 1}, {'mua': 1, 'mus': 1, 'g': 0.9, 'n': 1}], 'MediaFormat': 'muamus_float', 'Dim': [375, 100, 125], 'VolumeFile': '.\data/CompletePipelineExample_4711.bin'}} 2024-10-23 11:19:10,581 - INFO - ['.\mcx\bin\mcx.exe', '-f', '.\data/CompletePipelineExample_4711.json', '-O', 'F', '-a', '1', '-F', 'jnii', '--printgpu']
============================= GPU Information ================================ Device 1 of 1: NVIDIA GeForce RTX 4060 Ti Compute Capability: 8.9 Global Memory: 17175150592 B Constant Memory: 65536 B Shared Memory: 49152 B Registers: 65536 Clock Speed: 2.57 GHz Number of SMs: 34 Number of Cores: 4352 Auto-thread: 139264 Auto-block: 64 .############################################################################### .# Monte Carlo eXtreme (MCX) -- CUDA # .# Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu> # .# https://mcx.space/ & https://neurojson.io/ # .# # .# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org # .# Department of Bioengineering, Northeastern University, Boston, MA, USA # .############################################################################### .# The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365 # .############################################################################### .# Open-source codes and reusable scientific data are essential for research, # .# MCX proudly developed human-readable JSON-based data formats for easy reuse.# .# # .#Please visit our free scientific data sharing portal at https://neurojson.io/# .# and consider sharing your public datasets in standardized JSON/JData format # .############################################################################### $Rev::f959c7$v2024.6 $Date::2024-06-22 15:23:34 -04$ by $Author::Qianqian Fang$ .###############################################################################

GPU=1 (NVIDIA GeForce RTX 4060 Ti) threadph=71 extra=112256 np=10000000 nthread=139264 maxgate=1 repetition=1 initializing streams ... init complete : 2 ms requesting 1280 bytes of shared memory launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ... simulation run# 1 ... kernel complete: 621 ms retrieving fields ... transfer complete: 627 ms normalizing raw data ... source 1, normalization factor alpha=0.000002 data normalization complete : 663 ms saving data to file ... compressing data [zlib] ...compression ratio: 88.8% after encoding: 118.4% saving data complete : 1166 ms

simulated 10000000 photons (10000000) with 139264 threads (repeat x1) MCX simulation speed: 16501.65 photon/ms total simulated energy: 10000000.00 absorbed: 23.33845% (loss due to initial specular reflection is excluded in the total) fluence.shape (375, 100, 125, 1, 1) fluence.shape (375, 100, 125) 2024-10-23 11:19:12,269 - INFO - Simulating the optical forward process...[Done] 2024-10-23 11:19:12,281 - DEBUG - Running <class 'simpa.core.processing_components.monospectral.noise.gaussian_noise.GaussianNoise'> 2024-10-23 11:19:12,281 - INFO - Applying Gaussian Noise Model... 2024-10-23 11:19:12,282 - DEBUG - Noise model mode: noise_mode_multiplicative 2024-10-23 11:19:12,282 - DEBUG - Noise model mean: 1 2024-10-23 11:19:12,283 - DEBUG - Noise model std: 0.01 2024-10-23 11:19:12,283 - DEBUG - Noise model non-negative: True 2024-10-23 11:19:12,339 - INFO - Applying Gaussian Noise Model...[Done] 2024-10-23 11:19:12,339 - DEBUG - Running <class 'simpa.core.simulation_modules.acoustic_module.k_wave_adapter.KWaveAdapter'> 2024-10-23 11:19:12,339 - INFO - Simulating the acoustic forward process... 2024-10-23 11:19:12,339 - DEBUG - OPTICAL_PATH: /simulations/optical_forward_model_output/ 2024-10-23 11:19:12,456 - DEBUG - field_of_view_extent: [-15 15 0 0 0 20] 2024-10-23 11:19:12,457 - WARNING - The key for the Settings dictionary should be a tuple in the form of ('detector_element_width_mm', (data_type_1, data_type_2, ...)). The tuple of data types specifies all possible types, the value can have. The key 'detector_element_width_mm' has been given the value 0.24 2024-10-23 11:19:12,457 - DEBUG - Added parameter ('voxel_spacing_mm', <class 'numbers.Number'>) to kWave settings via global_settings 2024-10-23 11:19:12,457 - WARNING - Did not find parameter ('model_sensor_frequency_response', <class 'bool'>) in any settings. 2024-10-23 11:19:12,457 - DEBUG - Added parameter ('medium_alpha_power', <class 'numbers.Number'>) to kWave settings via componentsettings 2024-10-23 11:19:12,457 - DEBUG - Added parameter ('gpu', (<class 'bool'>, <class 'numpy.bool'>)) to kWave settings via global_settings 2024-10-23 11:19:12,457 - DEBUG - Added parameter ('pml_inside', <class 'bool'>) to kWave settings via component_settings 2024-10-23 11:19:12,458 - DEBUG - Added parameter ('pml_alpha', <class 'numbers.Number'>) to kWave settings via component_settings 2024-10-23 11:19:12,458 - DEBUG - Added parameter ('plot_pml', <class 'bool'>) to kWave settings via component_settings 2024-10-23 11:19:12,458 - DEBUG - Added parameter ('recordmovie', (<class 'bool'>, <class 'numpy.bool'>)) to kWave settings via component_settings 2024-10-23 11:19:12,458 - DEBUG - Added parameter ('movie_name', <class 'str'>) to kWave settings via component_settings 2024-10-23 11:19:12,458 - DEBUG - Added parameter ('acoustic_logscale', (<class 'bool'>, <class 'numpy.bool'>)) to kWave settings via component_settings 2024-10-23 11:19:12,458 - WARNING - Did not find parameter sensor_directivity_pattern in any settings. 2024-10-23 11:19:12,458 - WARNING - Did not find parameter ('initial_pressure_smoothing', <class 'bool'>) in any settings. 2024-10-23 11:19:12,495 - INFO - Simulating 2D.... 2024-10-23 11:19:12,581 - INFO - ['C:\Program Files\MATLAB\R2023b\bin\matlab.exe', '-nodisplay', '-nosplash', '-automation', '-wait', '-r', "addpath('C:\Users\jeremie.gillet\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\simpa\core\simulation_modules\acoustic_module');simulate_2D('C:\Users\jeremie.gillet\simpa\data\CompletePipelineExample_4711.hdf5.mat');exit;"] Traceback (most recent call last): File "c:\Users\jeremie.gillet\simpa\simpa_examples\optical_and_acoustic_simulation.py", line 222, in run_optical_and_acoustic_simulation(spacing=config.spacing, path_manager=config.path_manager, File "c:\Users\jeremie.gillet\simpa\simpa_examples\optical_and_acoustic_simulation.py", line 198, in run_optical_and_acoustic_simulation sp.simulate(SIMULATION_PIPELINE, settings, device) File "C:\Users\jeremie.gillet\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\simpa\core\simulation.py", line 97, in simulate pipeline_element.run(digital_device_twin) File "C:\Users\jeremie.gillet\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\simpa\core\simulation_modules\acoustic_module\acoustic_adapter_base.py", line 74, in run time_series_data = self.forward_model(_device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\jeremie.gillet\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\simpa\core\simulation_modules\acoustic_module\k_wave_adapter.py", line 118, in forward_model time_series_data, global_settings = self.k_wave_acoustic_forward_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\jeremie.gillet\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\simpa\core\simulation_modules\acoustic_module\k_wave_adapter.py", line 248, in k_wave_acoustic_forward_model raw_time_series_data = sio.loadmat(optical_path)[Tags.DATA_FIELD_TIME_SERIES_DATA]


KeyError: 'time_series_data'
kdreher commented 1 month ago

Okay, this looks like everything runs smoothly until MATLAB/k-wave is called. A couple of things to check:

jeremieglt commented 1 month ago

It appears to be working out now. The k-wave binaries were not located at the correct place. Thanks for your time and for pointing out the source of the problem !

kdreher commented 1 month ago

Glad to hear it works now :)