search

IMSY-DKFZ / simpa

The Simulation and Image Processing for Photonics and Acoustics (SIMPA) toolkit.
https://simpa.readthedocs.io/en/main/
Other
73 stars 17 forks source link

Segmentation Loader does not work with Benchmarking Script #364

Open jgroehl opened 1 month ago

jgroehl commented 1 month ago

Describe the bug

There is a silent error in the benchmarking script where the segmentation loader is not running through as it produces the following error message:

Error message:

Traceback (most recent call last): File "/home/kris/Projects/SIMPA/Merging_Session/simpa/simpa_examples/segmentation_loader.py", line 113, in run_segmentation_loader(spacing=config.spacing, path_manager=config.path_manager, visualise=config.visualise) File "/home/kris/Projects/SIMPA/Merging_Session/simpa/simpa_examples/segmentation_loader.py", line 92, in run_segmentation_loader sp.simulate(pipeline, settings, sp.RSOMExplorerP50(element_spacing_mm=1.0)) File "/home/kris/Projects/SIMPA/Merging_Session/simpa/simpa/core/simulation.py", line 94, in simulate pipeline_element.run(digital_device_twin) File "/home/kris/Projects/SIMPA/Merging_Session/simpa/simpa/core/simulation_modules/volume_creation_module/init.py", line 64, in run volumes = self.create_simulation_volume() File "/home/kris/Projects/SIMPA/Merging_Session/simpa/simpa/core/simulation_modules/volume_creation_module/volume_creation_module_segmentation_based_adapter.py", line 32, in create_simulation_volume raise ValueError("x_dim of volumes and segmentation must perfectly match but was {} and {}" ValueError: x_dim of volumes and segmentation must perfectly match but was 320 and 160

We disabled the running of this test for now, but the test has to be fixed AND the integration with the benchmark has to be tested.

Specify a priority (low, medium, high)

LOW

To Reproduce Run the benchmarking script and add the segmentation loader example to the performance_check.py script. We only ran with 0.3 spacing and no step.

Current Behavior A silent error in the benchmarking script. There is an error message, but the script is running though.

Expected behavior The test is actually run during benchmark.

Screenshots See above

Environment (please complete the following information):

jgroehl commented 1 month ago

It seems that an entry for the segmentation loader is also included into the results list, when the script is not explicitly included in performance_check.py:L31-34:

examples = [simpa_examples.run_linear_unmixing, simpa_examples.run_minimal_optical_simulation,
            simpa_examples.run_minimal_optical_simulation_uniform_cube, simpa_examples.run_msot_invision_simulation,
            simpa_examples.run_optical_and_acoustic_simulation,
            simpa_examples.run_perform_iterative_qPAI_reconstruction, **simpa_examples.segmentation_loader**]

Why does this happen? Can either everything or nothing be dynamically loaded?