search

i4Ds / Karabo-Pipeline

The Karabo Pipeline can be used as Digital Twin for SKA
https://i4ds.github.io/Karabo-Pipeline/
MIT License
11 stars 4 forks source link

Dask-related test failures in pull request #571

Open mpluess opened 5 months ago

mpluess commented 5 months ago

After creating a pull request unrelated to Dask, two tests involving Dask failed. See this log for details: https://github.com/i4Ds/Karabo-Pipeline/actions/runs/9172253662/job/25218346513?pr=569

To avoid the test failures, I disabled Dask in the two affected tests, see comments with "TODO DASK_TEST_ISSUE" prefix in the code base.

This should be looked at in detail though to make sure Dask is working as expected.

mpluess commented 5 months ago

Something seems to be off indeed with Dask. In source_detection.ipynb, running the following code twice: restored = RascilImageCleaner( RascilImageCleanerConfig( imaging_npixel=imaging_npixel, imaging_cellsize=imaging_cellsize, ingest_vis_nchan=16, clean_nmajor=1, clean_algorithm="mmclean", clean_scales=[10, 30, 60], clean_threshold=0.12e-3, clean_nmoment=5, clean_psf_support=640, clean_restored_output="integrated", use_dask=True, ) ).create_cleaned_image( ms_file_path=visibility_askap.ms_file_path, ) results in a "RuntimeError: IOLoop is closed" as well (first run is fine, second one crashes).