search

mobie / mobie-utils-python

Python tools for MoBIE
MIT License
9 stars 5 forks source link

running on other OS than Linux #122

Open tischi opened 10 months ago

tischi commented 10 months ago

@constantinpape @martinschorb

I did not manage to run the conversion to OME-Zarr on my Mac (see MoBIE mattermost channel).

This raises a general question: on which OS does the code here run?

constantinpape commented 10 months ago

In principle everything should run on Mac and Windows and there are unit tests for it that run through without issues. https://github.com/mobie/mobie-utils-python/actions/runs/7186625802/job/19572434858

Maybe these don't cover all the cases for ome-zarr conversion?! To fix this I would need a minimal self-contained reproducible example where it fails.

eschalnajmi commented 1 month ago

Hi! I'm facing a very similar issue right now where code works as a .py on linux but not on macOS or windows. However, the code works in mac and windows when run as .ipynb file. It seems to get stuck on the downscaling when run as a .py file containing the exact same code. Potentially jupyter notebook is adding a layer of virtualisation?

The exact error traceback: `/Users/najmie/miniconda3/envs/mobie3/bin/python /Users/najmie/Downloads/mobie_python_project/project_creation.py najmie@CH7H1X6L5F mobie_python_project % /Users/najmie/miniconda3/envs/mobie3/bin/python /Users/najmie/Downloads/mobie_python_project/project_creation.py Only one channel found in data/em_20nm_z_40_145.tif - no conversion needed :) /Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/parameter.py:296: UserWarning: Parameter "dtype" with value "None" is not of type string. warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value)) DEBUG: Checking if DownscalingWorkflow(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, target=local, dependency=DummyTask, input_path=data/em_20nm_z_40_145.tif, input_key=, scale_factors=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], halos=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], dtype=None, int_to_uint=False, metadata_format=ome.zarr, metadata_dict={"resolution": [10.0, 10.0, 10.0], "unit": "nanometer", "setup_name": "em_20nm_z_40_145"}, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key_prefix=, force_copy=False, skip_existing_levels=False, scale_offset=0) is complete /Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/parameter.py:296: UserWarning: Parameter "scale_factor" with value "(1, 2, 2)" is not of type string. warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value)) DEBUG: Checking if WriteDownscalingMetadata(tmp_folder=tmp_dataset1_em_20nm_z_40_145, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, scale_factors=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], dependency=DownscalingLocal, metadata_format=ome.zarr, metadata_dict={"resolution": [10.0, 10.0, 10.0], "unit": "nanometer", "setup_name": "em_20nm_z_40_145"}, output_key_prefix=, scale_offset=0, prefix=downscaling) is complete INFO: Informed scheduler that task DownscalingWorkflow_tmp_dataset1_emDummyTask_None_f01a3395a1 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s3, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s4, scale_factor=(1, 2, 2), scale_prefix=s4, halo=[1, 2, 2], effective_scale_factor=[1, 16, 16], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task WriteDownscalingMetadataDownscalingLocalresolution___ome_zarr_784397d098 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s2, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s3, scale_factor=(1, 2, 2), scale_prefix=s3, halo=[1, 2, 2], effective_scale_factor=[1, 8, 8], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal_d657c21b11 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s1, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s2, scale_factor=(1, 2, 2), scale_prefix=s2, halo=[1, 2, 2], effective_scale_factor=[1, 4, 4], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal_d82adb898c has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s0, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s1, scale_factor=(1, 2, 2), scale_prefix=s1, halo=[1, 2, 2], effective_scale_factor=[1, 2, 2], dimension_separator=/, dependency=CopyVolumeLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal_7fa9471af6 has status PENDING DEBUG: Checking if CopyVolumeLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=data/em_20nm_z_40_145.tif, input_key=, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s0, prefix=initial_scale, dtype=None, int_to_uint=False, fit_to_roi=False, effective_scale_factor=[], dimension_separator=/, dependency=DummyTask) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emCopyVolumeLocal_452731ba01 has status PENDING DEBUG: Checking if DummyTask() is complete INFO: Informed scheduler that task CopyVolumeLocal_tmp_dataset1_emDummyTask_07bedbbf06 has status PENDING INFO: Informed scheduler that task DummyTask__99914b932b has status DONE INFO: Done scheduling tasks INFO: Running Worker with 1 processes DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 7 INFO: [pid 11765] Worker Worker(salt=8065394207, workers=1, host=CH7H1X6L5F, username=najmie, pid=11765) running CopyVolumeLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=data/em_20nm_z_40_145.tif, input_key=, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s0, prefix=initial_scale, dtype=None, int_to_uint=False, fit_to_roi=False, effective_scale_factor=[], dimension_separator=/, dependency=DummyTask) Only one channel found in data/em_20nm_z_40_145.tif - no conversion needed :) /Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/parameter.py:296: UserWarning: Parameter "dtype" with value "None" is not of type string. warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value)) DEBUG: Checking if DownscalingWorkflow(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, target=local, dependency=DummyTask, input_path=data/em_20nm_z_40_145.tif, input_key=, scale_factors=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], halos=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], dtype=None, int_to_uint=False, metadata_format=ome.zarr, metadata_dict={"resolution": [10.0, 10.0, 10.0], "unit": "nanometer", "setup_name": "em_20nm_z_40_145"}, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key_prefix=, force_copy=False, skip_existing_levels=False, scale_offset=0) is complete /Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/parameter.py:296: UserWarning: Parameter "scale_factor" with value "(1, 2, 2)" is not of type string. warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value)) DEBUG: Checking if WriteDownscalingMetadata(tmp_folder=tmp_dataset1_em_20nm_z_40_145, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, scale_factors=[[1, 2, 2], [1, 2, 2], [1, 2, 2], [1, 2, 2]], dependency=DownscalingLocal, metadata_format=ome.zarr, metadata_dict={"resolution": [10.0, 10.0, 10.0], "unit": "nanometer", "setup_name": "em_20nm_z_40_145"}, output_key_prefix=, scale_offset=0, prefix=downscaling) is complete INFO: Informed scheduler that task DownscalingWorkflow_tmp_dataset1_emDummyTask_None_f01a3395a1 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s3, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s4, scale_factor=(1, 2, 2), scale_prefix=s4, halo=[1, 2, 2], effective_scale_factor=[1, 16, 16], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task WriteDownscalingMetadata_DownscalingLocal_resolution___ome_zarr_784397d098 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s2, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s3, scale_factor=(1, 2, 2), scale_prefix=s3, halo=[1, 2, 2], effective_scale_factor=[1, 8, 8], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal___d657c21b11 has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s1, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s2, scale_factor=(1, 2, 2), scale_prefix=s2, halo=[1, 2, 2], effective_scale_factor=[1, 4, 4], dimension_separator=/, dependency=DownscalingLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal_d82adb898c has status PENDING DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s0, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s1, scale_factor=(1, 2, 2), scale_prefix=s1, halo=[1, 2, 2], effective_scale_factor=[1, 2, 2], dimension_separator=/, dependency=CopyVolumeLocal) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emDownscalingLocal_7fa9471af6 has status PENDING DEBUG: Checking if CopyVolumeLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=data/em_20nm_z_40_145.tif, input_key=, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s0, prefix=initial_scale, dtype=None, int_to_uint=False, fit_to_roi=False, effective_scale_factor=[], dimension_separator=/, dependency=DummyTask) is complete INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_emCopyVolumeLocal_452731ba01 has status PENDING INFO: Informed scheduler that task CopyVolumeLocal_tmp_dataset1_emDummyTask_07bedbbf06 has status DONE INFO: Done scheduling tasks INFO: Running Worker with 1 processes DEBUG: Asking scheduler for work... DEBUG: Pending tasks: 6 INFO: [pid 11783] Worker Worker(salt=2341089979, workers=1, host=CH7H1X6L5F, username=najmie, pid=11783) running DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s0, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s1, scale_factor=(1, 2, 2), scale_prefix=s1, halo=[1, 2, 2], effective_scale_factor=[1, 2, 2], dimension_separator=/, dependency=CopyVolumeLocal) ERROR: [pid 11783] Worker Worker(salt=2341089979, workers=1, host=CH7H1X6L5F, username=najmie, pid=11783) failed DownscalingLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, input_key=s0, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s1, scale_factor=(1, 2, 2), scale_prefix=s1, halo=[1, 2, 2], effective_scale_factor=[1, 2, 2], dimension_separator=/, dependency=CopyVolumeLocal) Traceback (most recent call last): File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/worker.py", line 210, in run new_deps = self._run_get_new_deps() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/worker.py", line 138, in _run_get_new_deps task_gen = self.task.run() ^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 95, in run raise e File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 81, in run self.run_impl() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/downscaling/downscaling.py", line 184, in run_impl self.submit_jobs(n_jobs, self.scale_prefix) File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 567, in submit_jobs tasks = [pp.submit(self._submit, job_id, job_prefix) for job_id in range(n_jobs)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 567, in tasks = [pp.submit(self._submit, job_id, job_prefix) for job_id in range(n_jobs)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/process.py", line 808, in submit self._adjust_process_count() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/process.py", line 767, in _adjust_process_count self._spawn_process() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/process.py", line 785, in _spawn_process p.start() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/context.py", line 288, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/popen_fork.py", line 19, in init__ self._launch(process_obj) File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 42, in _launch prep_data = spawn.get_preparation_data(process_obj._name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/spawn.py", line 164, in get_preparation_data _check_not_importing_main() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/multiprocessing/spawn.py", line 140, in _check_not_importing_main raise RuntimeError(''' RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    To fix this issue, refer to the "Safe importing of main module"
    section in https://docs.python.org/3/library/multiprocessing.html

DEBUG: 1 running tasks, waiting for next task to finish INFO: Informed scheduler that task DownscalingLocal_tmp_dataset1_em__CopyVolumeLocal___452731ba01 has status FAILED DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: There are 6 pending tasks possibly being run by other workers DEBUG: There are 6 pending tasks unique to this worker DEBUG: There are 6 pending tasks last scheduled by this worker INFO: Worker Worker(salt=2341089979, workers=1, host=CH7H1X6L5F, username=najmie, pid=11783) was stopped. Shutting down Keep-Alive thread INFO: ===== Luigi Execution Summary =====

Scheduled 7 tasks of which:

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

Could not add em_20nm_z_40_145 to project :( ERROR: [pid 11765] Worker Worker(salt=8065394207, workers=1, host=CH7H1X6L5F, username=najmie, pid=11765) failed CopyVolumeLocal(tmp_folder=tmp_dataset1_em_20nm_z_40_145, max_jobs=1, config_dir=tmp_dataset1_em_20nm_z_40_145/configs, input_path=data/em_20nm_z_40_145.tif, input_key=, output_path=test_project/data/dataset1/images/ome-zarr/em_20nm_z_40_145.ome.zarr, output_key=s0, prefix=initial_scale, dtype=None, int_to_uint=False, fit_to_roi=False, effective_scale_factor=[], dimension_separator=/, dependency=DummyTask) concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 547, in _submit assert os.path.exists(script_path), script_path AssertionError: tmp_dataset1_em_20nm_z_40_145/copy_volume.py """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 81, in run self.run_impl() File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/copy_volume/copy_volume.py", line 163, in run_impl self.submit_jobs(n_jobs, self.prefix) File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 568, in submit_jobs [t.result() for t in tasks] File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 568, in [t.result() for t in tasks] ^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/_base.py", line 456, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/concurrent/futures/_base.py", line 401, in get_result raise self._exception AssertionError: tmp_dataset1_em_20nm_z_40_145/copy_volume.py

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/worker.py", line 210, in run new_deps = self._run_get_new_deps() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/luigi/worker.py", line 138, in _run_get_new_deps task_gen = self.task.run() ^^^^^^^^^^^^^^^ File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 92, in run self._write_log("task failed in run_impl with %s" % msg) File "/Users/najmie/miniconda3/envs/mobie3/lib/python3.11/site-packages/cluster_tools/cluster_tasks.py", line 281, in _write_log with open(log_file, 'a') as f: ^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'tmp_dataset1_em_20nm_z_40_145/copy_volume_initial_scale.log' DEBUG: 1 running tasks, waiting for next task to finish INFO: Informed scheduler that task CopyVolumeLocal_tmp_dataset1_em__DummyTask___07bedbbf06 has status FAILED DEBUG: Asking scheduler for work... DEBUG: Done DEBUG: There are no more tasks to run at this time DEBUG: There are 7 pending tasks possibly being run by other workers DEBUG: There are 7 pending tasks unique to this worker DEBUG: There are 7 pending tasks last scheduled by this worker INFO: Worker Worker(salt=8065394207, workers=1, host=CH7H1X6L5F, username=najmie, pid=11765) was stopped. Shutting down Keep-Alive thread INFO: ===== Luigi Execution Summary =====

Scheduled 8 tasks of which:

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

Could not add em_20nm_z_40_145 to project :( `

The code being run is in this repo: https://github.com/eschalnajmi/load_mobie_project

If anyone has figured out a way to fix this easily I would very very much appreciate it :)