search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.41k stars 1.01k forks source link

Rerunning a tool fails to replace collection ellements (for mapped over job?) #15070

Open bernt-matthias opened 1 year ago

bernt-matthias commented 1 year ago

Describe the bug

Tried to restart one job in a collection that was created by mapping over an input list. The tool creates one dataset and one collection, i.e. the result of the mapping over is a collection and a nested collection.

Minor problem: all elements of the nested collection are shown successful (but they are empty and clicking them does nothing).

Major problem: restart with replacing collection elements does not work (see video)

Traceback (most recent call last):
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/runners/__init__.py", line 248, in prepare_job
    job_wrapper.prepare()
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/jobs/__init__.py", line 1201, in prepare
    tool_evaluator.set_compute_environment(compute_environment, get_special=get_special)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/evaluation.py", line 129, in set_compute_environment
    self.param_dict = self.build_param_dict(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/evaluation.py", line 175, in build_param_dict
    self.__populate_output_collection_wrappers(param_dict, output_collections, job_working_directory)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/evaluation.py", line 316, in __populate_output_collection_wrappers
    wrapper = DatasetCollectionWrapper(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/wrappers.py", line 638, in __init__
    ] = DatasetCollectionWrapper(
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/wrappers.py", line 642, in __init__
    element_wrapper = self._dataset_wrapper(element_object, identifier=element_identifier, **kwargs)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/wrappers.py", line 494, in _dataset_wrapper
    return DatasetFilenameWrapper(dataset, **kwargs)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/tools/wrappers.py", line 372, in __init__
    path_rewrite = compute_environment and compute_environment.output_path_rewrite(dataset)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/job_execution/compute_environment.py", line 122, in output_path_rewrite
    dataset_path = self.job_io.get_output_path(dataset)
  File "/gpfs1/data/galaxy_server/galaxy/lib/galaxy/job_execution/setup.py", line 217, in get_output_path
    raise KeyError(f"Couldn't find job output for [{dataset}] in [{self.output_hdas_and_paths.values()}]")
KeyError: "Couldn't find job output for [<galaxy.model.HistoryDatasetAssociation(737621) at 0x7f3786ad5bb0>] in [dict_values([(<galaxy.model.HistoryDatasetAssociation(738013) at 0x7f37861e6d00>, <galaxy.job_execution.datasets.DatasetPath object at 0x7f3786b7bc70>)])]"

Galaxy Version and/or server at which you observed the bug Galaxy Version: 22.01 Commit: 30dc1bf4f9

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

Collection elements should be replaced.

Screenshots

https://user-images.githubusercontent.com/25689525/204320932-c00fa8b9-7cf8-4e9b-843a-4a8dcc4373d7.mp4

bernt-matthias commented 1 year ago

Side note: dataset 255 is a list:list:list. So the tool generates list:list and mapping over creates list:list:list.