extended-metadata: `AttributeError: 'MetadataCollection' object has no attribute 'bam_index'`

[bgruening]

The tool Split BAM by Reference into dataset list collection https://usegalaxy.eu/root?tool_id=toolshed.g2.bx.psu.edu/repos/iuc/bamtools_split_ref/bamtools_split_ref/2.5.1+galaxy0 generates the following stack trace on EU.

Its a bit hard to debug as the tool only generates one collection and this is empty so there is no stderr/stdout attached. However, looking at the JWD the results do exist.

EU is running celery and metadata_strategy: extended

galaxy@sn06:/data/jwd/main/052/496/52496496/_working$ ll outputs/
total 217624
-rw-rw-r-- 1 galaxy galaxy 15603176 Nov 19 22:41 split_bam00001Samtools_view_on_data_81__filtered_alignments.PB2.bam
-rw-rw-r-- 1 galaxy galaxy  4540697 Nov 19 22:41 split_bam00002Samtools_view_on_data_81__filtered_alignments.PB1.bam
-rw-rw-r-- 1 galaxy galaxy 32089329 Nov 19 22:41 split_bam00003Samtools_view_on_data_81__filtered_alignments.PA.bam
-rw-rw-r-- 1 galaxy galaxy  7704305 Nov 19 22:41 split_bam00004Samtools_view_on_data_81__filtered_alignments.HA.bam
-rw-rw-r-- 1 galaxy galaxy 32497085 Nov 19 22:41 split_bam00005Samtools_view_on_data_81__filtered_alignments.NP.bam
-rw-rw-r-- 1 galaxy galaxy 39097028 Nov 19 22:41 split_bam00006Samtools_view_on_data_81__filtered_alignments.NA.bam
-rw-rw-r-- 1 galaxy galaxy 39352065 Nov 19 22:41 split_bam00007Samtools_view_on_data_81__filtered_alignments.MP.bam
-rw-rw-r-- 1 galaxy galaxy 51032734 Nov 19 22:41 split_bam00008Samtools_view_on_data_81__filtered_alignments.NS.bam

Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: Traceback (most recent call last):
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 119, in __getitem__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     self.__getattribute__(key)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: AttributeError: 'MetadataCollection' object has no attribute 'bam_index'
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: During handling of the above exception, another exception occurred:
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: Traceback (most recent call last):
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     context = constructor(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 1114, in _init_compiled
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     param = {
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 1115, in <dictcomp>
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     key: processors[key](compiled_params[key])
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1636, in process
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return process_param(value, dialect)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/custom_types.py", line 382, in process_bind_param
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     value = uuid.UUID(value)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/usr/lib64/python3.8/uuid.py", line 168, in __init__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     hex = hex.replace('urn:', '').replace('uuid:', '')
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: AttributeError: 'dict' object has no attribute 'replace'
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: The above exception was the direct cause of the following exception:
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: Traceback (most recent call last):
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 122, in __getitem__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return self.__getattr__(key)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 146, in __getattr__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return self.spec[name].wrap(self.parent._metadata[name], self._object_session(self.parent))
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 410, in wrap
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return self.param.wrap(value, session)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 604, in wrap
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return session.query(galaxy.model.MetadataFile).filter_by(uuid=value).one()
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2865, in one
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return self._iter().one()
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2903, in _iter
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     result = self.session.execute(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 1712, in execute
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     result = conn._execute_20(statement, params or {}, execution_options)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return meth(self, args_10style, kwargs_10style, execution_options)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return connection._execute_clauseelement(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     ret = self._execute_context(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1725, in _execute_context
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     self._handle_dbapi_exception(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     util.raise_(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     raise exception
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     context = constructor(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 1114, in _init_compiled
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     param = {
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 1115, in <dictcomp>
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     key: processors[key](compiled_params[key])
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/venv/lib64/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1636, in process
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return process_param(value, dialect)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/custom_types.py", line 382, in process_bind_param
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     value = uuid.UUID(value)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/usr/lib64/python3.8/uuid.py", line 168, in __init__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     hex = hex.replace('urn:', '').replace('uuid:', '')
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: sqlalchemy.exc.StatementError: (builtins.AttributeError) 'dict' object has no attribute 'replace'
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: [SQL: SELECT metadata_file.id AS metadata_file_id, metadata_file.name AS metadata_file_name, metadata_file.hda_id AS metadata_file_hda_id, metadata_file.lda_id AS metadata_file_lda_id, metadata_file.create_time AS metadata_file_create_time, metadata_file.update_time AS metadata_file_update_time, metadata_file.object_store_id AS metadata_file_object_store_id, metadata_file.uuid AS metadata_file_uuid, metadata_file.deleted AS metadata_file_deleted, metadata_file.purged AS metadata_file_purged
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: FROM metadata_file
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: WHERE metadata_file.uuid = %(uuid_1)s]
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: During handling of the above exception, another exception occurred:
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: Traceback (most recent call last):
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/jobs/runners/__init__.py", line 628, in _finish_or_resubmit_job
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     job_wrapper.finish(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/jobs/__init__.py", line 1809, in finish
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     import_model_store.perform_import(history=job.history, job=job)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 286, in perform_import
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     self._import_datasets(object_import_tracker, datasets_attrs, history, new_history, job)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 581, in _import_datasets
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     self.app.datatypes_registry.set_external_metadata_tool.regenerate_imported_metadata_if_needed(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/tools/__init__.py", line 2901, in regenerate_imported_metadata_if_needed
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     job, *_ = self.tool_action.execute_via_app(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/tools/actions/metadata.py", line 117, in execute_via_app
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     cmd_line = external_metadata_wrapper.setup_external_metadata(
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/metadata/__init__.py", line 215, in setup_external_metadata
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     output_collections[name] = {
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 2111, in __exit__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     self._finalize()
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 1990, in _finalize
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     datasets_attrs_out.write(to_json(datasets_attrs))
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 1986, in to_json
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return json_encoder.encode([a.serialize(self.security, self.serialization_options) for a in attributes])
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/store/__init__.py", line 1986, in <listcomp>
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return json_encoder.encode([a.serialize(self.security, self.serialization_options) for a in attributes])
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/__init__.py", line 348, in serialize
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     return self._serialize(id_encoder, serialization_options)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/__init__.py", line 4659, in _serialize
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     rval = super()._serialize(id_encoder, serialization_options)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/__init__.py", line 4379, in _serialize
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     metadata = _prepare_metadata_for_serialization(id_encoder, serialization_options, self.metadata)
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/__init__.py", line 9556, in _prepare_metadata_for_serialization
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     for name, value in metadata.items():
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/usr/lib64/python3.8/_collections_abc.py", line 744, in __iter__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     yield (key, self._mapping[key])
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:   File "/opt/galaxy/server/lib/galaxy/model/metadata.py", line 124, in __getitem__
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]:     raise KeyError
Nov 19 22:48:39 sn06.galaxyproject.eu python[1745890]: KeyError

[bgruening]

Interesting if I disable metadata_strategy: extended, so I assume it takes the default, it is working as expected.

[mvdbeek]

Its a bit hard to debug as the tool only generates one collection and this is empty so there is no stderr/stdout attached.

Did you check the job in the database ? The traceback might be attached there, and the UI is just missing a link the to component (https://github.com/galaxyproject/galaxy/issues/14980)

[mvdbeek]

only generates one collection

I think the tool generates no collection ? https://github.com/galaxyproject/tools-iuc/blob/master/tool_collections/bamtools/bamtools_split/bamtools_split.xml#L46 ... but you should have a hidden dataset in the history if the discovery fails.

Fix is in https://github.com/galaxyproject/galaxy/pull/15038

[bgruening]

It is this tool https://github.com/galaxyproject/tools-iuc/blob/master/tool_collections/bamtools/bamtools_split/bamtools_split_ref.xml#L42

Failed collection:

Correct collection without extended metadata:

Thanks for the quick fix will try it now ...

[mvdbeek]

Fixed in https://github.com/galaxyproject/galaxy/pull/15038