search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.38k stars 998 forks source link

api/tools/create is non-specific when encoding "id"s in the outputs dictionary return value #2423

Closed blankenberg closed 8 years ago

blankenberg commented 8 years ago

In api/tools/create, the output_dict is overly aggressively trying to encode values that end in '_id', this includes e.g. metadata values that should not be encoded. Only API relevant IDs should be encoded (e.g. the actual HDA ID). This causes an exception to be raised when attempting to encode a metadata value that is None or "" and has a name that ends in _id.

The exception being raise by the framework is galaxy.exceptions.MalformedId("Attempted to encode None id")

example pre-encoding output_dict, where a metadata parameter named metadata_table_id and having a valid and correct value of None causes this issue:

{'misc_blurb': u'queued', 'peek': None, 'update_time': '2016-05-26T18:16:49.485598', 'metadata_table_shape': None, 'metadata_table_format': None, 'data_type': 'galaxy.datatypes.registry.Biom1', 'tags': [], 'deleted': False, 'history_id': 180, 'metadata_table_columns': None, 'visible': True, 'genome_build': u'?', 'create_time': '2016-05-26T18:16:48.510818', 'hid': 10, 'file_size': 0, 'metadata_data_lines': None, 'file_ext': u'biom1', 'metadata_table_matrix_element_type': None, 'id': 2973, 'misc_info': None, 'hda_ldda': 'hda', 'history_content_type': 'dataset', 'metadata_table_format_url': None, 'name': u'Convert BIOM on data 1', 'metadata_table_type': None, 'uuid': '3775db41-0b8c-4d97-a1bd-69afcc7b7f0c', 'state': u'new', 'metadata_table_generated_by': None, 'metadata_table_rows': None, 'metadata_table_date': None, 'metadata_table_matrix_type': None, 'metadata_table_id': None, 'model_class': 'HistoryDatasetAssociation', 'metadata_dbkey': u'?', 'purged': False}

Line https://github.com/galaxyproject/galaxy/blob/c63bf911bc9cae4d0400a3f3fad03402c3f7a18a/lib/galaxy/webapps/galaxy/api/tools.py#L281 has the non-specific 'id' or endswith('_id') call to encode_dict_ids.

        outputs = rval[ 'outputs' ]
        # TODO:?? poss. only return ids?
        for output_name, output in output_datasets:
            output_dict = output.to_dict()
            # add the output name back into the output data structure
            # so it's possible to figure out which newly created elements
            # correspond with which tool file outputs
            output_dict[ 'output_name' ] = output_name
            outputs.append( trans.security.encode_dict_ids( output_dict ) )

The tool does run, however, the user gets an error message popup and, until manual refresh, it doesn't look like the tool worked.

This also causes tool tests to fail, eg: https://github.com/galaxyproject/tools-iuc/pull/770 / https://travis-ci.org/galaxyproject/tools-iuc/builds/133149615

ERROR: Convert BIOM ( biom_convert ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tmp_EztYh/galaxy-dev/test/functional/test_toolbox.py", line 299, in test_tool
    self.do_it( td )
  File "/tmp/tmp_EztYh/galaxy-dev/test/functional/test_toolbox.py", line 60, in do_it
    raise e
RunToolException: Error creating a job for these tool inputs - Attempted to encode None id
blankenberg commented 8 years ago

xref https://github.com/galaxyproject/galaxy/issues/2137