Query with asterisk does not work in TAP

[olebole]

When I include an asterisk as 'catch-all' in a TAP query, an error is returned:

>>> import pyvo
>>> tap_service = pyvo.dal.TAPService('https://gaia.aip.de/tap')
>>> tap_service.run_sync('SELECT TOP 10 * FROM GDR2.GAIA_SOURCE;')               
---------------------------------------------------------------------------
E20                                       Traceback (most recent call last)
/usr/lib/python3/dist-packages/pyvo/dal/query.py in execute_votable(self)
    236         try:
--> 237             return votableparse(self.execute_stream().read)
    238         except Exception as e:

/usr/lib/python3/dist-packages/astropy/utils/decorators.py in wrapper(*args, **kwargs)
    520 
--> 521             return function(*args, **kwargs)
    522 

/usr/lib/python3/dist-packages/astropy/io/votable/table.py in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
    166         return tree.VOTableFile(
--> 167             config=config, pos=(1, 1)).parse(iterator, config)
    168 

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in parse(self, iterator, config)
   3567                 tag_mapping.get(tag, self._add_unknown_tag)(
-> 3568                     iterator, tag, data, config, pos)
   3569             elif tag == 'DESCRIPTION':

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in _add_resource(self, iterator, tag, data, config, pos)
   3477         self.resources.append(resource)
-> 3478         resource.parse(self, iterator, config)
   3479 

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in parse(self, votable, iterator, config)
   3279                 tag_mapping.get(tag, self._add_unknown_tag)(
-> 3280                     iterator, tag, data, config, pos)
   3281             elif tag == 'DESCRIPTION':

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in _add_table(self, iterator, tag, data, config, pos)
   3224         self.tables.append(table)
-> 3225         table.parse(iterator, config)
   3226 

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in parse(self, iterator, config)
   2499                         self.array = self._parse_tabledata(
-> 2500                             iterator, colnumbers, fields, config)
   2501                         break

/usr/lib/python3/dist-packages/astropy/io/votable/tree.py in _parse_tabledata(self, iterator, colnumbers, fields, config)
   2584                             if i >= len(fields):
-> 2585                                 vo_raise(E20, len(fields), config, pos)
   2586 

/usr/lib/python3/dist-packages/astropy/io/votable/exceptions.py in vo_raise(exception_class, args, config, pos)
    100         config = {}
--> 101     raise exception_class(args, config, pos)
    102 

E20: None:16:28: E20: Data has more columns than are defined in the header (0)

During handling of the above exception, another exception occurred:

DALFormatError                            Traceback (most recent call last)
<ipython-input-3-d862315041f5> in <module>
----> 1 tap_service.run_sync('SELECT TOP 10 * FROM GDR2.GAIA_SOURCE;')

/usr/lib/python3/dist-packages/pyvo/dal/tap.py in run_sync(self, query, language, maxrec, uploads, **keywords)
    215         return self.create_query(
    216             query, language=language, maxrec=maxrec, uploads=uploads,
--> 217             **keywords).execute()
    218 
    219     # alias for service discovery

/usr/lib/python3/dist-packages/pyvo/dal/tap.py in execute(self)
    906            for errors parsing the VOTable response
    907         """
--> 908         return TAPResults(self.execute_votable(), url=self.queryurl, session=self._session)
    909 
    910     def submit(self):

/usr/lib/python3/dist-packages/pyvo/dal/query.py in execute_votable(self)
    238         except Exception as e:
    239             self.raise_if_error()
--> 240             raise DALFormatError(e, self.queryurl)
    241 
    242     def raise_if_error(self):

DALFormatError: E20: None:16:28: E20: Data has more columns than are defined in the header (0)

This is independent of whether one chooses ADQL or PostgreSQL as language. It does not happen in the Web interface. I tried it on several Daiquiri instances (MuseWIDE, Gaia, Rave, Applause), all with the same result.

[jochenklar]

Hi @olebole , this is a prolem with the metadata for this table. E.g. SELECT TOP 10 * FROM TAP_SCHEMA.schemas works. One idea would be to use the tap interface via a plain http client and then run stilts votlint on it.

[harenk]

The '' is not the problem: qstr[0]="select TOP 10 from gdr2.gaia_source where priam_flags IS NOT NULL AND flame_flags IS NOT NULL" qstr[1]="select TOP 10 * from gdr2.sso_source"

Both return with no error. There are also examples for other daiquiri instances It's the handling of NULL values, here for an int column. The error message is misleading.....

[kimakan]

AFAIK, the error does not occur anymore. I'm not quiet sure when this was fixed.