Ability to pull meta tables from VizieR queries?

[guillochon]

I'm trying to pull the "mag" tables from the following catalog: http://vizier.u-strasbg.fr/viz-bin/VizieR-3?-source=J/A%2bA/415/863/table1&-out.max=50&-out.form=HTML%20Table&-oc.form=sexa

However, it appears that the "mag" tables are actually just auto-generated by VizieR based upon some text files that were included with the data release. Is there any way to use astroquery to access these text files in tabular form, or am I stuck downloading those files and parsing them myself?

[keflavich]

@guillochon could you provide an astroquery example that's getting you the wrong results? In principle, what you're asking for should be accessible, and I'll help you figure out what's going on, but it would be helpful to have what you've already tried as a starting point.

[guillochon]

@keflavich Sorry for the delay, the command is very simple:

result = Vizier.get_catalogs("J/A+A/415/863/table1")
table = result[list(result.keys())[0]]
print(table)

You can see the "comp", "mag", and "flux" columns are just entries with the strings "comp", "mag", and "flux", not the tables that are viewable when using the VizieR interface. I think this is because that data is just a series of text files included alongside the catalog that VizieR processes in an automatic way, but aren't themselves VizieR catalogs.

[keflavich]

I see. I am not sure if there is presently a way to access the tables at present, since the VOTables Vizier returns only include the text as you're seeing it. But I'll ask the Vizier team if there is a better way.

[keflavich]

I received a very helpful & rapid response from the Vizier team:

The page http://vizier.u-strasbg.fr/doc/asu-summary.htx summarizes ASU options available for VizieR.
If you add -out.meta=L to your URL parameters (eg  http://vizier.u-strasbg.fr/viz-bin/votable?-source=J%2FA%2BA%2F415%2F863%2Ftable1&-out.meta=L ), you will see a LINK element inside the FIELD mag :

    <FIELD name="mag" ucd="meta.ref" datatype="char" arraysize="3">
      <LINK content-type="timeSerie" href="http://vizier.u-strasbg.fr/viz-bin/vizExec/Vgraph?J/A+A/415/863/./mag/${SN}mag.dat&amp;"/>
    </FIELD>

The href value of the LINK is a parametrized URL: you'll have to replace ${SN} with the actual content of the field SN for the considered row. Thus, for the row whose SN value is 1996af, the URL will be http://vizier.u-strasbg.fr/viz-bin/vizExec/Vgraph?J/A+A/415/863/./mag/1996afmag.dat&amp;

You will notice that some of the LINK have a gref attribute whose value is an internal identifier used at CDS and which won't be very useful for you. We plan in the mid-term to replace these attributes with href ones.

It looks like it is possible to get these, but it will require some work on our part.

[guillochon]

@keflavich Any chance of this happening any time soon? In the process of trying to automatically pull data from VizieR sources like this and it's a real pain to have to manually download the files for each source. Would be great if there was a hacky work-around for the time-being.

[keflavich]

cc @bsipocz any thoughts on whether this is sprintable?

[bsipocz]

@keflavich - Would be great, but I'm trying to finish the talk first, but maybe tomorrow we'll have some time to check this out.

[ManonMarchand]

The support of TimeSeries was asked again by VizieR users (on this catalog https://cdsarc.cds.unistra.fr/viz-bin/cat/II/331 ). It's a bit hard. Should the function return a list of astropy tables?

EDIT: to implement: tag -out.meta=hul resolves the link so it's easier than the suggested -out.meta=L

[ManonMarchand]

(as a motivation, this shows that there are people downloading these: https://github.com/astropy/astropy/issues/16006)

PS: It's actually hard to do it here, we're having a look about what we can change upstream, and these might become datalinks. But it will take some time.