Look at possibilities to vectorize queries

[bsipocz]

It would be nice to look at the possibilities of querying multiple objects at the same time, as doing it sequentially one-by-one has huge overheads when one has thousands of objects. When a module supports it check it out; or if isn't possible due to the remote API, cross it over.

• [ ] astroquery/alfalfa
• [ ] astroquery/alma
• [ ] astroquery/astrometry_net
• [ ] astroquery/atomic
• [ ] astroquery/besancon
• [ ] astroquery/cadc
• [ ] astroquery/casda
• [ ] astroquery/cds
• [ ] astroquery/cosmosim
• [ ] astroquery/dace
• [ ] astroquery/esa/hsa
• [ ] astroquery/esa/hubble
• [ ] astroquery/esa/iso
• [ ] astroquery/esa/jwst
• [ ] astroquery/esa/xmm_newton
• [ ] astroquery/esasky
• [ ] astroquery/eso
• [ ] astroquery/exoplanet_orbit_database
• [ ] astroquery/fermi
• [ ] astroquery/gaia
• [ ] astroquery/gama
• [ ] astroquery/gemini
• [ ] astroquery/heasarc
• [ ] astroquery/hips2fits
• [ ] astroquery/hitran
• [ ] astroquery/image_cutouts/first
• [ ] astroquery/imcce
• [ ] astroquery/ipac/irsa
• [ ] astroquery/ipac/irsa/ibe
• [ ] astroquery/ipac/irsa/irsa_dust
• [ ] astroquery/ipac/irsa/sha
• [ ] astroquery/ipac/ned
• [ ] astroquery/ipac/nexsci/nasa_exoplanet_archive
• [ ] astroquery/jplhorizons
• [ ] astroquery/jplsbdb
• [ ] astroquery/jplspec
• [ ] astroquery/lamda
• [ ] astroquery/linelists/cdms
• [ ] astroquery/magpis
• [ ] astroquery/mast
• [ ] astroquery/mpc
• [ ] astroquery/nasa_ads
• [ ] astroquery/nist
• [ ] astroquery/nvas
• [ ] astroquery/oac
• [ ] astroquery/ogle
• [ ] astroquery/open_exoplanet_catalogue
• [ ] astroquery/sdss
• [ ] astroquery/sha
• [ ] astroquery/simbad
• [ ] astroquery/skyview
• [ ] astroquery/solarsystem/neodys
• [ ] astroquery/solarsystem/pds
• [ ] astroquery/splatalogue
• [ ] astroquery/svo_fps
• [ ] astroquery/ukidss
• [ ] astroquery/utils
• [ ] astroquery/utils/tap
• [ ] astroquery/vamdc
• [ ] astroquery/vizier
• [ ] astroquery/vo_conesearch
• [ ] astroquery/vsa
• [ ] astroquery/wfau
• [ ] astroquery/xmatch

[jwoillez]

Vizier already supports this. You can even use the output of a first query as the input of another query. We have example somewhere in the documentation.

-Julien

On Apr 26, 2016, at 17:51, Brigitta Sipocz notifications@github.com wrote:

It would be nice to look at the possibilities of querying multiple objects at the same time, as doing it sequentially one-by-one has huge overheads when one has thousands of objects. When a module supports it check it out; or if isn't possible due to the remote API, cross it over. This is inspired by a conversiation on the python in astronomy facebook group [0].

alfalfa alma atomic besancon cosmosim data eso extern fermi gama heasarc ibe irsa irsa_dust lamda lcogt magpis nasa_ads ned nist nrao nvas ogle open_exoplanet_catalogue sdss sha simbad skyview splatalogue template_module tests ukidss utils vizier xmatch [0] - https://www.facebook.com/groups/astropython/permalink/1729453593965990/#

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

[bsipocz]

@jwoillez - Brilliant, thanks.

[eteq]

the IRSA dust service (which is what the fb post was about) supports a table upload. It's a completely different web API, but I don't see any reason why we couldn't just have that be transparent to the user... Will create an issue for that.

[bsipocz]

@keflavich - I've run into this one yet again, seeing a looping through coordinates before running a query in a user notebook. It would be nice to go through all the modules and make sure they work with vectorized SkyCoord inputs, and adding such a vector query to the test suite. If the remote API doesn't support vector query, then ideally we should do the looping in the astroquery code, so the user facing API stays the same, and work with both scalar and vector inputs.

[bsipocz]

cc @nkphysics - If you're still looking for things to work on, this would be another of those issues that are very modular, and a long-term pain point.

[nkphysics]

cc @nkphysics - If you're still looking for things to work on, this would be another of those issues that are very modular, and a long-term pain point.

I can certainly take a look into this.

[keflavich]

@bsipocz I agree, that's a good idea. This might actually be a good student project if we can find someone suited to the task. I'm unfortunately not volunteering to advise anyone additional right now.

[bsipocz]

We need this fixed for heasarc, so I'll look into it myself.

[lonestarchaser]

@bsipocz Does this work with astroquery.gaia? I have a vectorized list I'm trying to send into a cone search.

coord = SkyCoord(ra=ra_list*u.degree, dec=dec_list*u.degree, frame='icrs')
j = Gaia.cone_search_async(coord, radius=u.Quantity(2.0/360, u.deg),background='true')

I'm just getting a straight-up error of "null"

[/usr/local/lib/python3.10/dist-packages/astroquery/utils/tap/conn/tapconn.py](https://localhost:8080/#) in check_launch_response_status(self, response, debug, expected_response_status, raise_exception)
    670             errMsg = taputils.get_http_response_error(response)
    671             print(response.status, errMsg)
--> 672             raise requests.exceptions.HTTPError(errMsg)
    673         else:
    674             return isError

HTTPError: Error 500:
null

(Sorry if this is not where this is supposed to go - I'm new to this!!)

[cosmoJFH]

Hi @lonestarchaser, I am afraid, the code is not prepared to handle a list of coordinates:

dec_list=[-60,-50]
ra_list=[280,270]
coord = SkyCoord(ra=ra_list*u.degree, dec=dec_list*u.degree, frame='icrs')
coord
<SkyCoord (ICRS): (ra, dec) in deg
    [(280., -60.), (270., -50.)]>

j = Gaia.cone_search_async(coord, radius=u.Quantity(2.0/360, u.deg),background='true')
j.get_error(
... )
'Error 200: \nCannot parse query \'\n                SELECT\n                  TOP 50\n                  *,\n                  DISTANCE(\n                    POINT(\'ICRS\', ra, dec),\n                    POINT(\'ICRS\', [280.00000901 270.00000937], [-59.99999412 -49.99999447])\n                  ) AS dist\n                FROM\n                  gaiadr3.gaia_source\n                WHERE\n                  1 = CONTAINS(\n                    POINT(\'ICRS\', ra, dec),\n                    CIRCLE(\'ICRS\', [280.00000901 270.00000937], [-59.99999412 -49.99999447], 0.005555555555555556)\n                  )\n                ORDER BY\n                  dist ASC\n                \' for job \'1712050716228O\': Encountered " &lt;UNSIGNED_FLOAT> ".00000901 "" at line 7, column 39.\nWas expecting one of:\n    "." ...\n    "," ...\n    "+" ...\n    "-" ...\n    "*" ...\n    "/" ...\n    "." ...\n    "*" ...\n    "/" ...\n    "+" ...\n    "-" ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    "," ...\n    \n'

Since the function POINT cannot handle an array of coordinates.

[lonestarchaser]

Got it, and thank you! (I thought I was just losing my mind!) Do you have a recommendation for how I could do a multi-object search (1-1 match) using Python? I can do it manually using the IRSA catalog search and uploading my table, but I was hoping to automate the whole thing.

[cosmoJFH]

Hi, may be the example in section "2.6. Cross match"" of the documentation https://astroquery.readthedocs.io/en/latest/gaia/gaia.html could provide the functionality you need.

[lonestarchaser]

Thanks! Looks like I need to be an authenticated user, though? I did create an account here: https://gaia.aip.de/, but that doesn't seem to be the same set of credentials.

[cosmoJFH]

I have no experience with the archive at the Leibniz-Institute for Astrophysics Potsdam (AIP) https://gaia.aip.de/. I only used the archive at ESAC https://gea.esac.esa.int/archive/