Add `get_query_payload` consistently to all methods

bsipocz commented 3 years ago

This is super valuable for debugging issues, e.g. I'm having troubles to catch the typo in a Simbad.query_criteria() call, and it doesn't have this kwarg.

Addressing the whole issue can be long, but the task is a package novice one, basically breakpoints needed to be added to return the payload strings rather than sending them to the server (and plenty of examples are available).

I'm adding the list of all modules here for easier tracking. We should either tick the boxes when the kwarg added, or cross out the module name if this is not applicable:

[ ] alfalfa
[ ] alma
[ ] astrometry_net
[ ] atomic
[ ] besancon
[ ] cadc
[ ] casda
[ ] cds
[ ] cosmosim
[ ] dace
[ ] esa
[ ] esasky
[ ] eso
[ ] exoplanet_orbit_database
[ ] fermi
[ ] gaia
[ ] gama
[ ] gemini
[ ] heasarc
[ ] hips2fits
[ ] hitran
[ ] ibe
[ ] image_cutouts
[ ] imcce
[ ] irsa
[ ] irsa_dust
[ ] jplhorizons
[ ] jplsbdb
[ ] jplspec
[ ] lamda
[ ] lcogt
[ ] magpis
[ ] mast
[ ] mpc
[ ] nasa_ads
[ ] nasa_exoplanet_archive
[ ] ned
[ ] nist
[ ] noirlab
[ ] nrao
[ ] nvas
[ ] oac
[ ] ogle
[ ] open_exoplanet_catalogue
[ ] sdss
[ ] sha
[ ] simbad
[ ] skyview
[ ] solarsystem
[ ] splatalogue
[ ] svo_fps
[ ] template_module
[ ] ukidss
[ ] utils
[ ] vamdc
[ ] vizier
[ ] vo_conesearch
[ ] vsa
[ ] wfau
[ ] xmatch

jwoillez commented 3 years ago

It is not exactly what you are looking for, but if you want to check the payloads as they are sent to the servers, you can use:

from astroquery import log
log.setLevel('DEBUG')

bsipocz commented 3 years ago

Yes, you're right, that works, too. But I would think it's still valuable to unify the API, as this is a bit more noisy than what I would need (note: the typo is the extra comma between ra and dec in the region definition. I would expect with the one-liner payload I have some chance to spot it (well, it's already been returned from the server as part of the error in this case)):

In [6]: Simbad.query_criteria("region(circle, ICRS, +00 01 57.9, -15 27 50, 1)", otype="Star", cache=False)
DEBUG: HTTP request
    -----------------------------------------
    POST http://simbad.u-strasbg.fr/simbad/sim-script
    User-Agent: astroquery/0.4.2.dev6653 python-requests/2.25.1
    Accept-Encoding: gzip, deflate
    Accept: */*
    Connection: keep-alive
    Cookie: JSESSIONID=70895933FD5399F3B323471C927B3ABE.new
    Content-Length: 173
    Content-Type: application/x-www-form-urlencoded

    script=%0Avotable+%7Bmain_id%2Ccoordinates%7D%0Avotable+open%0Aquery+sample++region%28circle%2C+ICRS%2C+%2B00+01+57.9%2C+-15+27+50%2C+1%29+%26++otype%3DStar+%0Avotable+close
    ----------------------------------------- [astroquery.query]
/Users/bsipocz/munka/devel/astroquery/astroquery/simbad/core.py:135: UserWarning: Warning: The script line number 3 raised an error (recorded in the `errors` attribute of the result table): 'region(circle, ICRS, +00 01 57.9, -15 27 50, 1) &  otype=Star': java.text.ParseException: 
  warnings.warn("Warning: The script line number %i raised "

bsipocz commented 3 years ago

Having access to the query payload would help in fixing e.g. #2099, too.

astropy / astroquery

Add `get_query_payload` consistently to all methods #2040