astropy / astroquery

Functions and classes to access online data resources. Maintainers: @keflavich and @bsipocz and @ceb8
http://astroquery.readthedocs.org/en/latest/
BSD 3-Clause "New" or "Revised" License
706 stars 399 forks source link

Obtain the spectral energy distribution from a NED query #2832

Open cosimoNigro opened 1 year ago

cosimoNigro commented 1 year ago

Hello,

is there a way, by using the NED query, to directly obtain the spectral energy distribution (SED) of a source? I am referring to the table provided in the "Photometry and SED" section of any NED web query.

Screenshot 2023-09-15 at 17 58 58

It does not look possible, as I don't see any related function in the NedClass' API. This feature would be incredibly useful as it would automatise a repetitive and manual task (downloading the tables manually). It would also help since the SED table that you manually download from NED it's a bit messy (it includes for example empty fields that are then difficult to parse from python - they result in in rows with different lengths).

Another question is: in case of this type of questions that are not real issues, is there another way to communicate / ask for support (e.g. a slack with a #help channel, GitHub discussions and so on...), without opening an issue?

Thank you!

bsipocz commented 1 year ago

Another question is: in case of this type of questions that are not real issues, is there another way to communicate / ask for support (e.g. a slack with a #help channel, GitHub discussions and so on...), without opening an issue?

I would argue that this is a valid issue. You have checked the Nedclass API, and raised request for a feature, to add access for the NED SED service.

For more generic, howto type question though we don't have an astroquery helpdesk, so you either try to get help from other users at e.g. https://community.openastronomy.org/, or directly at the archive's helpdesk. In many of the cases they are maintaining their respective modules, so would know the pythonic solutions for the given questions. (Or in the case they don't maintain these modules, they would see the user interest and may consider doing so in the future, e.g. we had quite a few cases for that scenario)

keflavich commented 1 year ago

I agree this is a valid issue - it's a question and/or feature request. I like having such questions here; if it turns out it's something that can be done, but is not documented, we turn it into a documentation issue.

That said, I don't know how to what you've requested, but I'm sure it is possible. In brief, one would need to implement a query to the tableSave endpoint (https://ned.ipac.caltech.edu/ffs/sticky/CmdSrv?cmd=tableSave) that encodes data like the following:

cmd: tableSave
request: {"startIdx":0,"pageSize":2147483647,"filters":"\"NED Units\" = 'Jy'","source":"http://rc.ned.ipac.caltech.edu/cgi-bin/datasearch?meas_type=bot&ebars_spec=ebars&label_spec=no&x_spec=freq&y_spec=Fnu_jy&xr=-1&of=xml_main&search_type=Photometry&objname=m31&hconst=67.8&omegam=0.308&omegav=0.692&wmap=4&corr_z=1&objid=58141","alt_source":"http://rc.ned.ipac.caltech.edu/cgi-bin/datasearch?meas_type=bot&ebars_spec=ebars&label_spec=no&x_spec=freq&y_spec=Fnu_jy&xr=-1&of=xml_main&search_type=Photometry&objname=m31&hconst=67.8&omegam=0.308&omegav=0.692&wmap=4&corr_z=1&objid=58141","META_INFO":{"title":"photandseds","tbl_id":"photandseds_table","col.No..fixed":"true","col.Observed Passband.fixed":"true","col.Photometry Measurement.fixed":"true","col.Uncertainty.fixed":"true","col.Units.fixed":"true","col.Refcode.PrefWidth":"20","col.Units.PrefWidth":"6","col.Upper limit of uncertainty.PrefWidth":"10","col.Lower limit of uncertainty.PrefWidth":"10","col.Upper limit of Flux Density.PrefWidth":"10","col.Lower limit of Flux Density.PrefWidth":"10","col.Significance.PrefWidth":"16","col.Frequency Mode.PrefWidth":"22","col.Coordinates Targeted.PrefWidth":"24","col.Spatial Mode.PrefWidth":"22","col.Qualifiers.PrefWidth":"22","col.Comments.PrefWidth":"26","col.Refcode.Links":"[{\"href\":\"reflookup?refcode=\"}]","resultSetID":"DATA_a2fda9d5438748ff3127079c32ca8ff0","resultSetRequest":"{\"RequestClass\":\"ServerRequest\",\"META_INFO\":{\"col.Photometry Measurement.fixed\":\"true\",\"col.Upper limit of uncertainty.PrefWidth\":\"10\",\"col.Spatial Mode.PrefWidth\":\"22\",\"col.Observed Passband.fixed\":\"true\",\"col.Units.PrefWidth\":\"6\",\"title\":\"photandseds\",\"col.No..fixed\":\"true\",\"col.Qualifiers.PrefWidth\":\"22\",\"col.Uncertainty.fixed\":\"true\",\"col.Units.fixed\":\"true\",\"col.Significance.PrefWidth\":\"16\",\"col.Coordinates Targeted.PrefWidth\":\"24\",\"tbl_id\":\"photandseds_table\",\"col.Lower limit of uncertainty.PrefWidth\":\"10\",\"col.Lower limit of Flux Density.PrefWidth\":\"10\",\"col.Refcode.Links\":\"[{\\\"href\\\":\\\"reflookup?refcode=\\\"}]\",\"col.Frequency Mode.PrefWidth\":\"22\",\"col.Comments.PrefWidth\":\"26\",\"col.Upper limit of Flux Density.PrefWidth\":\"10\",\"col.Refcode.PrefWidth\":\"20\"},\"tbl_id\":\"photandseds_table\",\"startIdx\":0,\"pageSize\":1000,\"alt_source\":\"http:\\/\\/rc.ned.ipac.caltech.edu\\/cgi-bin\\/datasearch?meas_type=bot&ebars_spec=ebars&label_spec=no&x_spec=freq&y_spec=Fnu_jy&xr=-1&of=xml_main&search_type=Photometry&objname=m31&hconst=67.8&omegam=0.308&omegav=0.692&wmap=4&corr_z=1&objid=58141\",\"id\":\"IpacTableFromSource\",\"source\":\"http:\\/\\/rc.ned.ipac.caltech.edu\\/cgi-bin\\/datasearch?meas_type=bot&ebars_spec=ebars&label_spec=no&x_spec=freq&y_spec=Fnu_jy&xr=-1&of=xml_main&search_type=Photometry&objname=m31&hconst=67.8&omegam=0.308&omegav=0.692&wmap=4&corr_z=1&objid=58141\",\"filters\":\"\\\"NED Units\\\" = 'Jy'\",\"ffSessionId\":\"FF-Session-1694808146291\"}"},"id":"IpacTableFromSource","RequestClass":"ServerRequest","ffSessionId":"FF-Session-1694808146291","inclCols":"\"No.\",\"Observed Passband\",\"Photometry Measurement\",\"Uncertainty\",\"Units\",\"Frequency\",\"Flux Density\",\"Upper limit of uncertainty\",\"Lower limit of uncertainty\",\"Upper limit of Flux Density\",\"Lower limit of Flux Density\",\"NED Uncertainty\",\"NED Units\",\"Refcode\",\"Significance\",\"Published frequency\",\"Frequency Mode\",\"Coordinates Targeted\",\"Spatial Mode\",\"Qualifiers\",\"Comments\""}
file_name: photandseds
file_format: ipac
cosimoNigro commented 1 year ago

Thank you very much for the feedbacks!

Actually, looking more carefully in the API, it is possible already by using the get_table function with the table="photometry" argument, for example

sed_table = Ned.get_table("3C 279", table="photometry")

which returns an astropy.Table like the one shown in the NED webpage Screenshot 2023-09-16 at 12 42 38

Unfortunately uncertainties (and upper limits) are inconveniently formatted as strings (with their initial characters, +/- or <, used to distinguish actual measurements and upper limits). So it takes a bit of manipulation before getting proper SED numerical values that one can work with - for example, just plotting them:

measurement = [_.startswith("+/-") for _ in sed_table["NED Uncertainty"]]
ul = [_.startswith("<") for _ in sed_table["NED Uncertainty"]]

nu_meas = sed_table["Frequency"][measurement]
f_nu_meas = sed_table["Flux Density"][measurement]
f_nu_err_meas = [float(_.strip("+/-")) for _ in sed_table["NED Uncertainty"][measurement]]

nuFnu_meas = nu_meas.quantity * f_nu_meas.quantity
nuFnu_meas_err = nu_meas.quantity * np.asarray(f_nu_err_meas) * u.Jy

nu_ul = sed_table["Frequency"][ul]
f_nu_ul = [float(_.strip("<")) for _ in sed_table["NED Uncertainty"][ul]]

nuFnu_ul = nu_ul.quantity * np.asarray(f_nu_ul) * u.Jy

plt.errorbar(
    nu_meas,
    nuFnu_meas.to_value("erg cm-2 s-1"),
    yerr=nuFnu_meas_err.to_value("erg cm-2 s-1"),
    marker=".",
    ls="",
    color="k",
    alpha=0.6,
)
plt.errorbar(
    nu_ul,
    nuFnu_ul.to_value("erg cm-2 s-1"),
    yerr=0.25 * nuFnu_ul.to_value("erg cm-2 s-1"),
    uplims=True,
    marker=".",
    ls="",
    color="k",
    alpha=0.6
)
plt.xscale("log")
plt.yscale("log")

plt.xscale("log")
plt.yscale("log")

plt.ylim([1e-14, 1e-8])
plt.xlabel(r"$\nu\,/\,{\rm Hz}$")
plt.ylabel(r"$\nu F_{\nu}\,/\,({\rm erg}\,{\rm cm}^{-2}\,{\rm s}^{-1})$")
plt.show()

image

Of course the inconvenient uncertainty format is a NED issue, not an astroquery one. I think it would be valuable to add this example - i.e. how to get a MWL SED from NED - in astroquery's documentation. It is a useful feature of the package that users might not be aware of. It eases a manual task very commonly performed (at least in astrophysical modelling).