search

nasa-fornax / fornax-demo-notebooks

Demo notebooks for the Fornax project
https://nasa-fornax.github.io/fornax-demo-notebooks/
BSD 3-Clause "New" or "Revised" License
8 stars 19 forks source link

Add herschel_get_spec functionality to spectroscopy notebook #284

Closed jkrick closed 1 month ago

jkrick commented 3 months ago

This new module downloads and extracts "reduced" Herschel spectroscopy for a set of sources. It is completely functional, but not perfect. Description of imperfections (with short rants included) below:

  1. Some decisions had to be made as this is a complicated dataset with complicated sets of rules about when a final spectrum is generated and which fluxes of the many reported are the best to use for any given target.

    • only observations which produce a HPSSPEC directory are considered. There seems to be no way to ascertain this in the archive search, so the code downloads all tar files for all observations and then checks to see if this directory exists.

    • source resolution matters, ie., recommendations for which reduced spectrum to use are based on wether or not it is a point source or how extended it is. I have followed the advice in https://www.cosmos.esa.int/documents/12133/996891/Product+decision+trees to use the flux column which has the largest sum. It is beyond the scope of this notebook to calculate if the sources is extended in Herschel band or not.

    • The Herschel directory tree in the downloaded tar files is extensive, and there is no way to customize and only download the one file that I need, so there is a lot of code around tracking filenames and paths and searching paths for filenames.

    • The Herschel archive breaks the connection at some point (testing on Arp220 which has many observations > 50, will hit this error). I cannot for the life of me figure out how to catch the specific exceptions, and have spent too long trying to do this. So I know that blanket exceptions are not good practice, but they are the only way I know how to get this to work. I have opened an issue on this both here (#282 ) and at astroquery (to try to fix this at the source).

    • I have written into the function an option to delete tar files after the spectrum is extracted.

  2. I re-numbered the sections to account for the fact that Herschel data is being accessed through the ESA archive and not IRSA as was originally suggested in the outline. The use of astroquery's ESA module made this module possible in an automated fashion from python, so it is not possible to switch to IRSA. IRSA only houses an API (which may even just access the ESA API???).

  3. It works with the plotting function before Andreas's newest PR #281 , but probably needs to be checked for compatibility with new function. I have opened an issue for this (#283 ).

afaisst commented 1 month ago

Works. The only problem is that it takes ages (meaning hours) to get the Herschel data for only 5 sources.