Notebook: Define Spectroscopy Use Case Notebook

[jkrick]

Afaisst: Currently working with Thomas Lai on JWST spectrum use case. Once that is done, how to generalize?

[jkrick]

It would be nice to think about how this could eventually be turned into a Fornax demo notebook. Fornax demo notebooks need to:

• [ ] touch all three archives, IRSA, MAST, HEASARC
• [ ] Involve big data
• [ ] runs to completion on the SMCE

It is ok if the notebook reproduces work already published, ie., a good notebook would be one that tries to reproduce some seminal work, or expand upon some previous work. But, of course, original ideas are also good.

No one will judge the science case, so the ties to all three archives can be tenuous. Real science doesn't have a three archive requirement.

Lastly, I think it is important to be something you are actually interested in so that the work is more fun and might lead to a publication, which is a win win for all.

[jkrick]

Vandana suggested it would look good if there was Euclid & DESI and/or SPHEREX & SDSS data in the spectroscopy use case.

[afaisst]

A list of possible spectra:

Asterisk indicates NASA.

IRSA:

• Keck spectra on COSMOS. All reduced and ready to use, about 10,000 spectra
• *Spitzer IRS spectra. We have the experts in house.
• IRTF has a large library of stellar spectra
• *SPHEREx: Spectra/cubes hosted at IRSA, but that takes a couple of years. First release in >FY25
• *Euclid: Spectra hosted at IRSA in FY25?
• *Herschel: has some spectra. Not sure about reduction state

MAST:

• *HST spectra (slit and slitless): Need our own reduction for good quality. Slitless grism spectra need to be extracted by us.
• *JWST spectra (slit and cubes): likely reduction. For JWST MIRI, we can build on Thomas' work.

SDSS:

• A lot of optical spectra. Reduced and ready to use.

HEASARC They don't really have "spectra". But could as ancillary data, as we did with IceCube in the lightcurve use case.

• *Chandra

NOIRLAB Spectral from the SPARCL archive

• DESI spectra
• Sloan

CADC CADC Canadian data center, has a nice search interface for many facilities (Gemini, CFHT, HST, JWST, etc)

Long term, among some COSMOS folks we have an NSF and ADAP program to re-reduce all spectra that exist on COSMOS. We so far focussed on Keck and Subaru. This still takes a while and in the end we could store them at IRSA. This would give us the possibility to use optimally reduced and calibrated spectra on COSMOS.

What the notebook should do:

We could follow the same theme as for the lightcurve notebook: Create a tool to query spectra in the different archives for a give position on the sky. The difference is that spectra are sparse, so there is not much overlap. The biggest overlap is likely in the famous fields. For example, COSMOS is covered by Keck, Spitzer, IRTF, SPHEREx, Euclid, HST, JWST, and SDSS. Alternatively, we could think of SDSS+Spitzer as pair (see below). Another difficulty is that some of the spectra are not (or barely/badly) reduced. The reduction of the retrieved raw spectra would need to be included in such a notebook. The notebook would conclude with providing UV to IR spectra for a given source.

We can use this as preparation for Euclid and SPHEREx. Use for now the Sloan, DESI, etc. We need to know these data and know how to deal with this before the large NASA missions (Euclid / SPHEREx) start.

Possible science themes

Think of science that can benefit from combining spectra at different wavelengths.

• finding and studying extreme galaxies in the low-redshift. For example metal-poor galaxies that could serve as analogs of high-redshift galaxies (green peas, etc). The search for these could involve Machine Learning as well. We could combine UV, optical, and IR spectra of the brightest such galaxies to study the connection between different properties (stars, dust, metals, gas). A science goal would be to study the dust/PAH emission of metal-poor galaxies (what do their dust-grains look like?) That could inform observations of high-z galaxies with JWST. There do exist many studies (for example the evolution of the ionized continuum escape fraction as a function of redshift using local high-z analogs: Faisst (2016) )
• Connecting to the lightcurve notebook: We can gather the spectra of the variable AGN and characterize their black hole masses from broad emission lines. Study the feeding of massive and less massive black holes, or connect variability to metallicity of gas content of the host galaxies. There must be studies about that in the literature, which would allow us to compare our results. We don't have to do this necessarily to have a broader impact on different science topics.
• Reproduce JWST work. For example exoplanet atmosphere measured from multi We would have to re-reduce

To Do & Questions

• [ ] We should prioritize the missions and how much effort we want to put in to reduce the spectra.
• [ ] We want to focus on NASA, how can we do this best?

[afaisst]

Discussion from Nov 1, 2023

We discussed the above topcis. A summary is below

• We don't want to re-reduce the spectra on MAST. That is too time consuming. However, we can use them as "quick-look". Check what sources have JWST spectroscopy.
• It is fine to design the notebook around available large spectroscopy surveys (such as SDSS, DESI, Spitzer, etc). We first have to understand how to ingest these data before we can combine them with Euclid and SPHEREx and other NASA missions.
• Better to focus on different science case other than AGN to make use cases attractive for different science fields.
• Another component is to put spectra in Parquet file format. A good starting point is to convert the Spitzer/IRS spectra to Parquet and store them on the cloud.

[afaisst]

Created a new directory on GitHub and added a notebook template that we can use: https://github.com/fornax-navo/fornax-demo-notebooks/tree/spectroscopy_faisst_Dec723

[afaisst]

Updates:

• From Brigitta: "Re the IRSA spectro datasets from Andreas’ table, Angela has confirmed that the ones available from Gator are available through TAP, too, so we can use astroquery or pyvo"
• Completed prototype of HST spectrum query

[afaisst]

Updates:

• Herschel spectroscopy is now added to the notebook (by Jessica Krick). However, it takes very long, mainly because need to download ~800MB of data for a single spectrum. For now, we called this good enough. Needs to be resolved (with ESA?) for larger numbers of sources.
• v1 of the spectroscopy notebook is complete (July 24, 2024)