Deep catalogue for the southern hemisphere - DELVE

[smarttgit]

DELVE = DECam Local Volume Exploration Survey

Has produced a high-Galactic-latitude wide-area survey of ~17,000 deg2 to a depth of griz ~ 23.5. Data release available

https://delve-survey.github.io

Has star - galaxy separation : https://datalab.noirlab.edu/query.php?name=delve_dr2.objects

Essential catalogue for us for the southern hemisphere

[genghisken]

Requested

[thespacedoctor]

Just need our beefy machine now to host new catalogues.

[thespacedoctor]

I am going to read the DR2 paper and determine what columns we need from DELVE.

https://ui.adsabs.harvard.edu/abs/2022ApJS..261...38D/abstract

[genghisken]

So... Looking again at what we downloaded... What's the difference between the Legacy Survey and DELVE? Is it a subset of the Legacy survey or a completely different set?

What I downloaded originally was the Legacy survey DR 10.0. See FITS files in db6:/nvmeraid/raid0/db6data/catalogues/legacysurvey. I think, from earlier discussions along with Matt, we made the assumption that DELVE is a subset. I'm now not sure this is correct having scanned the paper above. https://www.legacysurvey.org

[smarttgit]

Yes, DELVE appears to be a subset of DR10. See the text in "Contents of DR10" above. Why do you think this is not the case ? Fairly sure we did come to that conclusion previously.

[genghisken]

No explicit mention of DELVE in the legacy survey catalogues. Maybe it comes under the umbrella of "Extra DECam data". And reference to the Legacy Survey in the DELVE paper.

[smarttgit]

Agree it really isn't clear what is in DR10 and what it is composed off. This para seems to suggest all the southern cats (DELVE) are in DR10 https://www.legacysurvey.org/dr10/description/. I'm fairly sure DELVE is in your DR10.

DR10 includes images from DECaLS 𝑔,𝑟,𝑧-band observations (survey program 2014B-0404) observed between 9th August 2014 and 7th March 2019. DR10 also includes 𝑔,𝑟,𝑖,𝑧-band DECam observations from a range of non-DECaLS surveys, including observations that were conducted between 8th January 2013 and 30th August 2021. The bulk of these additional DECam observations are from the Dark Energy Survey (including the full six years of the survey); the DELVE Survey, and the DeROSITA Survey. Information on the exact observations included in DR10 can be derived from the Legacy Surveys survey-ccds-* files. To obtain northern observations from BASS and MzLS, see DR9 of the Legacy Surveys.

And that the schema is the same

[genghisken]

Excellent. Sorry - I don't know how I missed the DELVE statement buried in that paragraph. Great - means we're good to go.

We have an issue with the databases on both db5 and db6. Either my original Ansible build ran incorrectly or (more likely I think) an Ubuntu update process upgraded MariaDB by uninstalling & reinstalling. This completely overwrote my specific database config (and trashed the databases to boot), so I'll need to negotiate with Robert to permanently stop those updates happening (if indeed that's the problem - still investigating). I'll rebuild the databases over the weekend.

Assuming I understand the workflow of what happened to the databases, next week Dave & I will migrate the Sherlock database from db4 to db6.

[thespacedoctor]

The story is a little more complicated, but the bottom line is that DECaLS DR10 is the superior data to DELVE DR2. I'm writing up notes and will post an executive summary soon.

[thespacedoctor]

EXECUTIVE SUMMARY:

DELVE was an imaging survey to fill in 'missing' patches of the sky in publicly available DeCAM data. The motivation was to map the local universe to catalogue the dwarf-galaxy population. The data-release contains ALL publicly available DeCAM data (including DELVE images), processed with the same pipeline. Not much care was needed/taken in star-galaxy separation.

The DESI Legacy Surveys has the remit of identifying galaxies and QSO targets for the DESI spectroscopic survey. Much more care has been taken to fit/model each individual source in the imaging data with their Tractor package. Star-galaxy separation should be very good. As of DR10, the Legacy Surveys folded in most/all publicly available DeCAM data from other surveys, including DELVE and DeROSITA.

TLDR summaries to follow.

[thespacedoctor]

DELVE

DELVE : DECam Local Volume Exploration survey.

DECam has been used by a few extensive surveys, like the Dark Energy Survey (DES) and the DECam Legacy Survey (DECaLS), and hundreds of smaller programs. Combined, these surveys provide a patchy coverage of the southern celestial hemisphere.

The motivation behind the DELVE survey is to achieve complete DeCAM imaging coverage of the high-galactic-latitude southern sky to study dwarf satellite galaxies surrounding the Milky Way and within the Local-Volume. The DELVE and DeROSITAS surveys have recently targeted regions of the sky that have lacked DECam imaging with the aim of filling gaps and increasing coverage. The DELVE survey included 126 nights of new DeCAM data.

The DELVE DR2 catalogue results from processing this new data with archival DECam data with the same reduction pipeline (DESDM “Final Cut” pipeline). The catalogue provides PSF and aperture photometry for ∼2.5 billion sources within a sky area covering >21,000 deg^2 (g, r, i, and z-bands). In the sky region of ∼17,000 deg^2 imaged in all four filters, there are ∼618 million sources.

Important to note:

• DELVE DR2 does not include the coadded images (expected in a future release)

• However, the weighted average of the individual single-epoch photometry measurements is given for each source (pseudo-stacked).

• detections are clustered into ‘objects’ with a 0.5 arcsec radius.

• Star-galaxy separation done with rudimentary SourceExtractor spread_-model parameter. extended_class value of 0 indicates high-confidence stars, 1 is likely stars, 2 is likely galaxies, and 3 is high-confidence galaxies. extended_class is calculated for each band but g-band is favoured as it has the widest coverage and deepest limiting magnitude. Not sure this is going to be good enough for Sherlock at fainter magnitudes.

  

• An additional product with the DR2 catalogue gives photometric redshifts derived using a Mixture Density Network. Only documentation lacks much detail on the accuracy of the individual photo-zs

DR2_MAIN Table Columns:

[thespacedoctor]

DESI Legacy Survey DR10

The primary motivation behind the DESI Legacy Imaging Surveys is to provide galaxy and QSO targets for the Dark Energy Spectroscopic Instrument (DESI) survey (5000-fiber multi-object spectrograph on the Mayall 4 m telescope at the Kitt Peak National Observatory). As such, there is a need for clean star-galaxy separation, which has driven the development of their Tractor Package (I assume the name comes from "extractor").

The original footprint of the Legacy Surveys was designed to complement the DESI Survey footprint (extragalactic sky above a Galactic latitude of b = 15° that can be observed spectroscopically from Kitt Peak), but with DR10 the Legacy Surveys have now folded in publicly available DECam data from DELVE and DeROSITA (and many other publicly available data) to expand that footprint in the Southern Hemisphere greatly. The footprint is now > 20,000 square degrees.

The DESI Legacy Imaging Surveys consist of three public projects:

1. Dark Energy Camera Legacy Survey (Blanco telescope at the Cerro Tololo Inter-American Observatory)
2. the Beijing–Arizona Sky Survey (University of Arizona Steward Observatory 90 inch Bart Bok Telescope at Kitt Peak National Observatory), and
3. the Mayall z-band Legacy Survey (Mayall Telescope at the Kitt Peak National Observatory)

The main product of interest for Sherlock is the Tractor Catalogue, which was built using a "probabilistic inference-based approach to estimate source shapes and brightnesses".

The Tractor Catalogue includes:

• source positions,
• fluxes (grz optical bands & some i-band in the South. Four WISE mid-infrared bands, W1-4).
• shape parameters, and
• morphological quantities

There are 5 types of models used in the Tractor package. Each individual catalogue source is labelled with the model found to best fit its profile (χ² minimisation problem)

• PSF: dominated by stars but also contains compact galaxies and QSOs.
• EXP: exponential disks (spiral galaxies)
• REX: round exponential galaxies with a variable radius
• DEV: de Vaucouleurs profiles (elliptical galaxies)
• SER: Sersic Massive elliptical galaxies)

A sixth model, DUP, is given to Gaia sources coincident with sources Tractor has chosen to fit with an extended model. I assume they want these sources in the catalogue for calculating astrometric solutions based on Gaia positions.

Note:

• WISE photometry is 'forced' on new image stacks of WISE data at the location of sources detected in the optical data.
• The 3 Legacy Surveys projects are now complete.
• Tractor catalogues are presented in "bricks" spanning approximately 0.25 deg × 0.25 deg, with an average area of 0.0623 deg^2. Each brick is a box in RA, Dec coordinates. The brick images have some overlap.
• There are \~2.8 billion unique sources in DR10 spread over 366,898 bricks.
• Bright foreground sources are treated as special cases and extracted independently, as it was found bright stars trigger diffuse galaxy model fitting due to large halos (Sources that lie inside the boundary of a foreground object have MASKBITS set)
• Primary objects in the catalogue have BRICK_PRIMARY==True. We only need primary sources for Sherlock.
• The image stacks are provided, but were not used in the Tractor fits.
• All the info we need seems to be contained within the ls_dr10.tractor table. I still need to determine which columns are needed, but they are likely going to be the columns in presented in bold on the linked schema page.