Leak from true stars into redrock galaxies

londumas commented 6 years ago

The following plot shows the distribution of redshift for SDSS (v5_10_0) galaxies (orange) and QSOs (green). All the objects have "ZWARN==0" The galaxy redshift distribution has sharp peaks, for example: z \in [1.019, 1.023]. Attached is a list of targets in this redshift range. The majority of these targets are stars, maybe M stars.

Distribution: histo_z__zoom

Stack of spectra in observed wavelength: stack_spectra_lobs

List of targets list_bad_target_in_peak.txt

sbailey commented 6 years ago

Visually scanning a few of these with the SDSS webviewer, many of them are Ldwarfs for which redrock doesn't have a template class (new issue #58 requests them). Some others are M6III. Redrock does have an M-type PCA template, but perhaps it doesn't have enough diversity and could be trained on a larger set.

A snippet of code to turn the list_bad_target_in_peak.txt into SDSS spectral viewer URLs (though note that this assumed 4-digit plate numbers, and thus fails on the last one):

import sys
for line in open(sys.argv[1]):
    plate = line[0:4]
    mjd = line[4:4+5]
    fiber = str(int(line[9:9+4]))
    url = 'https://dr14.sdss.org/optical/spectrum/view?plate={}&mjd={}&fiberid={}'.format(plate, mjd, fiber)
    print(url)

sbailey commented 6 years ago

Cross matching to ZWARNING=0 matches in DR12 (because that is the spAll that I happened to have on my laptop):

Ldwarf: 710
M6III: 212
M7IIIevar: 67
other: 11

i.e. most of these could be addressed by adding Ldwarf templates and improving the M-star templates.

moustakas commented 6 years ago

The coolest stars we currently have in the stellar template set is 2600 K (based on the BT-Settle library). @callendeprieto what do you recommend for extending into the L-dwarf regime?

callendeprieto commented 6 years ago

I haven't worked with those, but my understanding is that models are not as realistic as for warmer objects. For consistency one could expand the template set with cooler BT-Settle models from Allard.

On 12/16/2017 05:15 AM, Moustakas wrote:

The coolest stars we currently have in the stellar template set is 2600 K (based on the BT-Settle library). @callendeprieto https://github.com/callendeprieto what do you recommend for extending into the L-dwarf regime?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/desihub/redrock/issues/57#issuecomment-352162084, or mute the thread https://github.com/notifications/unsubscribe-auth/AXwt-T4fwxLaNj9zLT6eCAe5NtA0sW-Dks5tA1HygaJpZM4RD0Un.

ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en

londumas commented 6 years ago

When comparing DR12_SuperSet and redrock we can see that ~7% of 'visually inspected' stars are flagged as galaxies by redrock. The following file gives their targetid, plate, mjd, fiberid: star_in_superSet__galaxy_in_redrock.txt

sbailey commented 6 years ago

@londumas please check the DR12 CLASS/SUBCLASS of these to identify which type of stars they are? Are the "star_in_superSet_galaxy_in_redrock.txt" also L-dwarf and MIII6 and M7IIIevar SUBCLASS in DR12?

londumas commented 6 years ago

@sbailey, the DR12 CLASS/SUBCLASS info doesn't exist in "SuperSet", I would have to look into the "spAll-*.fits". The only thing I can say for now, is that it was a QSO TARGET.

ngbusca commented 6 years ago

It would be great to look at a few examples to see whether the wrong classification can be confirmed by a non expert or expert human eye, or whether redrock is actually getting it right. I've seen a few (very very few) examples where flagrant high-z quasars are classified as stars in DR12Q.

londumas commented 6 years ago

@sbailey, The CLASS of these "visually inspected" stars are {'GALAXY': 2434, 'QSO': 4199, 'STAR': 5392} in SpAll. It is quite even along the difference subclasses: maybe WD, F and A stars are a little bit more represented.

londumas commented 6 years ago

In a similar way, when comparing the truth table of ELG and LRG to redrock, we get some true stars flagged as galaxies. They all seem to be either "L" or "M" stars. stars_in_truthTable__galaxies_in_redrock.txt The truth tables are here: https://trac.sdss.org/wiki/eBOSS/Pipeline/RedshiftFitting#no1

londumas commented 6 years ago

Here is the current status of the distribution of eBOSS galaxies from redrock. The following plot shows the tail of the distribution for 70 eBOSS plates. One of the previous line has disappeared from updating the M-star templates. I am investigating the other lines. zoomed_distribution_eboss_galaxies

Here are the result comparing with SDSS SpALL:

line z in [0.9842, 0.9857] are SDSS sky flibers
line z in [1.683, 1.684] are K, F, G stars
line z in [0.3147, 0.3152] are K, F, G stars
line z in [0.4957, 0.4965] most certainly SDSS sky fibers
line z in [1.5974, 1.6004] sky fibers and sky fibers looking at a star

londumas commented 6 years ago

Here are the result of running redrock on DR7 data (/uufs/chpc.utah.edu/common/home/sdss05/dr7/das/spectro/ss_26/):

We can see that multiple remaining lines in the redshift distribution of galaxies. These are either sky-lines or stars.

capture d ecran de 2018-04-02 17-33-42

The contribution of stars to the redrock quasars is very limited.

capture d ecran de 2018-04-02 17-34-07

londumas commented 6 years ago

An idea of solving this issue is to produce new stellar PCAs from a mixture of desisim.STAR and the observed stars classified as galaxies by redrock.

sbailey commented 6 years ago

Fixed via PR #119 to compare best fits to archetypes to avoid unphysical fits.

desihub / redrock

Leak from true stars into redrock galaxies #57