Comparison between fastpp and FAST-IDL

[kevinhainline]

Hello! I'm really excited to use fastpp, but I'm really confused as to a few things with respect to the comparison between the results from fastpp and FAST-IDL. When I use the same parameter file with both FAST-IDL and fastpp, I get similar masses, but vastly different ages, tau values, and SFRs (the tau values and ages don't have any correlation), even when I set:

C_INTERVAL=68
BEST_AT_ZPHOT=1
ZPHOT_CONF=68

as suggested in the README. The chi-square values are larger for FAST-IDL by quite a bit than for fastpp, as well. For this test, I'm running with N_SIM = 0, for quick usage.

In addition, I'm confused as to what the fitting redshifts correspond to. In the .param file, I set:

NAME_ZPHOT     = 'z_peak'

and the output redshifts in the .fout file don't seem to correspond to z_peak values, even with BEST_AT_ZPHOT = 1. Here, both the z_spec value in the .zout and .cat files are set to -9999.

Perhaps I am not understanding how the code works, but I just don't know why I can't seem to get fast to try to fit at the z_peak values.

[cschreib]

Hi Kevin. I've just checked that the example catalog and param files provided with FAST++ still produce the exact same output as FAST-IDL, so there must be something odd with either your catalog or the param file. Could you upload them here (at least a few galaxies from your catalog, no need for the whole thing) so I can check?

For the redshift, I also just tested on the example catalog and param files that changing the redshift column from "z_m2" to "z_1" worked as expected, so again it may be something specific to your catalog. One thing to keep in mind: FAST (++ or IDL) creates a redshift grid with the step specified in "Z_STEP", and for each galaxy it will use the closest redshift in this grid. So, for example, if the grid has a step of 0.05 and you have z_peak=2.01 and z_m2=2.00, there will be no difference and both will use z=2.00.

[kevinhainline]

Hello! Thank you for your quick reply. I went and ran both my installation of fast and fast++ on the HDF catalog, and got very similar results, so it's likely that it's my parameter file, or my catalog. I went and made a stripped down version of my catalog, which is generated from mock JWST/NIRCam images, so I'm going to have to include my filter file. You can download everything at this dropbox link. In this file, I'm not including the libraries that I'm using, so they currently point to '/Path/to/FAST/FAST_v1.0/Libraries/', and will have to be changed.

(FIRST, as a weird aside, my flux units in my catalog file for EAZY are erg/s/cm^2/Hz, and when I set AB_ZEROPOINT = -48.57, well, fastpp works, but FAST-IDL doesn't find solutions for the fluxes. So, I am going to be providing the flux units in terms of microJy, which seems to work.)

So, first, when I run EAZY, I don't use a prior, because I'm testing our ability to detect faint high-redshift galaxies with filters on a telescope that isn't on orbit yet, so right now, there is no "z_m2" output, instead I'm using "z_peak":

# id z_spec z_a z_m1 chi_a l68 u68  l95 u95  l99 u99  nfilt q_z z_peak peak_prob z_mc
# EAZY 2015-05-08
1   -9999.0000  0.683  0.638 5.150842e-01     0.544  0.726   0.445  0.780   0.391  0.816     9    5.91759e-02  0.6381   0.999  0.6329

Now, if I run FAST-IDL on the files I sent you, here is the output:

# FAST version:  1.0
# Photometric catalog file: FAST_input_SEXtractor_Fluxes_5_8_19_close_PhotzInput.cat
# Photometric redshift file: FAST_input_SEXtractor_Fluxes_5_8_19_close_PhotzInput.zout
# AB ZP:        23.93
# Library:      FSPS (Conroy et al.)    
# SFH:          Delayed exponential SFH: SFR ~ t exp(-t/tau)  
# Stellar IMF:  Chabrier
# Dust law:     Kriek & Conroy (2013) average dust attenuation law
# metallicity:  0.0190
# log(tau/yr):  7.0    - 10.0, in steps of 0.50
# log(age/yr):  7.6    - 10.0, in steps of 0.20
# A_V:          0.0    -  3.0, in steps of 0.10
# z:            0.0100 - 15.0000, in steps of 0.0500
# Filters:      15  16  17  18  19  20  21  22  24
# ltau: log[tau/yr], lage: log[age/yr], lmass: log[mass/Msol], lsfr: log[sfr/(Msol/yr)], lssfr: log[ssfr*yr], la2t: log[age/tau]
# For sfr=0. lsfr is set to -99
#    id         z      ltau     metal      lage        Av     lmass      lsfr     lssfr      la2t      chi2
      1    0.6381      8.50    0.0190      9.40      0.00     10.90      0.05    -10.85      0.90  4.84e+02
      2    0.2496     10.00    0.0190     10.00      0.00      9.55     -0.10     -9.65      0.00  2.57e+02
      3    1.4196      8.00    0.0190      8.60      0.30      9.07      0.11     -8.96      0.60  6.68e+00
      4    0.4644      7.00    0.0190      7.60      0.70      6.73     -1.29     -8.02      0.60  8.34e+00
      5    3.8147      8.50    0.0190      7.60      0.20      6.32     -0.95     -7.27     -0.90  4.20e-01

(etc etc etc)

while if I run fastpp, I get this:

# FAST++ version: 1.2.1
# Photometric catalog file: FAST_input_SEXtractor_Fluxes_5_8_19_close_PhotzInput.cat
# Photometric redshift file: FAST_input_SEXtractor_Fluxes_5_8_19_close_PhotzInput.zout
# Template error function: /Users/knh/Desktop/NIRCam/photometric_redshifts/fastpp/share/TEMPLATE_ERROR.fast.v0.2
# AB ZP:       23.93
# Library:     FSPS (Conroy et al.)    
# SFH:         Delayed exponential SFH: SFR ~ t exp(-t/tau)   (inst. SFR)
# Stellar IMF: Chabrier
# Dust law:    Kriek & Conroy (2013) average dust attenuation law
# metallicity: 0.019
# log(tau/yr): 7       - 10      in steps of 0.5
# log(age/yr): 7.6     - 10      in steps of 0.2
# A_V:         0       - 3       in steps of 0.1
# z:           0.01    - 15      in steps of 0.05 (linear)
# Filters:     15  16  17  18  19  20  21  22  24
# ltau: log[tau/yr], lage: log[age/yr], lmass: log[mass/Msol], lsfr: log[sfr/(Msol/yr)], lssfr: log[ssfr*yr], la2t: log[age/tau]
# For value=0. log[value] is set to -99
#     id         z      ltau     metal      lage        Av     lmass      lsfr     lssfr      la2t           chi2
       1      0.66         7     0.019       8.2       1.5     10.89     -1.67    -12.56       1.2   5.964030e-02
       2      0.26       9.5     0.019       9.8         0      9.53     -0.11     -9.64       0.3   6.689383e-02
       3      1.41         7     0.019         8       0.5         9     -1.24    -10.24         1   2.549603e-01
       4      0.46         7     0.019       8.6       0.4      7.47    -15.08    -22.54       1.6   3.875577e-01
       5      3.81       8.5     0.019       7.6       0.2      6.33     -0.95     -7.27      -0.9   4.966331e-01

etc etc etc

Now, notice that the chi2 values are quite different, and while the redshifts and masses are similar, the SFRs, taus, ages, and Av values are quite different. I'm sure that it's something dumb that I'm not noticing (especially since FAST-IDL isn't super well documented), and I'd love to know what I'm doing that's stupid. Thank you again for your help!

[kevinhainline]

I should also point out that, for object 1, if I plot the output from setting BEST_FIT = 1, I get this

The red are the observed fluxes from the .cat file, the blue dots and blue line are the fluxes and model fit from fastpp, and the green dots and green line are the fluxes and fit from FAST-IDL. I don't exactly know why the fluxes from FAST-IDL are so different from its own fit spectrum, but I'm beginning to trust fastpp's output a lot more than what I'm getting from FAST-IDL.

If I instead change the flux catalog to be at AB_ZEROPOINT = 25.0, then things agree:

Maybe I am completely not understanding how FAST-IDL is dealing with fluxes.

[cschreib]

So the reason why you get different fit results and chi2 values is that FAST++ currently behaves differently than FAST-IDL when you do not provide a template error function file (i.e., when you set the path to an empty string). FAST-IDL will not apply the template error function, but FAST++ will use the default template error function file. This is a mistake, because this means it is essentially impossible not to use a template error function with FAST++... I will fix this and create a new release. Sorry about that, and thanks for reporting the issue. As an aside, I would in general strongly recommend using the template error function, especially in your case when you have very high S/N photometry.

As for the reason why the zero points are messed up with FAST-IDL, this is simply because they have hard-coded the zero point of the output model fluxes to 25 (in fast_best_fit.pro line 34). That's their mistake ;)

[cschreib]

Done!