BEAGLE does not correctly fit multiple objects, constraining redshift and mass

[kevinhainline]

So, after running BEAGLE on one object, and having the fit constrain the mass and redshift, and then running it on multiple lines of the same object, and having it run correctly, I tried running it with the same parameters on two different objects. I'm pretty concerned with the results, which indicate that the redshift and mass in the output files bounce back and forth between the values from the two different objects:

https://i.imgur.com/hZwLQRX.png

This is the fits viewer view of the output file, showing the "galaxy properties" for the first 14 lines. Notice that the redshift and mass, which should stay the same for the entire fitting, bounce back and forth between two values, which happen to be the input values from the mock catalogue file. I would rather not run BEAGLE for each individual object, but instead I'd like to provide a long data file with many objects, all with individual redshifts and masses.

Here are all of the files that you can use to run this test, including the filter files, the data file, and the parameter file.

for_BEAGLE_report.zip

I ran this again for a longer list of individual objects, and the masses and redshifts change for all of the objects, which again is not what I want. Ideally, the output would have a fits file for each object, and if I opened the output fits file for each object, the galaxy properties would show that all of the fits have the same redshift/mass. Am I wrong in interpreting how this works?

[kevinhainline]

Update: When I look at very large numbers of objects, each line in the output file corresponds to a redshift and mass from some different object in the input file, with each subsequent line in the output file having the mass and redshift from each subsequent line in the input file, until the last object is reached in the output file, after which it starts over at the beginning.

So, if there are, say 1125 lines in the output file, the redshifts and masses will correspond to lines in the input file that cycle through the full list multiple times.

[jacopo-chevallard]

Hi Kevin, this makes sense, as the from_file option was introduced to create mock catalogues, i.e. continuously cycling across the parameters. I thought it would have worked in the way we want for the fitting, but apparently some code modifications are required. I'll take a look asap, hope it just requires a quick fix (I'll let you know otherwise).

[jacopo-chevallard]

Implementing this was a bit more complicated than I thought, but now, in the new release 0.3.5 seems working as expected.

@eclake you need to recompile mcfor as well, since the release 0.1.1 is required.

@kevinhainline forget about the from_file and MOCK INPUT PARAMETERS. In order to use this new release you must pass the name of the FITS file containing the parameters to be held fixed through the PRIORS CATALOGUE keyword, while mass and redshift must be defined as

PARAMETER  = name:redshift     type:fitted     prior:distribution:dirac     prior:colName:redshift

as you can see in the modified parameter file that I'm attaching BEAGLE_2_obj_test.param.zip

As soon as a new docker relase is available @eclake will notify you. Then @kevinhainline will (hopefully!) be able to close the issue.

[jacopo-chevallard]

Also, I started to update sec 5 of the manual, but this section is still mostly incomplete.

[kevinhainline]

Ok, I'll wait until there's a version that I can pull and then test things out in the way you described, with the prior keyword.

[eclake]

Thanks Jacopo, Kevin

0.3.5 now ready to download from dockerhub (eclake/beagle:0.3.5)

[kevinhainline]

Upon pulling eclake/beagle:0.3.5, and running it on a fit with different objects, I get this error:

% docker run --rm -it -v /Users/knh/Desktop/NIRCam/BEAGLE/BEAGLE-general-master:/BEAGLE_virtual_workdir --env-file env.list eclake/beagle:0.3.5 1 /BEAGLE_virtual_workdir/params/BEAGLE_mock_0.3.5_1.param

 Reading Filter File: /BEAGLE_virtual_workdir/filters/FILTERBIN.RES
 298 filters defined, out of 300                                maximum     ...done
 ---> fixed                         sfh_type                                   0
 ---> fitted                        mass                                       0
 ---> fitted                        redshift                                   0
 ---> dependent                     nebular_logU                               0
 ---> fixed                         nebular_xi                                 0
 ---> dependent                     nebular_Z                                  0
 ---> fixed                         attenuation_type                           0
 ---> fitted                        tauV_eff                                   0
 ---> fixed                         mu                                         0
 ---> fitted                        tau                                        1
 ---> fitted                        metallicity                                1
 n_fitted:            5
 *****************************************************
 MultiNest v3.9
 Copyright Farhan Feroz & Mike Hobson
 Release Oct 2014

 no. of live points =  300
 dimensionality =    5
 *****************************************************
 Starting MultiNest
 generating live points
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
         [ERROR: Redshift must be positive! ]
 [FUNCTION: "compute_IGM_transmission_dp" ]
 [MODULE: "IGM_absorption" ]
 [BACKTRACE:   
#0  0x7F13E9CD6777
#1  0x5362D4 in print_backtrace at lib_messages.f90:299
#2  0x42FA66 in __igm_absorption_MOD_compute_igm_transmission_dp at IGM_absorption.f90:117
#3  0x468C22 in __galaxy_sed_MOD_compute_sed at galaxy_SED.f90:377 (discriminator 6)
#4  0x47A3BE in __likelihood_function_MOD_likelihood at likelihood_function.f90:150
#5  0x47DCD3 in __likelihood_function_MOD_mc_likelihood at likelihood_function.f90:89 (discriminator 3)
#6  0x57D55B in getloglike at nested_sampling.f90:202 (discriminator 2)
#7  0x5A27A6 in __nested_MOD_gen_initial_live at nested.F90:536
#8  0x5A3590 in __nested_MOD_nestsample at nested.F90:352
#9  0x5A482C in __nested_MOD_nestrun at nested.F90:239
#10  0x57D92D in __nested_sampling_MOD_run_nested_sampling at nested_sampling.f90:134
#11  0x572D0A in __prosit_MOD_sample_pdf at PROSIT.f90:235 (discriminator 1)
#12  0x415659 in beagle at BEAGLE.f90:436 (discriminator 2)
 ]
 [ ***************************************** ]
 [ ----> EXITING FUNCTION / SUBROUTINE <---- ] 
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Here's the data file that I wanted to run this on, which has, as far as I can tell, positive redshifts:

BEAGLE_mock_0.3.5_1.fits.zip

And here's my parameter file, which is very similar to what Jacopo suggested.

# This is a stripped down version of the BEAGLE parameter file for 
# with checking the correct installation of BEAGLE using Docker.
# This file will be used for producing artificial photometry using
# an analytic star formation history.
# Verbose mode: 0 ---> no info
#               1 ----> print only errors
#               2 ---> print errors and warnings
#               3 ----> print errors, warnings and info 
VERBOSE = 1

# Seed for the random number generator
SEED = 994189

#****************************************************************
#******************** COSMOLOGICAL PARAMETERS *******************
#****************************************************************
#
# If left blank, default parameter values will be used.

HUBBLE_0 = 

OMEGA_MATTER =

OMEGA_LAMBDA =

#****************************************************************
# ******************* TEMPLATES  *****************************
#****************************************************************

#TEMPLATES = $BEAGLE_TEMPLATES/bc03/bc03_miles_chab_spectra
TEMPLATES NEBULAR = $BEAGLE_TEMPLATES/ineb_Jan16_C100/cb2013_n2_mup100_N015_O01_deplO70_C100_Jan16
EMISSION LINES CONFIGURATION = $BEAGLE_TEMPLATES/ineb_Jan16_C100/cb2013_n2_mup100_N015_O01_deplO70_C100_Jan16_line_wavelengths_PHOTOMETRY.dat

SHRINK TEMPLATES WL RANGE = 900 60000
REBIN TEMPLATES = 10

#SPECTRAL INDICES CONFIGURATION =

#SPECTRAL INDICES CATALOGUE = 

#****************************************************************
# ******************* PHOTOMETRY *****************************
#****************************************************************

FILTERS FILE = $BEAGLE_FILTERS/filters_NIRCat.dat

PHOTOMETRIC CATALOGUE = $BEAGLE_DATA/BEAGLE_mock_0.3.5_1.fits

FIT PHOTOMETRY = T
FIT SPECTRUM = F

#****************************************************************
#******************** DUST ATTENUATION AND IGM ABSORPTION *******
#****************************************************************
#
IGM ABSORPTION = Inoue

# File containing set of radiative transfer (RT) calculations of dust attenuation of starlight
#RT CALCULATIONS = Tuffs_bulge_NEW-28-11.dat Tuffs_thick_NEW-28-11.dat Tuffs_thin_NEW-28-11.dat
#RT COMPONENTS NAMES = bulge thick_disc thin_disc

# **************************************************
# ************** SF BIN #1 **************************
# **************************************************
SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:rising
#SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:delayed
#options for analytic SFHs are 'ssp' (for a burst), 'constant', 'exponential', 'delayed', 'rising'

# In log(yr)
SF_PARAMETER  = name:tau                      type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[7.,10.5]

# in log(Z/Z_sun)
SF_PARAMETER  = name:metallicity              type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[-2.2,0.24]

# In log M/M_sun
#SF_PARAMETER  = name:mass                     type:fitted  order_priority:0  prior:distribution:uniform  prior:range:[5.,12.] 
SF_PARAMETER  = name:mass                     type:fitted     prior:distribution:dirac    prior:colName:mass

# **************************************************
# **************  OTHER SF PARAMETERS **************************
# **************************************************

# In log(M_sun/yr)
#SF_PARAMETER  = name:sfr                type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[-4.,4.]        mock:type:random 

#****************************************************************
# ******************* PRINTED OUTPUT  *****************************
#****************************************************************

RESULTS DIRECTORY = $BEAGLE_RESULTS/BEAGLE_mock_0.3.5_1/

PRIORS CATALOGUE = $BEAGLE_DATA/BEAGLE_mock_0.3.5_1.fits
#MOCK INPUT PARAMETERS = fileName:/BEAGLE_virtual_workdir/data/BEAGLE_mock.fits

#MOCK CATALOGUE NAME = 

PRINT PHOTOMETRY = T
PRINT SPECTRUM = T
PRINT SPECTRAL INDICES = F
PRINT SF AND ChE HISTORIES = F

#****************************************************************
# ******************* PARAMETERS HANDLING  *****************************
#****************************************************************

#PDF SAMPLER FILE = $BEAGLE_PARAM_DIR/MCMC_example.param
PDF SAMPLER FILE = $BEAGLE_PARAM_DIR/MCMC_new.param

PARAMETER  = name:redshift     type:fitted    prior:distribution:dirac    prior:colName:redshift

PARAMETER  = name:nebular_logU type:dependent

PARAMETER  = name:nebular_xi   type:fixed    value:0.3

PARAMETER  = name:nebular_Z    type:dependent 

PARAMETER  = name:attenuation_type  type:fixed  char_value:CF00   
# values can be: CF00, Calzetti, CCWW13_universal, CCWW13_full

PARAMETER  = name:tauV_eff     type:fitted  order_priority:0      prior:distribution:uniform   prior:range:[0.,2.]

PARAMETER  = name:mu           type:fixed   value:0.4

[jacopo-chevallard]

I cannot reproduce your error, BEAGLE runs smoothly on your input file... Perhaps it was just a problem with the creation of the Docker image, we have so many versions of BEAGLE by now! @eclake, can you check? (I'm using the current develop branch to run, but it shouldn't differ from release 0.3.5...)

[eclake]

@jacopo-chevallard - you were absolutely right - am experimenting with scripts to automate the process more and had only synced origin/master so was still building 0.3.3, sorry for that!

I'm pushing 0.3.5 again (and have verified that it is running with the scripts above). @kevinhainline - I'm not 100% sure whether running docker pull will overwrite your local tagged version cleanly or not, so just to be on the safe side, it would be best to remove that image before pulling from docker by first finding the image id:

> docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
eclake/beagle       0.3.5               677b18ab1981        14 minutes ago      1.43 GB

and then removing it before calling docker pull:

> docker rmi -f 677b18ab1981
> docker pull eclake/beagle:0.3.5

[kevinhainline]

Emma and Jacopo, Good news! It looks like it's running fine with that latest version of 0.3.5, which I removed and then reinstalled, as described above. Thank you so much for your hard work getting this ready. I tested this on a set of three objects, and then a set of objects moving out to z ~ 15 from the mock catalogue, and found that the individual multinest samples have the same redshift and mass, which is dependent on the value from the input data file.

I have one more question (although maybe I should submit another issue): in my run of eleven galaxies going out in redshift, one of the objects at z ~ 8.37 (M_* = 891719.8 M_sun) had a fit corresponding to the maximum ln(likelihood) of:

The other objects all had fits that did a pretty great job of matching the observed photometry, but this best-fit underpredicts the flux by quite a bit. Is this just because the mass of the object is too small for a fit with a rising star formation history at that redshift? If that's the case, then it's not an issue with BEAGLE, it's with our catalogue.

Thank you both, again!

[jacopo-chevallard]

Closing, continues on #13