search

jacopo-chevallard / BEAGLE-general

Basic information to get started with the galaxy spectral modelling tool Beagle
https://www.iap.fr/beagle/
9 stars 0 forks source link

Multinest: ERROR: "locate" failed, the condition x(i) <= x < x(i+1) is not satisfied! #11

Closed kevinhainline closed 8 years ago

kevinhainline commented 8 years ago

After explicitly specifying the location of the MOCK INPUT PARAMETERS file in the parameters file, and running (emma's test version of) BEAGLE, I get this error:

[KNHDesktop:~/Desktop/NIRCam/BEAGLE] knh% docker run --rm -it -v /Users/knh/Desktop/NIRCam/BEAGLE/BEAGLE-general-master:/BEAGLE_virtual_workdir --env-file env.list eclake/beagle:test 1 /BEAGLE_virtual_workdir/params/BEAGLE_mock_test_1.param

 Reading Filter File: /BEAGLE_virtual_workdir/filters/FILTERBIN.RES
 298 filters defined, out of 300                                maximum     ...done
 ---> fixed                         sfh_type                                   0
 ---> from_file                     mass                                       0
 ---> from_file                     redshift                                   0
 ---> dependent                     nebular_logU                               0
 ---> fixed                         nebular_xi                                 0
 ---> dependent                     nebular_Z                                  0
 ---> fixed                         attenuation_type                           0
 ---> fitted                        tauV_eff                                   0
 ---> fixed                         mu                                         0
 ---> fitted                        tau                                        1
 ---> fitted                        metallicity                                1
 n_fitted:            3
 *****************************************************
 MultiNest v3.9
 Copyright Farhan Feroz & Mike Hobson
 Release Oct 2014

 no. of live points =  300
 dimensionality =    3
 *****************************************************
 Starting MultiNest
 generating live points
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
         [ERROR: "locate" failed, the condition x(i) <= x < x(i+1) is not satisfied! ]
 [FUNCTION: "locate_sp" ]
 [MODULE: "lib_interpolation" ]
 [BACKTRACE:   
#0  0x7FB83768F777
#1  0x535884 in print_backtrace at lib_messages.f90:299
#2  0x51FA3C in __lib_interpolation_MOD_locate_sp at lib_interpolation.f90:1199 (discriminator 1)
#3  0x52073B in __lib_interpolation_MOD_get_linear_weights_sp at lib_interpolation.f90:2868
#4  0x525DE2 in __lib_interpolation_MOD_linear_interpolate_grid_2d_sp at lib_interpolation.f90:2185 (discriminator 1)
#5  0x43A49F in __sps_routines_MOD_get_sps_spectra_age_z at SPS_routines.f90:157 (discriminator 2)
#6  0x46A375 in __galaxy_sed_MOD_compute_sed at galaxy_SED.f90:215
#7  0x4795EE in __likelihood_function_MOD_likelihood at likelihood_function.f90:150
#8  0x47CF03 in __likelihood_function_MOD_mc_likelihood at likelihood_function.f90:89 (discriminator 3)
#9  0x57CB19 in getloglike at nested_sampling.f90:193 (discriminator 2)
#10  0x5A1BC6 in __nested_MOD_gen_initial_live at nested.F90:536
#11  0x5A29B0 in __nested_MOD_nestsample at nested.F90:352
#12  0x5A3C4C in __nested_MOD_nestrun at nested.F90:239
#13  0x57CD4D in __nested_sampling_MOD_run_nested_sampling at nested_sampling.f90:133
#14  0x57229A in __prosit_MOD_sample_pdf at PROSIT.f90:235 (discriminator 1)
#15  0x415659 in beagle at BEAGLE.f90:436 (discriminator 2)
 ]
 [ ***************************************** ]
 [ ----> EXITING FUNCTION / SUBROUTINE <---- ]                                          
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 [ +++++ ERROR DETECTED: Do you want to continue with the execution of the program? (y/n)]

So, this looks like a Multinest issue. I've included the input photometry file here:

BEAGLE_mock.fits.zip

And the new MCMC file, as well as the parameter file, are located below:

MCMC_new.param:

# Seed of the random number generator
# if not present, empty, or 0 it uses the system seed
# if > 0 use the user defined seed
# if < 0 use seed form system clock 
# NB the user defined seed is used *in combination* with the default (system) seed, so in order to get the same sequences
# in different computers you need to get the actual seed of the generator *after* initialization of the MCMC
SEED = 87884

# < 1 ---> fraction of samples to be discarded as "burn-in"
BURN IN = 0.2

# Maximum number of iterations of the chain. The pdf sampling will stop if it has reached converegnce *or*
# if it has reached the maximum number of iterations
MAX ITER = 1000000

# Number of walkers to be used in the affine-invariant algorithm
NUMBER OF WALKERS = 300

# This parameter determines the number of independent samples to be drawn in order to achieve a given Monte Carlo accuracy.
# The Monte Carlo accuracy determines the error-bars on the error-bars of your model parameters.
# Do not put too small values, 5 % is ok most of the times !
# MC accuracy  |  number of effective samples 
#     10 %     |     100                
#     5 %      |     400                
#     2 %      |     2500               
#     1 %      |     10000              
MONTE CARLO ACCURACY = 0.05

# Whether to "thin" the chain or not, used only in post-processing to save memory
# The thin factor is automatically computed form the integrated auto-correlation function
THIN CHAIN = F

# Whether the sampler need to initialize MPI, or this has been done in the main program
# INITIALIZE MPI = 

# Type of sampling method: 
# ----> metropolis-hastings
# ----> affine-invariant
MCMC ALGORITHM = nested-sampling
#MCMC ALGORITHM = metropolis-hastings
#MCMC ALGORITHM = affine-invariant

# Type of adaptive proposal to be used with the metropolis-hastings sampling method
# ----> AM ("adaptive Metropolis"), in which the covariance matrix of the multivariate 
#       Gaussian proposal distribution is adaptively updated to reach an optimal acceptance rate
# ----> SCAM ("single-component adaptive Metropolis"), in which the variance of the univariate
#       Gaussian proposal distribution is adaptively updated to reach an optimal acceptance rate
# ----> RAM ("robust adaptive Metropolis"), in which the proposal transition (positive, semi-definite) 
#       matrix is updated to reach an optimal acceptance rate
PROPOSAL = AM

# Type of distribution used to draw the starting points of the chain
# ----> uniform
# ----> Gaussian
STARTING DISTRIBUTION = uniform

# If you already have an estimate of the posterior covariance put here the filename 
# containing the covariance matrix. It has to be a square matrix, n_par x n_par
#PROPOSAL MATRIX = 

# This parameter controls the frequency of the proposal adaptation, i.e. the proposal (co)variance/transition 
# matrix is updated every "UPDATE ITERATIONS" iterations
# default value = 100
UPDATE ITERATIONS = 100

# This parameter controls the frequency of the calculation of the Monte Carlo accuracy. It should be few tens of
# times the auto-correlation length, since the Monte Carlo error will decrase as 1 / sqrt(n_eff), where n_eff = n/tau_integrated
# Oherwise precious time is wasted in the computation of the auto-correlation functions.
# default value = 5000
CONVERGENCE ITERATIONS = 100

#****************************************************************
# ******************* MultiNest parameters  *********************
#****************************************************************

# Do Nested Importance Sampling? (default = F)
IS = F

# Do multi-modal Sampling? (default = T)
MULTI-MODAL = T

# Sample with constant efficiency? (default = F)
CONSTANT EFFICIENCY = F

# Evidence tolerance factor (default = 0.5)
EVIDENCE TOLERANCE = 0.50

# Enlargment factor. It determines the sampling efficiency. 
#0.8 and 0.3 are recommended for parameter estimation & evidence evaluation respectively.
# (default = 0.8)
ENLARGMENT FACTOR = 0.8

# Maximum (expected) number of modes (default = 10)
NUMBER OF MODES = 10

# Number of parameters to use in the clustering algorithm for the modes
# detection (used only if MULTI-MODAL = T) (default = number of parameters) If
# this number if lower then the number of free parameters, then only the first
# <NUMBER OF CLUSTERING> parameters are used
NUMBER OF CLUSTERING = 3

# Resume from previous run (default = F)
NEST RESUME = F

# Output information on the sampling process?
VERBOSE = T

# The final PDF computed by MultiNest will contain samples (i.e. points) with
# very low probabilities, making the output file potentially very large. This
# parameter determines the minimum relative probability of the PDF samples that
# are printed to the output file, i.e. only sample with p_i >= <MIN RELATIVE
# PROBABILITY> * max(p) will be included in the output file
MIN RELATIVE PROBABILITY = 1.E-04

#****************************************************************
# ******************* End MultiNest parameters  *****************
#****************************************************************

And BEAGLE_mock_test_1.param: 

# This is a stripped down version of the BEAGLE parameter file for 
# with checking the correct installation of BEAGLE using Docker.
# This file will be used for producing artificial photometry using
# an analytic star formation history.
# Verbose mode: 0 ---> no info
#               1 ----> print only errors
#               2 ---> print errors and warnings
#               3 ----> print errors, warnings and info 
VERBOSE = 1

# Seed for the random number generator
SEED = 994189

#****************************************************************
#******************** COSMOLOGICAL PARAMETERS *******************
#****************************************************************
#
# If left blank, default parameter values will be used.

HUBBLE_0 = 

OMEGA_MATTER =

OMEGA_LAMBDA =

#****************************************************************
# ******************* TEMPLATES  *****************************
#****************************************************************

#TEMPLATES = $BEAGLE_TEMPLATES/bc03/bc03_miles_chab_spectra
TEMPLATES NEBULAR = $BEAGLE_TEMPLATES/ineb_Jan16_C100/cb2013_n2_mup100_N015_O01_deplO70_C100_Jan16
EMISSION LINES CONFIGURATION = $BEAGLE_TEMPLATES/ineb_Jan16_C100/cb2013_n2_mup100_N015_O01_deplO70_C100_Jan16_line_wavelengths_PHOTOMETRY.dat

SHRINK TEMPLATES WL RANGE = 900 60000
REBIN TEMPLATES = 10

#SPECTRAL INDICES CONFIGURATION =

#SPECTRAL INDICES CATALOGUE = 

#****************************************************************
# ******************* PHOTOMETRY *****************************
#****************************************************************

#FILTERS FILE = $BEAGLE_FILTERS/filters_fit_photometry_example.dat

FILTERS FILE = $BEAGLE_FILTERS/filters_NIRCat.dat

#PHOTOMETRIC CATALOGUE = $BEAGLE_DATA/cat_fit_photometry_example.fits

PHOTOMETRIC CATALOGUE = $BEAGLE_DATA/BEAGLE_mock.fits
#PHOTOMETRIC CATALOGUE = /Users/knh/Desktop/NIRCam/BEAGLE/BEAGLE-general-master/data/BEAGLE_mock.fits

FIT PHOTOMETRY = T
FIT SPECTRUM = F

#****************************************************************
#******************** DUST ATTENUATION AND IGM ABSORPTION *******
#****************************************************************
#
IGM ABSORPTION = Inoue

# File containing set of radiativa transfer (RT) calculations of dust attenuation of starlight
#RT CALCULATIONS = Tuffs_bulge_NEW-28-11.dat Tuffs_thick_NEW-28-11.dat Tuffs_thin_NEW-28-11.dat
#RT COMPONENTS NAMES = bulge thick_disc thin_disc

# **************************************************
# ************** SF BIN #1 **************************
# **************************************************
SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:rising
#options for analytic SFHs are 'ssp' (for a burst), 'constant', 'exponential', 'delayed', 'rising'

# In log(yr)
SF_PARAMETER  = name:tau                      type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[7.,10.5]

# in log(Z/Z_sun)
SF_PARAMETER  = name:metallicity              type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[-2.2,0.24]

# In log M/M_sun
#SF_PARAMETER  = name:mass                     type:fitted  order_priority:0  prior:distribution:uniform  prior:range:[5.,12.] 
SF_PARAMETER  = name:mass                     type:from_file  

# **************************************************
# **************  OTHER SF PARAMETERS **************************
# **************************************************

# In log(M_sun/yr)
#SF_PARAMETER  = name:sfr                type:fitted  order_priority:1  prior:distribution:uniform  prior:range:[-4.,4.]        mock:type:random 

#****************************************************************
# ******************* PRINTED OUTPUT  *****************************
#****************************************************************

RESULTS DIRECTORY = $BEAGLE_RESULTS/BEAGLE_mock_1/

#MOCK INPUT PARAMETERS = fileName:$BEAGLE_DATA/BEAGLE_mock.fits
MOCK INPUT PARAMETERS = fileName:/BEAGLE_virtual_workdir/data/BEAGLE_mock.fits

#MOCK CATALOGUE NAME = 

PRINT PHOTOMETRY = T
PRINT SPECTRUM = T
PRINT SPECTRAL INDICES = F
PRINT SF AND ChE HISTORIES = F

#****************************************************************
# ******************* PARAMETERS HANDLING  *****************************
#****************************************************************

#PDF SAMPLER FILE = $BEAGLE_PARAM_DIR/MCMC_example.param
PDF SAMPLER FILE = $BEAGLE_PARAM_DIR/MCMC_new.param

PARAMETER  = name:redshift     type:from_file

PARAMETER  = name:nebular_logU type:dependent

PARAMETER  = name:nebular_xi   type:fixed    value:0.3

PARAMETER  = name:nebular_Z    type:dependent 

PARAMETER  = name:attenuation_type  type:fixed  char_value:CF00   
# values can be: CF00, Calzetti, CCWW13_universal, CCWW13_full

PARAMETER  = name:tauV_eff     type:fitted  order_priority:0      prior:distribution:uniform   prior:range:[0.,2.]

PARAMETER  = name:mu           type:fixed   value:0.4
eclake commented 8 years ago

I managed to get the example to run cleanly with eclake/beagle:0.3.3

kevinhainline commented 8 years ago

I get the same error with eclake/beagle:0.3.3:

[KNHDesktop:~/Desktop/NIRCam/BEAGLE] knh% docker run --rm -it -v /Users/knh/Desktop/NIRCam/BEAGLE/BEAGLE-general-master:/BEAGLE_virtual_workdir --env-file env.list eclake/beagle:0.3.3 1 /BEAGLE_virtual_workdir/params/BEAGLE_mock_test_1.param

 Reading Filter File: /BEAGLE_virtual_workdir/filters/FILTERBIN.RES
 298 filters defined, out of 300                                maximum     ...done
 ---> fixed                         sfh_type                                   0
 ---> from_file                     mass                                       0
 ---> from_file                     redshift                                   0
 ---> dependent                     nebular_logU                               0
 ---> fixed                         nebular_xi                                 0
 ---> dependent                     nebular_Z                                  0
 ---> fixed                         attenuation_type                           0
 ---> fitted                        tauV_eff                                   0
 ---> fixed                         mu                                         0
 ---> fitted                        tau                                        1
 ---> fitted                        metallicity                                1
 n_fitted:            3
 *****************************************************
 MultiNest v3.9
 Copyright Farhan Feroz & Mike Hobson
 Release Oct 2014

 no. of live points =  300
 dimensionality =    3
 *****************************************************
 Starting MultiNest
 generating live points
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
         [ERROR: "locate" failed, the condition x(i) <= x < x(i+1) is not satisfied! ]
 [FUNCTION: "locate_sp" ]
 [MODULE: "lib_interpolation" ]
 [BACKTRACE:   
#0  0x7FE9ED84C777
#1  0x5362C4 in print_backtrace at lib_messages.f90:299
#2  0x52047C in __lib_interpolation_MOD_locate_sp at lib_interpolation.f90:1199 (discriminator 1)
#3  0x52117B in __lib_interpolation_MOD_get_linear_weights_sp at lib_interpolation.f90:2868
#4  0x526822 in __lib_interpolation_MOD_linear_interpolate_grid_2d_sp at lib_interpolation.f90:2185 (discriminator 1)
#5  0x43A4BF in __sps_routines_MOD_get_sps_spectra_age_z at SPS_routines.f90:157 (discriminator 2)
#6  0x46AF81 in __galaxy_sed_MOD_compute_sed at galaxy_SED.f90:216
#7  0x47A3BE in __likelihood_function_MOD_likelihood at likelihood_function.f90:150
#8  0x47DCD3 in __likelihood_function_MOD_mc_likelihood at likelihood_function.f90:89 (discriminator 3)
#9  0x57D559 in getloglike at nested_sampling.f90:193 (discriminator 2)
#10  0x5A2606 in __nested_MOD_gen_initial_live at nested.F90:536
#11  0x5A33F0 in __nested_MOD_nestsample at nested.F90:352
#12  0x5A468C in __nested_MOD_nestrun at nested.F90:239
#13  0x57D78D in __nested_sampling_MOD_run_nested_sampling at nested_sampling.f90:133
#14  0x572CDA in __prosit_MOD_sample_pdf at PROSIT.f90:235 (discriminator 1)
#15  0x415659 in beagle at BEAGLE.f90:436 (discriminator 2)
 ]
 [ ***************************************** ]
 [ ----> EXITING FUNCTION / SUBROUTINE <---- ]                                          
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 [ +++++ ERROR DETECTED: Do you want to continue with the execution of the program? (y/n)]
jacopo-chevallard commented 8 years ago

This is strange... This

#5  0x43A49F in __sps_routines_MOD_get_sps_spectra_age_z at SPS_routines.f90:157 (discriminator 2)
#6  0x46A375 in __galaxy_sed_MOD_compute_sed at galaxy_SED.f90:215

tells us that problem arises when interpolating the input SSPs (which are given at discrete points in an age-metallicity grid) onto the pairs (age, metallicity) describing the model SFH.

Kevin, can you make a couple tests: change the parameter mass from from_file to fitted, i.e. uncommenting the line before the one where you define 

SF_PARAMETER  = name:mass                     type:from_file

and commenting out the line above.

Re-run BEAGLE, and tell me if anything has changed.

Repeat the same test by changing 

SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:rising

into

SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:delayed

Then, does the example Emma provided run smoothly?

I don't think I'll be able to do more tests today, but I can try to reproduce your error tomorrow. Can you attach in a zip all files you're using to run your fitting? (param, filters, etc...)

eclake commented 8 years ago

Sorry Kevin, I haven't managed to reproduce the error locally, and have made sure that the docker image I'm using is the same as the version on dockerhub.

We might need an independent tester with the files that you've provided here.

Here is the link to the working directory I'm using I don't suggest copying all of it as it includes the template directory, but perhaps you could take a look at the files you expect me to be using and check them against the ones you're using?

kevinhainline commented 8 years ago

The code ran when I used this line:

SF_PARAMETER  = name:mass                     type:fitted  order_priority:0  prior:distribution:uniform  prior:range:[5.,12.]

instead of

SF_PARAMETER  = name:mass                     type:from_file

Should the mass have been in lg(mass) units? The input file has them in terms of solar masses.

If I go back to the original parameter file, and only change the star formation history:

from:

SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:rising

to:

SF_PARAMETER  = name:sfh_type                 type:fixed  char_value:delayed

I once again get the previous error:

[KNHDesktop:~/Desktop/NIRCam/BEAGLE] knh% docker run --rm -it -v /Users/knh/Desktop/NIRCam/BEAGLE/BEAGLE-general-master:/BEAGLE_virtual_workdir --env-file env.list eclake/beagle:0.3.3 1 /BEAGLE_virtual_workdir/params/BEAGLE_mock_test_2.param

 Reading Filter File: /BEAGLE_virtual_workdir/filters/FILTERBIN.RES
 298 filters defined, out of 300                                maximum     ...done
 ---> fixed                         sfh_type                                   0
 ---> from_file                     mass                                       0
 ---> from_file                     redshift                                   0
 ---> dependent                     nebular_logU                               0
 ---> fixed                         nebular_xi                                 0
 ---> dependent                     nebular_Z                                  0
 ---> fixed                         attenuation_type                           0
 ---> fitted                        tauV_eff                                   0
 ---> fixed                         mu                                         0
 ---> fitted                        tau                                        1
 ---> fitted                        metallicity                                1
 n_fitted:            3
 *****************************************************
 MultiNest v3.9
 Copyright Farhan Feroz & Mike Hobson
 Release Oct 2014

 no. of live points =  300
 dimensionality =    3
 *****************************************************
 Starting MultiNest
 generating live points
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
         [ERROR: "locate" failed, the condition x(i) <= x < x(i+1) is not satisfied! ]
 [FUNCTION: "locate_sp" ]
 [MODULE: "lib_interpolation" ]
 [BACKTRACE:   
#0  0x7F2A82F6F777
#1  0x5362C4 in print_backtrace at lib_messages.f90:299
#2  0x52047C in __lib_interpolation_MOD_locate_sp at lib_interpolation.f90:1199 (discriminator 1)
#3  0x52117B in __lib_interpolation_MOD_get_linear_weights_sp at lib_interpolation.f90:2868
#4  0x526822 in __lib_interpolation_MOD_linear_interpolate_grid_2d_sp at lib_interpolation.f90:2185 (discriminator 1)
#5  0x43A4BF in __sps_routines_MOD_get_sps_spectra_age_z at SPS_routines.f90:157 (discriminator 2)
#6  0x46AF81 in __galaxy_sed_MOD_compute_sed at galaxy_SED.f90:216
#7  0x47A3BE in __likelihood_function_MOD_likelihood at likelihood_function.f90:150
#8  0x47DCD3 in __likelihood_function_MOD_mc_likelihood at likelihood_function.f90:89 (discriminator 3)
#9  0x57D559 in getloglike at nested_sampling.f90:193 (discriminator 2)
#10  0x5A2606 in __nested_MOD_gen_initial_live at nested.F90:536
#11  0x5A33F0 in __nested_MOD_nestsample at nested.F90:352
#12  0x5A468C in __nested_MOD_nestrun at nested.F90:239
#13  0x57D78D in __nested_sampling_MOD_run_nested_sampling at nested_sampling.f90:133
#14  0x572CDA in __prosit_MOD_sample_pdf at PROSIT.f90:235 (discriminator 1)
#15  0x415659 in beagle at BEAGLE.f90:436 (discriminator 2)
 ]
 [ ***************************************** ]
 [ ----> EXITING FUNCTION / SUBROUTINE <---- ]                                          
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 [ +++++ ERROR DETECTED: Do you want to continue with the execution of the program? (y/n)]

I don't know which example Emma provided that you are referring to, so I can't run that. I'll package everything up as a zip file and attach it in a minute or two.

jacopo-chevallard commented 8 years ago

Yes, the mass must be in units of log10 !

kevinhainline commented 8 years ago

Ok, let me re-run things with log(mass) units.

It's running! And it took about 3 minutes to run things on this one object. It looks like this might have been the issue?

jacopo-chevallard commented 8 years ago

yes indeed! all units must be of the same type as defined in the parameter file!

kevinhainline commented 8 years ago

One more quick question - how is the "UV slope" as defined by BEAGLE calculated? I know this is the wrong place to ask this, but I just remembered I meant to ask in the telecon. I want to double-check that the resulting fits have UV slopes that make agree with our input slope.

jacopo-chevallard commented 8 years ago

The UV slope ic computed by considering the fluxes in the GALEX FUV and NUV bands, then fitting a straight line to the log of the fluxes.

Closing the issue now, assuming that the fitting is working properly.