Arrays of values to vary

[DanBThomas]

Hi

I have a parameter array in CLASS that I have been successfully using in montepython, where the parameters are varied as part of the MCMC.

I attempted to run a new chain recently where I increased the size of the array to 10. When I do so I get an error message (below).

I have checked for errors regarding the size of the array I am using, but it all seems to be specified correctly. I just get this error when the size of the array is increased to 10.

Do I need to do anything different in montepython with a varying array of size 10 compared to size 9?

Thanks Dan

The code halts at this point in the output: "Deduced starting covariance matrix: ['omega_b', 'omega_fld', 'H0', 'ln10^{10}A_s', 'n_s', 'tau_reio', 'w_values1', 'w_values__2', 'w_values3', 'w_values4', 'w_values__5', 'w_values6', 'w_values7', 'w_values__8', 'w_values9', 'w_values__10', 'A_cib_217', 'xi_sz_cib', 'A_sz', 'ps_A_100_100', 'ps_A_143_143', 'ps_A_143_217', 'ps_A_217_217', 'ksz_norm', 'gal545_A_100', 'gal545_A_143', 'gal545_A_143_217', 'gal545_A_217', 'galf_EE_A_100', 'galf_EE_A_100_143', 'galf_EE_A_100_217', 'galf_EE_A_143', 'galf_EE_A_143_217', 'galf_EE_A_217', 'gal f_TE_A_100', 'galf_TE_A_100_143', 'galf_TE_A_100_217', 'galf_TE_A_143', 'galf_TE_A_143_217', 'galf_TE_A_217', 'calib_100T', 'calib_217T', 'A_planck'] [[ 4.08e-04 0.00e+00 0.00e+00 ..., 0.00e+00 0.00e+00 0.00e+00] [ 0.00e+00 3.50e-03 0.00e+00 ..., 0.00e+00 0.00e+00 0.00e+00] [ 0.00e+00 0.00e+00 3.97e+01 ..., 0.00e+00 0.00e+00 0.00e+00] ..., [ 0.00e+00 0.00e+00 0.00e+00 ..., 5.33e-01 0.00e+00 0.00e+00] [ 0.00e+00 0.00e+00 0.00e+00 ..., 0.00e+00 1.96e+00 0.00e+00] [ 0.00e+00 0.00e+00 0.00e+00 ..., 0.00e+00 0.00e+00 6.25e-02]] BFLike Ntemp = 2876 BFLike Nq = 1407 BFLike Nu = 1407 BFLike Nside = 16 BFLike Nwrite = 32393560 cls file appears to have 5+ columns assuming it is a CAMB file with l, TT, EE, BB, TE info = 0 "

With the error: "Traceback (most recent call last): File "./montepython/MontePython.py", line 40, in sys.exit(run()) File "/home/thomas/montepython_public-2.1.4/montepython/run.py", line 44, in run sampler.run(cosmo, data, command_line) File "/home/thomas/montepython_public-2.1.4/montepython/sampler.py", line 43, in run mcmc.chain(cosmo, data, command_line) File "/home/thomas/montepython_public-2.1.4/montepython/mcmc.py", line 281, in chain Cholesky, Inverse_Cholesky, Rotation) is True: File "/home/thomas/montepython_public-2.1.4/montepython/mcmc.py", line 188, in get_new_position data.update_cosmo_arguments() File "/home/thomas/montepython_public-2.1.4/montepython/data.py", line 795, in update_cosmo_arguments values.append(self.cosmo_arguments[other_elem]) KeyError: 'w_values__2' "

The w-values array is the array that I increased to 10 elements.

[DanBThomas]

Follow up to this: Assuming I have understood correctly, the problem is caused by the line "match = re.search(r'%s__([2-9])' % original_name,other_elem)" where the range [2-9] can't simply be increased because of how python works.

I think I have found a way around that seems to work, but I am not very familiar with Python so I am sure there is a more elegant way to do it.

Attached is the old and new section from data.py below, in case it is useful to anyone. Or someone can spot a potential problem with what I've done! Dan data_mods.txt

[ThomasTram]

Hi Dan

We limited this to 2-9 since we never thought anyone would need to vary more than 9 extra parameters. I think there is an easy fix - I didn't test it, but perhaps you can do it?

Instead of

# Recover the values of all the other elements
values = [self.cosmo_arguments[elem]]
for other_elem in self.get_mcmc_parameters(['cosmo']):
    match = re.search(r'%s__([2-9])' % original_name,
                                      other_elem)

Just write

# Recover the values of all the array elements
values = []
for other_elem in self.get_mcmc_parameters(['cosmo']):
    match = re.search(r'%s__([1-9])' % original_name,
                                      other_elem)

You may even write re.search(r'%s__([1-99])' but 1-9 will work fine. The problem is that 2-9 does not match 10-19. It was 2-9 just so that __1 would not be double counted. But leaving the values array empty should fix it.

Cheers, Thomas

[DanBThomas]

This solution doesn't seem to work with the range [1-10].

My understanding of the syntax here when I looked into it was that the [1-9] runs across a single character rather than a value, so the maximum number of values is 10 (0-9).

Python seems to ignore the second digit of the second number in the range (i.e. [1-10] was treated as [1-1]; [1-30] was treated as [1-3] etc).

[ThomasTram]

Hi Dan

The problem is just the reconstruction of the original name. Here is a full solution:

elif len(elem)>3 and elem[-3:]==r'__1':
    original_name = elem[:-3]
    # Recover the values of all the other elements
    values = []
    for other_elem in self.get_mcmc_parameters(['cosmo']):
        match = re.search(r'%s__([1-9])' % original_name,
                                      other_elem)
        if match:
            values.append(self.cosmo_arguments[other_elem])
    # create the cosmo_argument
    self.cosmo_arguments[original_name] = ', '.join(
        ['%g' % value for value in values])
    # Delete the now obsolete entries of the dictionary
    for index in range(1, len(values)+1):
        del self.cosmo_arguments[
                        original_name + '__%i' % index]

Let me know if this works for you.

Cheers, Thomas

[DanBThomas]

Hey

I'm afraid I'm getting the same type of error.

As soon as I change [1-9] to [1-10] (or change it to [1-12], [1-30] etc) in the line " match = re.search(r'%s__([1-9])' % original_name, other_elem)" Then I start getting problems.

When I tried to look into this, my understanding is that this regular expression is trying to match a character rather than a value, and of course any 2 digit number is not a character.

This is why I was looking at using expressions like [0][2-9] and [1][0-9].

Does that make sense?

[DanBThomas]

Sorry Thomas, I think I may have misunderstood you here. Am I supposed to use your solution without changing [1-9] to [1-12] etc? Thanks Dan