rhayes777 / PyAutoFit

PyAutoFit: Classy Probabilistic Programming
https://pyautofit.readthedocs.io/
MIT License
59 stars 11 forks source link

Optimizer grid search does not use updated optimzer settings in pipeline #21

Closed Jammy2211 closed 5 years ago

Jammy2211 commented 5 years ago

The following phase uses the non_linear.ini config, as oppoed to the overwrites at the end.


    phase3.optimizer.const_efficiency_mode = True
    phase3.optimizer.n_live_points = 20
    phase3.optimizer.sampling_efficiency = 0.3
rhayes777 commented 5 years ago

All configuration is now copied for Multinest and DHS when creating new optimizers during a gridsearch

Jammy2211 commented 5 years ago

The fix woked in that it made the setting use the pipeline settings.

However, it messed up the order of the priors. On autofit v 0.17.1, below is an example model.results file:

Most likely model, Likelihood = -38009.097

lens_mass_einstein_radius 0.973 source_light_centre_0 0.084 source_light_centre_1 0.184 source_light_axis_ratio 0.466 source_light_phi 25.036 source_light_intensity 0.329 source_light_effective_radius 0.48 source_light_sersic_index 4.138 subhalo_mass_kappa_s 0.002 subhalo_mass_scale_radius 0.04 subhalo_mass_centre_0 -1.167 subhalo_mass_centre_1 -0.112

On autofitv 0.17.2, it looks like this:

Most likely model, Likelihood = -38009.097

lens_mass_einstein_radius 0.973 source_light_centre_0 0.084 source_light_centre_1 0.184 source_light_axis_ratio 0.466 source_light_phi 25.036 source_light_intensity 0.329 source_light_effective_radius 0.48 source_light_sersic_index 4.138 subhalo_mass_centre_0 -1.167 subhalo_mass_centre_1 -0.112 subhalo_mass_kappa_s 0.002 subhalo_mass_scale_radius 0.04

As you can see, the subhalo centre parameters are in a different location.

The problem is that, for both versions, the multinest output stores the results in the same order. That is, the last two parameters are always centre_0 and centre_1. Thus, on autofit 0.17.2, your fix isn't correctly updaitng the order of the priors in the model mapper.

rhayes777 commented 5 years ago

This was caused by copying all attributes from the root optimiser to child optimisers. In doing so the variable attribute was overwritten which is why the priors returned to their default order, rather than the grid search priors being last (having been updated).

The issue is now resolved.

Jammy2211 commented 5 years ago

This fixed the issue, however the sim link for the grid search also needs to be updated.

That is, the solution to this means that the sim link still points to the pipeline_name/phase_name/optimizzer folder, no the pipeline_name/phase_name/grid_parameter_values_00_etc/optimizer folder.

rhayes777 commented 5 years ago

Fixed

Jammy2211 commented 5 years ago

The optimizer backup folder also needs updating, as it currently does not write to the grid-search's sub folder :P.

Jammy2211 commented 5 years ago

Also happening with:

phase.info execution_time optimizer.pickle .metadata .start_time

rhayes777 commented 5 years ago

optimizer.pickle is saved and loaded at the piepline level. Individual phases in a grid search don't have an associated pickle anyway

rhayes777 commented 5 years ago

Same with .metadata

rhayes777 commented 5 years ago

Paths that were created in the NonLinearOptimizer constructor are now computed properties.

AttributeError is caught when copying dictionary items from the root optimizer in grid search to its children. As such computed paths are no longer overridden.