search

N3PDF / evolutionary_keras

An evolutionary algorithm implementation for Keras
GNU General Public License v3.0
10 stars 7 forks source link

added an initial version of a CMA optimizer #24

Closed RoyStegeman closed 4 years ago

RoyStegeman commented 4 years ago

I added a first version of a CMA-ES optimizer in keras as suggested in #16 , but some work is still to be done:

The optimizer used is https://pypi.org/project/cma/ by Nikolaus.Hansen, which (according to himself) is supposed to be more efficient than the relatively simple implementation discussed in his papers.

scarlehoff commented 4 years ago

I just tried running this in n3fit and seems to work ok (yey!) and the chi2 goes down. It is much slower but this is expected (and don't really now whether it can be improved, but we can try)

RoyStegeman commented 4 years ago

Yes that was to be expected. I have not had a look at Hansen's source code, but CMA is based on matrix manupulation, which I believe python is not too well suited for. Also, the n3fit network is somewhat large for using CMA anyway, but there is nothing we can do about that.

I just tried running this in n3fit and seems to work ok (yey!)

Thanks for testing, I had not run it in n3fit yet myself. Good to know it did not bump into any unforeseen errors.

RoyStegeman commented 4 years ago

@scarlehoff I would like to merge this. Can you have a look?

RoyStegeman commented 4 years ago

@scarlehoff Thank you for the many comments. I implemented all of them.

scarlehoff commented 4 years ago

This is the version you used for the last report in NNPDF right?

I'll change the setup.py and the conda recipe and then I'll merge this in if that's ok.

RoyStegeman commented 4 years ago

Yes but let me check, I think there are some things that I should adjust for more general use.

scarlehoff commented 4 years ago

ok, no rush, we can safely wait until the proceedings are published.

RoyStegeman commented 4 years ago

Okay, this should do. Feel free to merge after you made the changes to setup.py.

RoyStegeman commented 4 years ago

I think this should do, as far as making evolutionary-keras future proof is concerned. Pylint does give import errors, but I think that's a problem in pylint. Can you have a look at this error and the setup.py @scarlehoff ?

scarlehoff commented 4 years ago

This last part was better suited on a different branch (on a different PR) which can point to this one.

RoyStegeman commented 4 years ago

You want me to move them to a separate branch before you have a look?

scarlehoff commented 4 years ago

You want me to move them to a separate branch before you have a look?

Yes please, also because I think we want to wait for Tensorflow 2.2 https://github.com/tensorflow/tensorflow/releases/tag/v2.2.0-rc3 because they have changed the metrics in the tensorflow.keras module which can interfere with evolutionary_keras.

scarlehoff commented 4 years ago

I'll merge this as soon as the tests pass

RoyStegeman commented 4 years ago

Sure, thanks for checking the PRs.