Update

[RoyStegeman]

So the NGA algorithm that I replaced the run_and_fit function in n3fit with back in december has now been written in classes. The genetic optimizer (more to come after the first is working) can be implemented implemented in similar fashion the the gradient descent based methods that are already part of Keras, by using the GAModel instead of Keras's Model. Although it does not seem to be able to improve the network used to train on MNIST data in the test.py file (perhaps because the network is to big/parameter settings are more sensitive than n3fit?), it was able to fit the pdf's to some extend.

The fit function in GAModel does not (yet) have all the same functionality as the fit function of keras. Since I am not familiar with what the data used for n3fit looks like, the method might have to be adjusted for different types of input data (at the moment it just aims to decrease the first value in the output of the evaluate function). Also, how will we deal with the functionality incorporated in MetaModel?

Right now my question would be: what do we do now? I guess that if I need to make some adjustments to get it to work within n3fit, it might be best to discuss in some more detail what the n3fit fitting methodology should look like (over skype). I think it would be best to first have the NGA working completely such that I then know exactly what is needed to implement the other methods.

Also, at the moment the method also assumes that in the model the non trainable data comes after the trainable data, here I will add more flexibility later.

[scarlehoff]

Since I am not familiar with what the data used for n3fit looks like, the method might have to be adjusted for different types of input data

The input data should not matter as you don't do anything with it as it will be ultimately passed down to the evaluate or predict functions which are also Keras so you don't have to touch it.

how will we deal with the functionality incorporated in MetaModel?

You don't have to worry about that for now since that functionality inherits from GAModel. "Your task" is to produce an output as similar to Keras as possible so that MetaModel thinks it is dealing with Keras.

I think it would be best to first have the NGA working completely such that I then know exactly what is needed to implement the other methods.

Yes, I agree.

Also, at the moment the method also assumes that in the model the non trainable data comes after the trainable data, here I will add more flexibility later.

Yes, this is ok. You can deal with this later.

what do we do now?

Have a look at the comments I put in the code (they are all quick fixes) and I'll have a go at it later to understand what's missing

(over skype) We can try tomorrow afternoon/thursday if that works for you?

[scarlehoff]

Perfect, with this changes the code can be imported by n3fit and (if using a GD optimizer) will work as expected.

[RoyStegeman]

The input data should not matter as you don't do anything with it as it will be ultimately passed down to the evaluate or predict functions which are also Keras so you don't have to touch it.

True, although I'm pretty sure keras fit has a bit more functionality when it comes to different types of input data. Right now I only looked at the 'input'/'target' format, but I'll try to incorporate the rest of keras' functionality as well.

You don't have to worry about that for now since that functionality inherits from GAModel. "Your task" is to produce an output as similar to Keras as possible so that MetaModel thinks it is dealing with Keras.

Alright, thanks.

Have a look at the comments I put in the code (they are all quick fixes) and I'll have a go at it later to understand what's missing

Thanks, I added the suggested changes with exception of the class in the class.

We can try tomorrow afternoon/thursday if that works for you?

Tomorrow afternoon I have an empty schedule with regards to the school from around 14.30 onward, so that would be perfect. What time would suite you?

[scarlehoff]

True, although I'm pretty sure keras fit has a bit more functionality when it comes to different types of input data. Right now I only looked at the 'input'/'target' format, but I'll try to incorporate the rest of keras' functionality as well.

Uhm, right, it has some stuff like randomizing the input and so but don't worry about that for the moment as not everything is necessarily useful for GA.

Tomorrow afternoon I have an empty schedule with regards to the school from around 14.30 onward, so that would be perfect. What time would suite you?

Let's do 14.30 then (or 15.00 if you prefer)

[RoyStegeman]

Let's do 14.30 then (or 15.00 if you prefer)

14.30 is fine. Thanks for the feedback.

[RoyStegeman]

My skype adres is roystegeman@live.nl

[scarlehoff]

With the last batch of comments taken into consideration I am managing to run a fit (not sure how well is it fitting, but it runs, which is a lot!) with the current master of n3fit.

[scarlehoff]

Some of the comments you made on the other PR should be added too (such as the fact that parameters have to be passed down in the fit function or that the keys in the dict should be checked!)

[RoyStegeman]

Some of the comments you made on the other PR should be added too (such as the fact that parameters have to be passed down in the fit function or that the keys in the dict should be checked!)

That's done. It took me a while to figure out how to not mess up the repo with changes I made locally while synchronising.

[scarlehoff]

That's done. It took me a while to figure out how to not mess up the repo with changes I made locally while synchronising.

Ah, sorry, I didn't see them in the list of commits (sundays I try to avoid looking too closely the code :P)

[RoyStegeman]

No you were right, I made the commits after your comment. It took me that long because I didn't (don't) understand git ;)

[scarlehoff]

I've added a setup.py. I haven't changed anything from the code, but now you can install it by doing

python setup.py develop

So that you can import kerasGA.GAModel from anywhere in the python environment.

[RoyStegeman]

Can we merge this with master? We can then create new branches if we want to work on items that are still to-do.

[scarlehoff]

Let's wait until the fit with n3fit finishes.

[scarlehoff]

I'll merge this to master since the first milestone (having a functional NGA and a having done a fit with n3fit) is already reached. Any future changes should come in separated PR.