search

LunarEngineer / MentalGymnastics

This is a school project with potential.
MIT License
1 stars 0 forks source link

WANN? #10

Open LunarEngineer opened 3 years ago

LunarEngineer commented 3 years ago

Our original plan when constructing the gym was for Functions to have the capability to save and load weights which would then persist through time and be updated when the Function is used.

Alternatively, we could mimic the WANN style of not training and replicate the manner they use to fill all the weights in a structure with a single randomly drawn value while conducting multiple trials with different randomly drawn values.

I would appreciate any thoughts or feedback the team has on what you perceive the downstream effects of either method.

hagopi1611 commented 3 years ago

I think that we'll see better results if we use "pretrained" nets; that is, saving and loading the fully trained weights. I think extending this to the WANN style would be interesting, but maybe we need to capitalize on all the performance we can get given the short search time we have. But I don't at all have strong feelings on this.

cizumigawa3 commented 3 years ago

I think Vahe is right that we'll see better results if we were to persist the weights. At the same time, I think persisting the weights adds a little more overhead and complexity to our function bank/training methods. In the WANN scheme, we just need the functions to have their respective layers so we can tie the net together and go straight to getting metrics. Whereas persisting the weights would require us to save the weights somewhere in addition to training a little before getting the metrics, etc.

With about a week to go, I think it might be better to just get something working (even if it doesn't work so well?) But I'm also not tied to either opinion. The overhead of persisting isn't significant enough to rule it out completely.