Closed jswiergo closed 10 years ago
Hi,
This is pretty awesome stuff!! Great job!
Alex
From: Jacek Świergocki [mailto:notifications@github.com] Sent: 06 October 2013 20:39 To: opencog/destin Subject: [destin] More flexible structs + configurable sizes of nodes (#6)
I have added possibility to configure sizes of squares in DeSTIN topology. You can use nci[] parameter to create nodes with different sizes for each layer separately. Previously there was a rigid structure with each node above layer 0 being 2x2 square.
I have also rewritten a few internal structures towards flexibility for adding future functionalities like:
Previous rigid structures:
Note that: destin->nci[node->layer] differs from node->ni. The former means the number of inputs/links from childs, the latter means total input dimensionality.
You can merge this Pull Request by running
git pull https://github.com/jswiergo/destin dynamic-structs
Or view, comment on, or merge it at:
https://github.com/opencog/destin/pull/6
Commit Summary
File Changes
Patch Links:
I think you have some good stuff here, thanks.
I found that there's potentially an issue with the clustering quality with these changes. I'm not sure what may be causing it.
Here's a graph generated from Bindings/Pythong/letters_experiment.py before these changes at commit f93a954e: https://www.dropbox.com/s/kbtkpex1sa5uzeo/at%20f93a954e64.png
Stats at the end of the 4000 iterations: Variance: 0.00217987503856 Sep: 0.124732129276 Quality: 0.122552253306
Here's after these changes ( 2757775a ): https://www.dropbox.com/s/0gdv8x2tuwmdaiw/after_change.png
Variance: 0.0106844799593 Sep: 0.110241264105 Quality: 0.0995567813516
As you can see after these changes the learning graph seems different and the quality appears slightly lower at the end. Perhaps the particular random number sequence just plays out better by chance with the old way and that if we put a different seed on the random number generator each run then we would not be able to see much of a difference between the two after a few more runs. I was wondering if you could try that just to make sure these changes are not affecting the quality.
You can get the test images for letters_experiments.py at https://www.dropbox.com/s/jpqikjxfo2slfb6/destin_toshare.zip
Thanks for you good work.
Ted, Thanks for feedback. The different seed reason seems plausible. I will find out why is the difference. Here 8690883 I changed how the shared centroids (n->mu) for uniform destin are initialized. Now they are initialized with random number once for a layer (which is more logical). Before they were initialialized for each node of layer. So it changed random number sequence.
I have done more tests with different random seeds. For sure the difference comes from 8690883. The commits before give the same results. It seems that the algorithm quality is sensitive to initial positions of centroids.
The new code gives slightly better results with a seed being a satan's number (srand(666)): Variance: 0.00210919347592 Sep: 0.124882213771 Quality: 0.12277302146
Yep, looks like it was just influence by the choice of the random seed. Thanks again.
I have added possibility to configure sizes of squares in DeSTIN topology. You can use nci[] parameter to create nodes with different sizes for each layer separately. Previously there was a rigid structure with each node above layer 0 being 2x2 square.
I have also rewritten a few internal structures towards flexibility for adding future functionalities like:
Previous rigid structures:
are replaced by:
Now the observation is being made dynamically by "looking around" via pointers. Input offsets are used only for nodes from layer 0. The links between parents and children should allow modelling of future topologies easier.
Note that: destin->nci[node->layer] differs from node->ni. The former means the number of inputs/links from childs, the latter means total input dimensionality.