search

opencog / destin

Deep SpatioTemporal Inference Network (DeSTIN) for vision processing
http://wiki.opencog.org/w/DeSTIN
GNU Lesser General Public License v3.0
34 stars 28 forks source link

More flexible structs + configurable sizes of nodes #6

Closed jswiergo closed 10 years ago

jswiergo commented 10 years ago

I have added possibility to configure sizes of squares in DeSTIN topology. You can use nci[] parameter to create nodes with different sizes for each layer separately. Previously there was a rigid structure with each node above layer 0 being 2x2 square.

I have also rewritten a few internal structures towards flexibility for adding future functionalities like:

Previous rigid structures:

are replaced by:

Now the observation is being made dynamically by "looking around" via pointers. Input offsets are used only for nodes from layer 0. The links between parents and children should allow modelling of future topologies easier.

Note that: destin->nci[node->layer] differs from node->ni. The former means the number of inputs/links from childs, the latter means total input dimensionality.

Alex-van-der-Peet commented 10 years ago

Hi,

This is pretty awesome stuff!! Great job!

Alex

From: Jacek Świergocki [mailto:notifications@github.com] Sent: 06 October 2013 20:39 To: opencog/destin Subject: [destin] More flexible structs + configurable sizes of nodes (#6)

I have added possibility to configure sizes of squares in DeSTIN topology. You can use nci[] parameter to create nodes with different sizes for each layer separately. Previously there was a rigid structure with each node above layer 0 being 2x2 square.

I have also rewritten a few internal structures towards flexibility for adding future functionalities like:

Previous rigid structures:

Note that: destin->nci[node->layer] differs from node->ni. The former means the number of inputs/links from childs, the latter means total input dimensionality.

You can merge this Pull Request by running

git pull https://github.com/jswiergo/destin dynamic-structs

Or view, comment on, or merge it at:

https://github.com/opencog/destin/pull/6

Commit Summary

File Changes

Patch Links:

tpsjr7 commented 10 years ago

I think you have some good stuff here, thanks.

I found that there's potentially an issue with the clustering quality with these changes. I'm not sure what may be causing it.

Here's a graph generated from Bindings/Pythong/letters_experiment.py before these changes at commit f93a954e: https://www.dropbox.com/s/kbtkpex1sa5uzeo/at%20f93a954e64.png

Stats at the end of the 4000 iterations: Variance: 0.00217987503856 Sep: 0.124732129276 Quality: 0.122552253306

Here's after these changes ( 2757775a ): https://www.dropbox.com/s/0gdv8x2tuwmdaiw/after_change.png

Variance: 0.0106844799593 Sep: 0.110241264105 Quality: 0.0995567813516

As you can see after these changes the learning graph seems different and the quality appears slightly lower at the end. Perhaps the particular random number sequence just plays out better by chance with the old way and that if we put a different seed on the random number generator each run then we would not be able to see much of a difference between the two after a few more runs. I was wondering if you could try that just to make sure these changes are not affecting the quality.

You can get the test images for letters_experiments.py at https://www.dropbox.com/s/jpqikjxfo2slfb6/destin_toshare.zip

Thanks for you good work.

jswiergo commented 10 years ago

Ted, Thanks for feedback. The different seed reason seems plausible. I will find out why is the difference. Here 8690883 I changed how the shared centroids (n->mu) for uniform destin are initialized. Now they are initialized with random number once for a layer (which is more logical). Before they were initialialized for each node of layer. So it changed random number sequence.

jswiergo commented 10 years ago

I have done more tests with different random seeds. For sure the difference comes from 8690883. The commits before give the same results. It seems that the algorithm quality is sensitive to initial positions of centroids.

The new code gives slightly better results with a seed being a satan's number (srand(666)): Variance: 0.00210919347592 Sep: 0.124882213771 Quality: 0.12277302146

tpsjr7 commented 10 years ago

Yep, looks like it was just influence by the choice of the random seed. Thanks again.