Optimizing synapse connection.

[calumroy]

The HTM could be optimised to give better performance if references in each cell to the synapses connect to them were stored. This would make it quick to find out which synapses should be checked in which segments when trying to find which cells are predicting. This issue is related to increasing the speed of the HTM #5.

[calumroy]

Below is a profile for the latest HTM code. This profile was taken after running the HTM for 150 steps. See the tes_runtime nosetest. This was done so significant numbers of synpases had been created before profiling.

 5145339 function calls in 10.741 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.004    0.004    0.045    0.045 HTM_Balancer.py:1012(inhibition)
     2400    0.001    0.000    0.001    0.000 HTM_Balancer.py:1024(<lambda>)
        1    0.015    0.015    0.159    0.159 HTM_Balancer.py:1062(learning)
        1    0.003    0.003    9.639    9.639 HTM_Balancer.py:1090(updateActiveState)
        1    0.010    0.010    0.011    0.011 HTM_Balancer.py:1215(updatePredictiveState)
        1    0.048    0.048    0.397    0.397 HTM_Balancer.py:1264(sequenceLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1394(updateRegionInput)
        1    0.000    0.000   10.741   10.741 HTM_Balancer.py:1445(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1546(updateHTMInput)
     2400    0.001    0.000    0.001    0.000 HTM_Balancer.py:160(updateBoost)
        1    0.000    0.000   10.741   10.741 HTM_Balancer.py:1614(spatialTemporal)
     2400    0.024    0.000    0.171    0.000 HTM_Balancer.py:239(columnActiveNotBursting)
        1    0.017    0.017    0.017    0.017 HTM_Balancer.py:293(updateOutput)
     2400    0.057    0.000    0.248    0.000 HTM_Balancer.py:312(updateConnectedSynapses)
     2460    0.063    0.000    0.137    0.000 HTM_Balancer.py:394(neighbours)
       60    0.001    0.000    0.002    0.000 HTM_Balancer.py:415(updateOverlapDutyCycle)
       60    0.001    0.000    0.002    0.000 HTM_Balancer.py:429(increasePermanence)
       60    0.001    0.000    0.002    0.000 HTM_Balancer.py:437(kthScore)
     2400    0.008    0.000    0.024    0.000 HTM_Balancer.py:449(updateActiveDutyCycle)
     2400    0.007    0.000    0.007    0.000 HTM_Balancer.py:466(maxDutyCycle)
    24000    0.045    0.000    0.060    0.000 HTM_Balancer.py:473(deleteSegments)
       60    0.000    0.000    0.005    0.000 HTM_Balancer.py:524(columnActiveAdd)
       60    0.000    0.000    0.000    0.000 HTM_Balancer.py:53(__init__)
      600    0.004    0.000    0.048    0.000 HTM_Balancer.py:533(activeStateAdd)
       60    0.000    0.000    0.005    0.000 HTM_Balancer.py:551(learnStateAdd)
     3112    0.015    0.000    0.017    0.000 HTM_Balancer.py:560(columnActiveState)
    24000    0.128    0.000    0.144    0.000 HTM_Balancer.py:568(activeState)
    24600    0.122    0.000    0.137    0.000 HTM_Balancer.py:576(predictiveState)
  1464060    7.564    0.000    8.494    0.000 HTM_Balancer.py:584(learnState)
       60    1.117    0.019    9.576    0.160 HTM_Balancer.py:613(randomActiveSynapses)
     1200    0.002    0.000    0.002    0.000 HTM_Balancer.py:740(getBestMatchingSegment)
       60    0.001    0.000    0.002    0.000 HTM_Balancer.py:772(getBestMatchingCell)
       60    0.000    0.000    9.576    0.160 HTM_Balancer.py:814(getSegmentActiveSynapses)
       60    0.000    0.000    0.000    0.000 HTM_Balancer.py:862(adaptSegments)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:924(updateInput)
        1    0.071    0.071    0.491    0.491 HTM_Balancer.py:973(Overlap)
     2580    0.003    0.000    0.147    0.000 fromnumeric.py:1281(ravel)
       60    0.000    0.000    0.001    0.000 fromnumeric.py:683(sort)
      720    0.017    0.000    0.022    0.000 function_base.py:3112(delete)
      720    0.023    0.000    0.030    0.000 function_base.py:3305(insert)
     2580    0.008    0.000    0.170    0.000 function_base.py:3503(append)
      720    0.001    0.000    0.001    0.000 numeric.py:1299(rollaxis)
     4740    0.004    0.000    0.146    0.000 numeric.py:392(asarray)
     2640    0.003    0.000    0.006    0.000 numeric.py:462(asanyarray)
        1    0.000    0.000    0.000    0.000 sdrFunctions.py:4(joinInputArrays)
     2160    0.001    0.000    0.001    0.000 {isinstance}
  1756396    0.246    0.000    0.246    0.000 {len}
     1188    0.000    0.000    0.000    0.000 {max}
    80524    0.014    0.000    0.014    0.000 {method 'append' of 'list' objects}
       60    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
       60    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'flatten' of 'numpy.ndarray' objects}
     1440    0.000    0.000    0.000    0.000 {method 'item' of 'numpy.ndarray' objects}
     2580    0.003    0.000    0.003    0.000 {method 'ravel' of 'numpy.ndarray' objects}
       60    0.000    0.000    0.000    0.000 {method 'remove' of 'list' objects}
        1    0.001    0.001    0.001    0.001 {method 'sort' of 'list' objects}
       60    0.000    0.000    0.000    0.000 {method 'sort' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'tolist' of 'numpy.ndarray' objects}
     1782    0.000    0.000    0.000    0.000 {min}
    13743    0.219    0.000    0.219    0.000 {numpy.core.multiarray.array}
     2580    0.010    0.000    0.010    0.000 {numpy.core.multiarray.concatenate}
     1440    0.002    0.000    0.002    0.000 {numpy.core.multiarray.empty}
  1707457    0.854    0.000    0.854    0.000 {range}

[calumroy]

The theano calculator classes uses matricies to store synapse information. The matrix then then operated on. This is an efficient way and it allows the code to be run on a GPU (not tested yet) which should greatly reduce runtimes.