search

calumroy / HTM

HTM
3 stars 0 forks source link

HTM Speed #5

Open calumroy opened 10 years ago

calumroy commented 10 years ago

Speed

The HTM code needs to be optimized and refactored so it can run faster. Previous work has not focused on making the algorithms faster and as such the current implementation is very slow even for the small HTM layers being tested. Before the code is refactored to increase its speed profiling on the current code needs to be performed. Unit tests also need to be written to ensure the new code runs the same as the old code (this should have been done a long time ago but i'm slack and under resourced for this project :( ).

The algorithm could probably be optimized firstly by refactoring the structure of synapses and other objects in the code. After this a multithreading parralization of the algorithm could probably give vast improments in the size and speed of the HTM.

calumroy commented 10 years ago

I used cProfile module see https://docs.python.org/2/library/profile.html or http://www.huyng.com/posts/python-performance-analysis/

This gave the following results after 19 steps of the HTM.

 1052499 function calls in 2.039 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.002    0.001    0.021    0.010 HTM_Balancer.py:1033(updatePredictiveState)
        2    0.006    0.003    0.158    0.079 HTM_Balancer.py:1090(temporalLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1168(updateRegionInput)
        1    0.000    0.000    2.039    2.039 HTM_Balancer.py:1200(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1257(updateHTMInput)
        1    0.000    0.000    2.039    2.039 HTM_Balancer.py:1272(spatialTemporal)
      416    0.116    0.000    0.706    0.002 HTM_Balancer.py:150(updateConnectedSynapses)
      416    0.000    0.000    0.000    0.000 HTM_Balancer.py:160(updateBoost)
      300    0.001    0.000    0.007    0.000 HTM_Balancer.py:225(columnActiveNotBursting)
        2    0.002    0.001    0.002    0.001 HTM_Balancer.py:280(updateOutput)
      763    0.202    0.000    0.934    0.001 HTM_Balancer.py:330(neighbours)
      300    0.002    0.000    0.006    0.000 HTM_Balancer.py:351(updateOverlapDutyCycle)
       26    0.001    0.000    0.002    0.000 HTM_Balancer.py:365(increasePermanence)
      300    0.010    0.000    0.017    0.000 HTM_Balancer.py:376(kthScore)
      136    0.000    0.000    0.000    0.000 HTM_Balancer.py:38(__init__)
        2    0.001    0.000    0.005    0.003 HTM_Balancer.py:388(averageReceptiveFeildSize)
      416    0.002    0.000    0.005    0.000 HTM_Balancer.py:398(updateActiveDutyCycle)
      416    0.005    0.000    0.005    0.000 HTM_Balancer.py:415(maxDutyCycle)
     1248    0.009    0.000    0.122    0.000 HTM_Balancer.py:422(deleteEmptySegments)
        9    0.000    0.000    0.001    0.000 HTM_Balancer.py:434(deleteWeakSynapses)
       16    0.000    0.000    0.001    0.000 HTM_Balancer.py:447(columnActiveAdd)
       48    0.000    0.000    0.003    0.000 HTM_Balancer.py:456(activeStateAdd)
       36    0.000    0.000    0.003    0.000 HTM_Balancer.py:465(predictiveStateAdd)
       16    0.000    0.000    0.001    0.000 HTM_Balancer.py:474(learnStateAdd)
     6431    0.025    0.000    0.029    0.000 HTM_Balancer.py:483(columnActiveState)
     3603    0.017    0.000    0.020    0.000 HTM_Balancer.py:491(activeState)
     1312    0.011    0.000    0.012    0.000 HTM_Balancer.py:499(predictiveState)
    11248    0.059    0.000    0.066    0.000 HTM_Balancer.py:507(learnState)
       16    0.010    0.001    0.068    0.004 HTM_Balancer.py:515(randomActiveSynapses)
       19    0.000    0.000    0.000    0.000 HTM_Balancer.py:53(__init__)
       20    0.000    0.000    0.001    0.000 HTM_Balancer.py:569(segmentHighestScore)
      158    0.003    0.000    0.010    0.000 HTM_Balancer.py:584(segmentActive)
      290    0.005    0.000    0.017    0.000 HTM_Balancer.py:623(segmentNumSynapsesActive)
       96    0.000    0.000    0.017    0.000 HTM_Balancer.py:642(getBestMatchingSegment)
       16    0.000    0.000    0.009    0.001 HTM_Balancer.py:674(getBestMatchingCell)
       52    0.002    0.000    0.075    0.001 HTM_Balancer.py:717(getSegmentActiveSynapses)
       25    0.001    0.000    0.003    0.000 HTM_Balancer.py:767(adaptSegments)
        2    0.000    0.000    0.000    0.000 HTM_Balancer.py:833(updateInput)
        2    0.045    0.022    0.764    0.382 HTM_Balancer.py:836(Overlap)
        2    0.005    0.003    0.498    0.249 HTM_Balancer.py:866(inhibition)
        2    0.008    0.004    0.505    0.253 HTM_Balancer.py:899(learning)
        2    0.000    0.000    0.092    0.046 HTM_Balancer.py:927(updateActiveState)
     2514    0.004    0.000    0.024    0.000 _methods.py:35(_all)
        2    0.000    0.000    0.000    0.000 _methods.py:39(_count_reduce_items)
        2    0.000    0.000    0.000    0.000 _methods.py:49(_mean)
   104774    0.094    0.000    0.665    0.000 fromnumeric.py:1281(ravel)
      300    0.001    0.000    0.003    0.000 fromnumeric.py:683(sort)
     1373    0.060    0.000    0.115    0.000 function_base.py:3112(delete)
      116    0.003    0.000    0.004    0.000 function_base.py:3305(insert)
   104774    0.210    0.000    1.312    0.000 function_base.py:3503(append)
        2    0.000    0.000    0.000    0.000 function_base.py:417(average)
      116    0.000    0.000    0.000    0.000 numeric.py:1299(rollaxis)
     1257    0.002    0.000    0.011    0.000 numeric.py:136(ones)
   107638    0.071    0.000    0.506    0.000 numeric.py:392(asarray)
   105076    0.081    0.000    0.178    0.000 numeric.py:462(asanyarray)
       16    0.000    0.000    0.000    0.000 random.py:293(sample)
        2    0.000    0.000    0.000    0.000 {hasattr}
     4119    0.003    0.000    0.003    0.000 {isinstance}
        2    0.000    0.000    0.000    0.000 {issubclass}
   113711    0.017    0.000    0.017    0.000 {len}
       16    0.000    0.000    0.000    0.000 {math.ceil}
       16    0.000    0.000    0.000    0.000 {math.log}
       10    0.000    0.000    0.000    0.000 {math.pow}
        2    0.000    0.000    0.000    0.000 {math.sqrt}
     1617    0.000    0.000    0.000    0.000 {max}
     2514    0.003    0.000    0.027    0.000 {method 'all' of 'numpy.ndarray' objects}
       70    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
     1251    0.004    0.000    0.004    0.000 {method 'astype' of 'numpy.ndarray' objects}
      300    0.001    0.000    0.001    0.000 {method 'copy' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
      232    0.000    0.000    0.000    0.000 {method 'item' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'mean' of 'numpy.ndarray' objects}
      136    0.000    0.000    0.000    0.000 {method 'random' of '_random.Random' objects}
   104774    0.073    0.000    0.073    0.000 {method 'ravel' of 'numpy.ndarray' objects}
     2516    0.020    0.000    0.020    0.000 {method 'reduce' of 'numpy.ufunc' objects}
      300    0.001    0.000    0.001    0.000 {method 'sort' of 'numpy.ndarray' objects}
     2095    0.000    0.000    0.000    0.000 {min}
   214584    0.543    0.000    0.543    0.000 {numpy.core.multiarray.array}
     1257    0.001    0.000    0.001    0.000 {numpy.core.multiarray.can_cast}
   104774    0.260    0.000    0.260    0.000 {numpy.core.multiarray.concatenate}
     1257    0.005    0.000    0.005    0.000 {numpy.core.multiarray.copyto}
     1489    0.004    0.000    0.004    0.000 {numpy.core.multiarray.empty}
    39322    0.024    0.000    0.024    0.000 {range}
calumroy commented 10 years ago

Apparently appending to numpy arrays is slow see http://stackoverflow.com/questions/7133885/fastest-way-to-grow-a-numpy-numeric-array . Use the standard lists append instead.

calumroy commented 10 years ago

After deleting numpy arrays for the synapse creation and updating functions only (see commit ac027e92dd4ba56bc8fd4dd789c0cd064a0e37fb).

Below was the resulting profile for step 19 in the HTM.

1021158 function calls in 1.878 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.001    0.001    0.014    0.007 HTM_Balancer.py:1027(updatePredictiveState)
        2    0.003    0.001    0.023    0.011 HTM_Balancer.py:1084(temporalLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1162(updateRegionInput)
        1    0.000    0.000    1.878    1.878 HTM_Balancer.py:1194(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1251(updateHTMInput)
        1    0.000    0.000    1.878    1.878 HTM_Balancer.py:1266(spatialTemporal)
      416    0.115    0.000    0.681    0.002 HTM_Balancer.py:148(updateConnectedSynapses)
      416    0.000    0.000    0.000    0.000 HTM_Balancer.py:158(updateBoost)
      260    0.001    0.000    0.006    0.000 HTM_Balancer.py:223(columnActiveNotBursting)
        2    0.001    0.000    0.001    0.001 HTM_Balancer.py:278(updateOutput)
      779    0.219    0.000    0.970    0.001 HTM_Balancer.py:328(neighbours)
      260    0.002    0.000    0.005    0.000 HTM_Balancer.py:349(updateOverlapDutyCycle)
       26    0.001    0.000    0.001    0.000 HTM_Balancer.py:363(increasePermanence)
      260    0.009    0.000    0.014    0.000 HTM_Balancer.py:374(kthScore)
       36    0.000    0.000    0.000    0.000 HTM_Balancer.py:38(__init__)
        2    0.001    0.001    0.005    0.003 HTM_Balancer.py:386(averageReceptiveFeildSize)
      416    0.002    0.000    0.005    0.000 HTM_Balancer.py:396(updateActiveDutyCycle)
      416    0.005    0.000    0.005    0.000 HTM_Balancer.py:413(maxDutyCycle)
     1248    0.002    0.000    0.002    0.000 HTM_Balancer.py:420(deleteEmptySegments)
       48    0.000    0.000    0.000    0.000 HTM_Balancer.py:434(deleteWeakSynapses)
       16    0.000    0.000    0.001    0.000 HTM_Balancer.py:447(columnActiveAdd)
       28    0.000    0.000    0.002    0.000 HTM_Balancer.py:456(activeStateAdd)
        9    0.000    0.000    0.001    0.000 HTM_Balancer.py:465(predictiveStateAdd)
       16    0.000    0.000    0.001    0.000 HTM_Balancer.py:474(learnStateAdd)
    10340    0.042    0.000    0.049    0.000 HTM_Balancer.py:483(columnActiveState)
     2703    0.013    0.000    0.015    0.000 HTM_Balancer.py:491(activeState)
     1332    0.007    0.000    0.008    0.000 HTM_Balancer.py:499(predictiveState)
     5008    0.026    0.000    0.029    0.000 HTM_Balancer.py:507(learnState)
        6    0.004    0.001    0.026    0.004 HTM_Balancer.py:515(randomActiveSynapses)
        6    0.000    0.000    0.000    0.000 HTM_Balancer.py:53(__init__)
       10    0.000    0.000    0.001    0.000 HTM_Balancer.py:551(getActiveSegment)
       12    0.000    0.000    0.001    0.000 HTM_Balancer.py:569(segmentHighestScore)
      189    0.004    0.000    0.012    0.000 HTM_Balancer.py:584(segmentActive)
      171    0.003    0.000    0.009    0.000 HTM_Balancer.py:623(segmentNumSynapsesActive)
       66    0.000    0.000    0.009    0.000 HTM_Balancer.py:642(getBestMatchingSegment)
        6    0.000    0.000    0.003    0.000 HTM_Balancer.py:674(getBestMatchingCell)
       15    0.000    0.000    0.026    0.002 HTM_Balancer.py:717(getSegmentActiveSynapses)
       42    0.001    0.000    0.001    0.000 HTM_Balancer.py:765(adaptSegments)
        2    0.000    0.000    0.000    0.000 HTM_Balancer.py:827(updateInput)
        2    0.047    0.024    0.740    0.370 HTM_Balancer.py:830(Overlap)
        2    0.009    0.004    0.562    0.281 HTM_Balancer.py:860(inhibition)
        2    0.008    0.004    0.500    0.250 HTM_Balancer.py:893(learning)
        2    0.000    0.000    0.039    0.020 HTM_Balancer.py:921(updateActiveState)
        2    0.000    0.000    0.000    0.000 _methods.py:39(_count_reduce_items)
        2    0.000    0.000    0.000    0.000 _methods.py:49(_mean)
   105289    0.091    0.000    0.659    0.000 fromnumeric.py:1281(ravel)
      260    0.001    0.000    0.002    0.000 fromnumeric.py:683(sort)
       69    0.001    0.000    0.002    0.000 function_base.py:3112(delete)
       69    0.002    0.000    0.003    0.000 function_base.py:3305(insert)
   105289    0.204    0.000    1.298    0.000 function_base.py:3503(append)
        2    0.000    0.000    0.000    0.000 function_base.py:417(average)
       69    0.000    0.000    0.000    0.000 numeric.py:1299(rollaxis)
   105498    0.064    0.000    0.498    0.000 numeric.py:392(asarray)
   105551    0.073    0.000    0.172    0.000 numeric.py:462(asanyarray)
        6    0.000    0.000    0.000    0.000 random.py:293(sample)
        2    0.000    0.000    0.000    0.000 {hasattr}
      213    0.000    0.000    0.000    0.000 {isinstance}
        2    0.000    0.000    0.000    0.000 {issubclass}
   111867    0.017    0.000    0.017    0.000 {len}
        6    0.000    0.000    0.000    0.000 {math.ceil}
        6    0.000    0.000    0.000    0.000 {math.log}
        2    0.000    0.000    0.000    0.000 {math.sqrt}
     2156    0.001    0.000    0.001    0.000 {max}
      111    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
      260    0.001    0.000    0.001    0.000 {method 'copy' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
       54    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
      138    0.000    0.000    0.000    0.000 {method 'item' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'mean' of 'numpy.ndarray' objects}
       36    0.000    0.000    0.000    0.000 {method 'random' of '_random.Random' objects}
   105289    0.070    0.000    0.070    0.000 {method 'ravel' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'reduce' of 'numpy.ufunc' objects}
      260    0.001    0.000    0.001    0.000 {method 'sort' of 'numpy.ndarray' objects}
     2153    0.000    0.000    0.000    0.000 {min}
   212646    0.543    0.000    0.543    0.000 {numpy.core.multiarray.array}
   105289    0.262    0.000    0.262    0.000 {numpy.core.multiarray.concatenate}
      138    0.000    0.000    0.000    0.000 {numpy.core.multiarray.empty}
    33841    0.020    0.000    0.020    0.000 {range}
calumroy commented 10 years ago

More work needs to be done in removing numpy array append calls and converting them to normal array appends to make the code faster. The spatial pooling algorithm relies heavily on numpy append function calls.

calumroy commented 10 years ago

The neighbours function was updated to use standard lists instead on numpy. This seems to have greatly reduced the number of function calls and improved the speed.

Step 19

623595 function calls in 1.196 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       763    0.115    0.000    0.250    0.000 HTM_Balancer.py:328(neighbours)
calumroy commented 10 years ago

The updatedConnectedSynapses Function was also converted to standard arrays. This also greatly reduced the number of function calls.

Step 19

302368 function calls in 0.672 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.001    0.001    0.017    0.009 HTM_Balancer.py:1029(updatePredictiveState)
        2    0.003    0.002    0.022    0.011 HTM_Balancer.py:1086(temporalLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1164(updateRegionInput)
        1    0.000    0.000    0.672    0.672 HTM_Balancer.py:1196(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1253(updateHTMInput)
        1    0.000    0.000    0.672    0.672 HTM_Balancer.py:1268(spatialTemporal)
      416    0.036    0.000    0.148    0.000 HTM_Balancer.py:148(updateConnectedSynapses)
calumroy commented 9 years ago

A nose test was created (test_runtime.py) to see what currently slows the HTM down the most. A large HTM network was setup and run, width = 80, height = 20, num cells = 10. A simple moving vertical line was the input. below is a single call to spatialTemporal update. 

  902358 function calls in 2.097 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.029    0.029 HTM_Balancer.py:1012(inhibition)
     1600    0.001    0.000    0.001    0.000 HTM_Balancer.py:1024(<lambda>)
        1    0.008    0.008    0.197    0.197 HTM_Balancer.py:1062(learning)
        1    0.000    0.000    0.742    0.742 HTM_Balancer.py:1090(updateActiveState)
        1    0.008    0.008    0.009    0.009 HTM_Balancer.py:1215(updatePredictiveState)
        1    0.031    0.031    0.259    0.259 HTM_Balancer.py:1264(sequenceLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1394(updateRegionInput)
        1    0.000    0.000    2.097    2.097 HTM_Balancer.py:1445(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1546(updateHTMInput)
     1600    0.001    0.000    0.001    0.000 HTM_Balancer.py:160(updateBoost)
        1    0.000    0.000    2.097    2.097 HTM_Balancer.py:1614(spatialTemporal)
     1600    0.016    0.000    0.107    0.000 HTM_Balancer.py:239(columnActiveNotBursting)
        1    0.012    0.012    0.013    0.013 HTM_Balancer.py:293(updateOutput)
     1600    0.118    0.000    0.558    0.000 HTM_Balancer.py:312(updateConnectedSynapses)
       98    0.000    0.000    0.000    0.000 HTM_Balancer.py:39(__init__)
     1640    0.080    0.000    0.181    0.000 HTM_Balancer.py:394(neighbours)
       40    0.000    0.000    0.001    0.000 HTM_Balancer.py:415(updateOverlapDutyCycle)
       40    0.003    0.000    0.004    0.000 HTM_Balancer.py:429(increasePermanence)
       40    0.001    0.000    0.001    0.000 HTM_Balancer.py:437(kthScore)
     1600    0.004    0.000    0.013    0.000 HTM_Balancer.py:449(updateActiveDutyCycle)
     1600    0.008    0.000    0.008    0.000 HTM_Balancer.py:466(maxDutyCycle)
    16000    0.030    0.000    0.039    0.000 HTM_Balancer.py:473(deleteSegments)
       14    0.000    0.000    0.001    0.000 HTM_Balancer.py:524(columnActiveAdd)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:53(__init__)
      140    0.001    0.000    0.011    0.000 HTM_Balancer.py:533(activeStateAdd)
       14    0.000    0.000    0.001    0.000 HTM_Balancer.py:551(learnStateAdd)
     2216    0.010    0.000    0.011    0.000 HTM_Balancer.py:560(columnActiveState)
    15958    0.081    0.000    0.090    0.000 HTM_Balancer.py:568(activeState)
    16070    0.079    0.000    0.087    0.000 HTM_Balancer.py:576(predictiveState)
   128084    0.650    0.000    0.723    0.000 HTM_Balancer.py:584(learnState)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:592(findActiveCell)
        7    0.000    0.000    0.001    0.000 HTM_Balancer.py:603(findLearnCell)
        7    0.085    0.012    0.728    0.104 HTM_Balancer.py:613(randomActiveSynapses)
      140    0.000    0.000    0.000    0.000 HTM_Balancer.py:740(getBestMatchingSegment)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:772(getBestMatchingCell)
        7    0.000    0.000    0.728    0.104 HTM_Balancer.py:814(getSegmentActiveSynapses)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:862(adaptSegments)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:924(updateInput)
        1    0.193    0.193    0.860    0.860 HTM_Balancer.py:973(Overlap)
     1668    0.002    0.000    0.392    0.000 fromnumeric.py:1281(ravel)
       40    0.000    0.000    0.000    0.000 fromnumeric.py:683(sort)
      168    0.004    0.000    0.005    0.000 function_base.py:3112(delete)
      168    0.005    0.000    0.007    0.000 function_base.py:3305(insert)
     1668    0.005    0.000    0.408    0.000 function_base.py:3503(append)
      168    0.000    0.000    0.000    0.000 numeric.py:1299(rollaxis)
     2172    0.002    0.000    0.389    0.000 numeric.py:392(asarray)
     1708    0.002    0.000    0.004    0.000 numeric.py:462(asanyarray)
        7    0.000    0.000    0.000    0.000 random.py:293(sample)
        1    0.000    0.000    0.000    0.000 sdrFunctions.py:4(joinInputArrays)
      504    0.000    0.000    0.000    0.000 {isinstance}
   262018    0.030    0.000    0.030    0.000 {len}
        7    0.000    0.000    0.000    0.000 {math.ceil}
        7    0.000    0.000    0.000    0.000 {math.log}
     1050    0.000    0.000    0.000    0.000 {max}
   217251    0.028    0.000    0.028    0.000 {method 'append' of 'list' objects}
       40    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        7    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'flatten' of 'numpy.ndarray' objects}
      336    0.000    0.000    0.000    0.000 {method 'item' of 'numpy.ndarray' objects}
       70    0.000    0.000    0.000    0.000 {method 'random' of '_random.Random' objects}
     1668    0.002    0.000    0.002    0.000 {method 'ravel' of 'numpy.ndarray' objects}
        1    0.001    0.001    0.002    0.002 {method 'sort' of 'list' objects}
       40    0.000    0.000    0.000    0.000 {method 'sort' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'tolist' of 'numpy.ndarray' objects}
     4650    0.001    0.000    0.001    0.000 {min}
     7499    0.485    0.000    0.485    0.000 {numpy.core.multiarray.array}
     1668    0.007    0.000    0.007    0.000 {numpy.core.multiarray.concatenate}
      336    0.000    0.000    0.000    0.000 {numpy.core.multiarray.empty}
   207280    0.097    0.000    0.097    0.000 {range}

As seen the methods which are slowing the HTM down the most is randomActiveSynapses which is calling learnState many times. Also neighbours is slow and updateConnectedSynapses.

calumroy commented 9 years ago

LearnState Function

How can this function be improved?

It gets called 112000 times from the function randomActiveSynapses. We could fix randomActiveSynapses instead. This function doesn't need to call learnState this many times.

calumroy commented 9 years ago

randomActiveSynapses

This function now doesn't call learnState as much. See profile below.

 594978 function calls in 1.523 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002    0.024    0.024 HTM_Balancer.py:1020(inhibition)
     1600    0.000    0.000    0.000    0.000 HTM_Balancer.py:1032(<lambda>)
        1    0.011    0.011    0.213    0.213 HTM_Balancer.py:1070(learning)
        1    0.000    0.000    0.116    0.116 HTM_Balancer.py:1098(updateActiveState)
        1    0.010    0.010    0.011    0.011 HTM_Balancer.py:1223(updatePredictiveState)
        1    0.033    0.033    0.271    0.271 HTM_Balancer.py:1272(sequenceLearning)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1402(updateRegionInput)
        1    0.000    0.000    1.523    1.523 HTM_Balancer.py:1453(spatialTemporal)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:1554(updateHTMInput)
     1600    0.001    0.000    0.001    0.000 HTM_Balancer.py:160(updateBoost)
        1    0.000    0.000    1.523    1.523 HTM_Balancer.py:1622(spatialTemporal)
     1600    0.017    0.000    0.112    0.000 HTM_Balancer.py:243(columnActiveNotBursting)
        1    0.009    0.009    0.009    0.009 HTM_Balancer.py:297(updateOutput)
     1600    0.121    0.000    0.582    0.000 HTM_Balancer.py:316(updateConnectedSynapses)
       14    0.000    0.000    0.000    0.000 HTM_Balancer.py:39(__init__)
     1640    0.084    0.000    0.192    0.000 HTM_Balancer.py:398(neighbours)
       40    0.000    0.000    0.001    0.000 HTM_Balancer.py:419(updateOverlapDutyCycle)
       40    0.003    0.000    0.004    0.000 HTM_Balancer.py:433(increasePermanence)
       40    0.000    0.000    0.001    0.000 HTM_Balancer.py:441(kthScore)
     1600    0.004    0.000    0.013    0.000 HTM_Balancer.py:453(updateActiveDutyCycle)
     1600    0.008    0.000    0.008    0.000 HTM_Balancer.py:470(maxDutyCycle)
    16000    0.031    0.000    0.041    0.000 HTM_Balancer.py:477(deleteSegments)
       14    0.000    0.000    0.001    0.000 HTM_Balancer.py:528(columnActiveAdd)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:53(__init__)
      140    0.001    0.000    0.009    0.000 HTM_Balancer.py:537(activeStateAdd)
       14    0.000    0.000    0.001    0.000 HTM_Balancer.py:555(learnStateAdd)
     2216    0.009    0.000    0.010    0.000 HTM_Balancer.py:564(columnActiveState)
    15958    0.083    0.000    0.094    0.000 HTM_Balancer.py:572(activeState)
    16070    0.082    0.000    0.092    0.000 HTM_Balancer.py:580(predictiveState)
    32084    0.167    0.000    0.186    0.000 HTM_Balancer.py:588(learnState)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:596(findActiveCell)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:607(findLearnCell)
        7    0.013    0.002    0.104    0.015 HTM_Balancer.py:617(randomActiveSynapses)
      140    0.000    0.000    0.000    0.000 HTM_Balancer.py:748(getBestMatchingSegment)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:780(getBestMatchingCell)
        7    0.000    0.000    0.104    0.015 HTM_Balancer.py:822(getSegmentActiveSynapses)
        7    0.000    0.000    0.000    0.000 HTM_Balancer.py:870(adaptSegments)
        1    0.000    0.000    0.000    0.000 HTM_Balancer.py:932(updateInput)
        1    0.193    0.193    0.888    0.888 HTM_Balancer.py:981(Overlap)
     1668    0.002    0.000    0.412    0.000 fromnumeric.py:1281(ravel)
       40    0.000    0.000    0.000    0.000 fromnumeric.py:683(sort)
      168    0.003    0.000    0.004    0.000 function_base.py:3112(delete)
      168    0.004    0.000    0.005    0.000 function_base.py:3305(insert)
     1668    0.005    0.000    0.428    0.000 function_base.py:3503(append)
      168    0.000    0.000    0.000    0.000 numeric.py:1299(rollaxis)
     2172    0.002    0.000    0.409    0.000 numeric.py:392(asarray)
     1708    0.002    0.000    0.004    0.000 numeric.py:462(asanyarray)
        7    0.000    0.000    0.000    0.000 random.py:293(sample)
        1    0.000    0.000    0.000    0.000 sdrFunctions.py:4(joinInputArrays)
      504    0.000    0.000    0.000    0.000 {isinstance}
   156412    0.020    0.000    0.020    0.000 {len}
        7    0.000    0.000    0.000    0.000 {math.ceil}
        7    0.000    0.000    0.000    0.000 {math.log}
     1050    0.000    0.000    0.000    0.000 {max}
   217167    0.029    0.000    0.029    0.000 {method 'append' of 'list' objects}
       40    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        7    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'flatten' of 'numpy.ndarray' objects}
      336    0.000    0.000    0.000    0.000 {method 'item' of 'numpy.ndarray' objects}
       70    0.000    0.000    0.000    0.000 {method 'random' of '_random.Random' objects}
     1668    0.002    0.000    0.002    0.000 {method 'ravel' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.001    0.001 {method 'sort' of 'list' objects}
       40    0.000    0.000    0.000    0.000 {method 'sort' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'tolist' of 'numpy.ndarray' objects}
     4650    0.001    0.000    0.001    0.000 {min}
     7499    0.511    0.000    0.511    0.000 {numpy.core.multiarray.array}
     1668    0.007    0.000    0.007    0.000 {numpy.core.multiarray.concatenate}
      336    0.000    0.000    0.000    0.000 {numpy.core.multiarray.empty}
   101674    0.049    0.000    0.049    0.000 {range}