Issue #8 Add ScopeLink benchmarks

[vsbogd]

Issue https://github.com/opencog/benchmark/issues/8

I have added benchmarks for creating and adding ScopeLink into atomspace. I have added same EvaluationLink tests to compare performance.

(0) Benchmark results (atomspace and cogutil debug versions) see release results below in comment https://github.com/opencog/benchmark/pull/13#issuecomment-393573327:

$ ./atomspace/atomspace/benchmark
2018-05-30 18:30:47
Running ./atomspace/atomspace/benchmark
Run on (12 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 256K (x6)
  L3 Unified 12288K (x1)
-----------------------------------------------------------------------------
Benchmark                                      Time           CPU Iterations
-----------------------------------------------------------------------------
BM_CreateScopeLinkWithVariableList          5127 ns       5127 ns     132751
BM_CreateScopeLinkWithoutVariableList       3669 ns       3669 ns     192123
BM_CreateScopeLinkWithLambdaLink            2925 ns       2925 ns     232632
BM_AddSameScopeLink                        12310 ns      12310 ns      56377
BM_AddScopeLink/65536                      29296 ns      29296 ns      23745
BM_CreateEvaluationLink                     1405 ns       1405 ns     487478
BM_AddSameEvaluationLink                    4865 ns       4865 ns     145954
BM_AddEvaluationLink/65536                 10993 ns      10993 ns      62096

(1) I have used Google Benchmark to create this stuff but it can be discussed. Pros:

• benchmark code and benchmark tooling are separated - we can separate benchmark code and tools in atomspace_bm as well but it may be simpler to use well known already implemented tool
• library is actively maintained and has some community around it

Contras:

• new format of results representation - it requires diary.txt results transition and I am not sure it is handful for others
• Google Benchmark is not present in Ubuntu repository so it should be installed by hand but installation process is not that hard and have no other dependencies

(2) I have found comparing BM_AddSameScopeLink and BM_AddSameEvaluationLink profiles that main additional complexity for ScopeLink adding is copying ScopeLink. It leads to second ScopeLink.init() method call which is heavy.

BM_AddSameScopeLink profile:

      - 83,87% _ZL19BM_AddSameScopeLinkRN9benchmark5StateE                                           ▒
         - 83,80% _ZN7opencog9AtomSpace8add_atomERKNS_6HandleEb                                      ▒
            - 78,35% _ZN7opencog9AtomTable3addESt10shared_ptrINS_4AtomEEb                            ▒
               - 46,39% _ZN7opencog10createLinkIJRSt6vectorINS_6HandleESaIS2_EERtEEES2_DpOT_         ▒
                  - 36,55% _ZNK7opencog11ClassServer7factoryERKNS_6HandleE                           ▒
                     - 36,23% _ZN7opencog9ScopeLink7factoryERKNS_6HandleE                            ▒
                        - 33,79% _ZSt11make_sharedIN7opencog9ScopeLinkEJRKSt6vectorINS0_6HandleESaIS3▒
                           - 33,60% _ZSt15allocate_sharedIN7opencog9ScopeLinkESaIS1_EJRKSt6vectorINS0▒
                              - 33,43% _ZNSt10shared_ptrIN7opencog9ScopeLinkEEC2ISaIS1_EJRKSt6vectorI▒
                                 - 33,36% _ZNSt12__shared_ptrIN7opencog9ScopeLinkELN9__gnu_cxx12_Lock▒
                                    - 23,29% _ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC2IN▒
                                       - 22,56% _ZNSt23_Sp_counted_ptr_inplaceIN7opencog9ScopeLinkESa▒
                                          - 22,30% _ZNSt16allocator_traitsISaIN7opencog9ScopeLinkEEE9▒
                                             - 22,19% _ZN9__gnu_cxx13new_allocatorIN7opencog9ScopeLin▒
                                                - 22,07% _ZN7opencog9ScopeLinkC2ERKSt6vectorINS_6Hand▒
                                                   + 18,55% _ZN7opencog9ScopeLink4initEv             ▒
                                                   + 1,41% _ZN7opencog4LinkC2ERKSt6vectorINS_6HandleE▒
                                                   + 1,39% _ZN7opencog9VariablesC2Ev (inlined)       ▒
                                                     0,52% _ZN7opencog9ScopeLink9skip_initEt         ▒

BM_AddSameEvaluationLink profile:

   - 88,48% _ZL24BM_AddSameEvaluationLinkRN9benchmark5StateE                                         ▒
      - 88,44% _ZN7opencog9AtomSpace8add_atomERKNS_6HandleEb                                         ▒
         - 82,46% _ZN7opencog9AtomTable3addESt10shared_ptrINS_4AtomEEb                               ▒
            - 32,87% _ZN7opencog10createLinkIJRSt6vectorINS_6HandleESaIS2_EERtEEES2_DpOT_            ▒
               - 17,07% _ZNK7opencog11ClassServer7factoryERKNS_6HandleE                              ▒
                  - 16,54% _ZN7opencog8FreeLink7factoryERKNS_6HandleE                                ▒
                     - 12,09% _ZSt11make_sharedIN7opencog8FreeLinkEJRKSt6vectorINS0_6HandleESaIS3_EEt▒
                        - 11,88% _ZSt15allocate_sharedIN7opencog8FreeLinkESaIS1_EJRKSt6vectorINS0_6Ha▒
                           - 11,75% _ZNSt10shared_ptrIN7opencog8FreeLinkEEC2ISaIS1_EJRKSt6vectorINS0_▒
                              - 11,56% _ZNSt12__shared_ptrIN7opencog8FreeLinkELN9__gnu_cxx12_Lock_pol▒
                                 + 5,81% _ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC2IN7ope▒
                                 + 2,69% _ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC2IN7ope▒
                                 + 1,93% _ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC2IN7ope▒
                                   0,53% _ZNSt12__shared_ptrIN7opencog8FreeLinkELN9__gnu_cxx12_Lock_p

[linas]

The code looks reasonable to me. I'll let Nil look at it and merge. I made comments, but those don't need to be fixed before merging.

Nil might want to review the complexity of the ScopeLink -- it seems to be over-simplified, and not very much like a "typical" scope link.

[linas]

re diary.txt, for now, leave that alone, and/or start a new one. Some time late, in a few weeks/months, I'll rename diary.txt to something else, update it, make comments about old vs new.

[linas]

Anyway, I like it. Results on scale-free graphs might be the most useful, the most real-life-like

[vsbogd]

I have found that numbers above are not correct as I used debug versions of libraries by accident. Correct numbers are below and they are ten times better:

$ ./atomspace/atomspace/benchmark 
2018-05-31 18:18:44
Running ./atomspace/atomspace/benchmark
Run on (12 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 256K (x6)
  L3 Unified 12288K (x1)
-----------------------------------------------------------------------------
Benchmark                                      Time           CPU Iterations
-----------------------------------------------------------------------------
BM_CreateScopeLinkWithVariableList           583 ns        583 ns    1207156
BM_CreateScopeLinkWithoutVariableList        441 ns        441 ns    1601252
BM_CreateScopeLinkWithLambdaLink             409 ns        409 ns    1713858
BM_AddSameScopeLink                         1160 ns       1160 ns     601468
BM_AddScopeLink/4096                        1316 ns       1316 ns     505942
BM_AddScopeLink/8192                        1614 ns       1614 ns     423016
BM_AddScopeLink/16384                       1691 ns       1691 ns     381658
BM_CreateEvaluationLink                      292 ns        292 ns    2386244
BM_AddSameEvaluationLink                     793 ns        793 ns     896268
BM_AddEvaluationLink/16384                  1014 ns       1014 ns     670038
BM_AddEvaluationLink/32768                  1070 ns       1070 ns     585950
BM_AddEvaluationLink/65536                  1185 ns       1185 ns     437785

[ngeiswei]

@vsbogd Just curious, what was compiled in debug mode? google benchmark? atomspace? cogutil? opencog benchmark? All?

[vsbogd]

atomspace and cogutil - I have two versions built: debug and release and debug was loaded because of LD_LIBRARY_PRELOAD override.

[vsbogd]

Nevertheless profile analysis gives the same results: ScopeLink adding:

-   86,57%     0,57%  benchmark  benchmark            [.] _ZL19BM_AddSameScopeLinkRN9benchmark5StateE▒
   - 86,01% _ZL19BM_AddSameScopeLinkRN9benchmark5StateE                                              ◆
      - 85,99% _ZN7opencog9AtomSpace8add_atomERKNS_6HandleEb                                         ▒
         - 78,02% _ZN7opencog9AtomTable3addESt10shared_ptrINS_4AtomEEb                               ▒
            - 32,35% _ZN7opencog10createLinkIJRSt6vectorINS_6HandleESaIS2_EERtEEES2_DpOT_            ▒
               - 23,10% _ZNK7opencog11ClassServer7factoryERKNS_6HandleE                              ▒
                  - 23,01% _ZN7opencog9ScopeLink7factoryERKNS_6HandleE                               ▒
                     - 22,84% ?? (inlined)                                                           ▒
                        - 21,90% ?? (inlined)                                                        ▒
                           - 21,49% ?? (inlined)                                                     ▒
                              - 21,47% ?? (inlined)                                                  ▒
                                 - 21,01% ?? (inlined)                                               ▒
                                    - ?? (inlined)                                                   ▒
                                       - 20,05% ?? (inlined)                                         ▒
                                          - 19,62% _ZN7opencog9ScopeLink7factoryERKNS_6HandleE       ▒
                                             - 19,27% _ZN7opencog9ScopeLinkC2ERKSt6vectorINS_6HandleE▒
                                                + 16,79% _ZN7opencog9ScopeLink17extract_variablesERKS▒

EvaluationLink adding:

-   89,26%     1,08%  benchmark  benchmark            [.] _ZL24BM_AddSameEvaluationLinkRN9benchmark5S▒
   - 88,18% _ZL24BM_AddSameEvaluationLinkRN9benchmark5StateE                                         ◆
      - 88,15% _ZN7opencog9AtomSpace8add_atomERKNS_6HandleEb                                         ▒
         - 80,14% _ZN7opencog9AtomTable3addESt10shared_ptrINS_4AtomEEb                               ▒
            + 32,21% _ZNK7opencog9AtomTable9getHandleERKSt10shared_ptrINS_4AtomEE                    ▒
            - 20,35% _ZN7opencog10createLinkIJRSt6vectorINS_6HandleESaIS2_EERtEEES2_DpOT_            ▒
               - 11,93% ?? (inlined)                                                                 ▒
                  - 11,01% ?? (inlined)                                                              ▒
                     - ?? (inlined)                                                                  ▒
                        - 7,20% ?? (inlined)                                                         ▒
                           - 5,56% ?? (inlined)                                                      ▒
                              - 4,37% ?? (inlined)                                                   ▒
                                 - 4,30% ?? (inlined)                                                ▒
                                    - 2,79% ?? (inlined)                                             ▒
                                       - 2,10% _ZN7opencog10createLinkIJRSt6vectorINS_6HandleESaIS2_E▒
                                          + 2,02% _ZN7opencog4Link4initERKSt6vectorINS_6HandleESaIS2_

[linas]

It looks more or less as I would expect.