Measure Lookup Performance

[MartinLoeper]

Measure the difference between forward and backward lookups. Explain which to use if both are applicable (see example in KAMP-DSL-Installer).

[MartinLoeper]

The following results are micro seconds for 100k executions of the LookupUtil.lookupMarkedObjectsWithLookupMethod method with forward and backward lookups.

FORWARD: 146172 - 150000 BACKWARD: 177000 - 200000

We see that forward lookups are generally 17% faster on average. It is also obvious that the statistical dispersion is larger for backward lookups.

I think that it is advisable to favor forward lookups over backward lookups whereever possible.

Here are the rules which I tested:

// we refer to the palladio component model (version 5.2)
import "http://palladiosimulator.org/PalladioComponentModel/5.2" as pcm

// we need these imports if we write code that targets the kamp submodules bp+is
import-package edu.kit.ipd.sdq.kamp4bp.core
import-package edu.kit.ipd.sdq.kamp4is.core
import-package edu.kit.ipd.sdq.kamp4is.model.modificationmarks
import-package edu.kit.ipd.sdq.kamp4bp.model.modificationmarks 

// the rules which are converted to source code into the gen.rule folder
rule test: pcm::OperationSignature -> interface__OperationSignature;                        
    insert edu.kit.ipd.sdq.kamp4is.model.modificationmarks.ISModificationmarksFactory#createISModifyInterface into edu.kit.ipd.sdq.kamp4is.model.modificationmarks.ISChangePropagationDueToDataDependencies#getInterfaceModifications;

rule testBackward: pcm::OperationSignature <- pcm::OperationInterface[signatures__OperationInterface];                        
    insert edu.kit.ipd.sdq.kamp4is.model.modificationmarks.ISModificationmarksFactory#createISModifyInterface into edu.kit.ipd.sdq.kamp4is.model.modificationmarks.ISChangePropagationDueToDataDependencies#getInterfaceModifications;

@Override
public void onRegistryReady() {
    // if you want to disable a generated rule, you can override it with a special type of that rule and a no-op apply method as follows:

    override(new TestRule() {
        @Override
        public void apply(AbstractArchitectureVersion version, ChangePropagationStepRegistry registry,
                AbstractChangePropagationAnalysis changePropagationAnalysis) {
            long startTime = System.nanoTime();
            for(int i = 0; i < 100000; i++) {
                LookupUtil.lookupMarkedObjectsWithLookupMethod(version, OperationSignature.class, org.palladiosimulator.pcm.repository.OperationInterface.class, TestRule::lookupOperationInterfacefromOperationSignature);
            }
            long estimatedTime = System.nanoTime() - startTime;
            //JOptionPane.showMessageDialog(null, "Elapsed time: " + (estimatedTime / 1000) + " micro seconds");
            System.out.println((estimatedTime / 1000));
        }
    });

    override(new TestBackwardRule() {
        @Override
        public void apply(AbstractArchitectureVersion version, ChangePropagationStepRegistry registry,
                AbstractChangePropagationAnalysis changePropagationAnalysis) {
            long startTime = System.nanoTime();
            for(int i = 0; i < 100000; i++) {
                LookupUtil.lookupMarkedObjectsWithLookupMethod(version, OperationSignature.class, org.palladiosimulator.pcm.repository.OperationInterface.class, TestBackwardRule::lookupOperationInterfacefromOperationSignature);
            }
            long estimatedTime = System.nanoTime() - startTime;
            //JOptionPane.showMessageDialog(null, "Elapsed time: " + (estimatedTime / 1000) + " micro seconds");
            System.out.println((estimatedTime / 1000));
        }
    });
}

[MartinLoeper]

Please take into account that this measurement was applied to the CoCoME project which is relatively large. Nevertheless the lookup performance might depend on the number of EObjects inside the tree.

We have to figure out which type of lookup is more sensitive regarding the number of entities in the resource set.