Rework the optimizing model pages

[lesquoyb]

Is your request related to a problem? Please describe.

I think the pages about optimizing models are a bit incomplete and now that we have better tools to assess the efficiency of a model we could add more information. The things I thought about so far:

• check that everything said on the pages is still working, obviously
• Add a section about the benchmark statement to explain to people how they can benchmark their models themselves
• talk about the few optimization parameters of gama
• new concatenate function for optimizing building big strings
• difference in performance between having a reflex in the agent or to iterate over them and call an action ?
• difference in performance of the multiple ways to interact with a container for basic scenarios
• talk about how to set the ram usage in the ini and/or in the headless command, also the heap workaround (see this issue)
• mention that headless mode is way more efficient than gui and general knowledge about what impacts execution time (losing focus on the window/doing something else in parallel, can't think of other things rn but there certainly are more)

Additional context If you think about anything else related to performances (options/operators/tricks that may have an impact, interesting scenarios to compare etc.) please tell me about it and I'll try to add it too

[lesquoyb]

I'm going to post the experiments I do as well as the results here so you can all see it and comment if I forgot something obvious. So the first one is Actions vs reflexes, I compared having an ask loop that calls actions into an agent vs reflexes into the agent calling the same action vs reflexes having the same code as the action:

model actionsvsreflexes

global {
        int size <- 100;
}

species a {

    action do_random_things {
        let m <- matrix_with({size,size}, rnd(0,255));
        m <- shuffle(m);
        loop i from:0 to:size-1 {
            loop j from:0 to:size-1 {
                m[i,j] <- rnd(255);
            }
        }
    }
}

species b parent:a {
    reflex call_do_random_things{
        do do_random_things;
    }
}

species c {
    reflex do_random_things_rewritten_in_a_reflex{
        let m <- matrix_with({size,size}, rnd(0,255));
        m <- shuffle(m);
        loop i from:0 to:size-1 {
            loop j from:0 to:size-1 {
                m[i,j] <- rnd(255);
            }
        }
    }
}

experiment e {

    int nb_agents <- 1000;

    init {
        ask simulation{
            create a number:myself.nb_agents;           
        }
        create simulation{
            create b number:myself.nb_agents;
        }

        create simulation{
            create c number:myself.nb_agents;
        }
    }   

    action _step_ {
        ask simulations{
            if length(a) >0 {
                benchmark "Manually call actions" repeat:10 {
                    ask a {
                        do do_random_things;
                    }
                }               
            }
            else if length(b) >  0 {
                benchmark "Reflexes call actions" repeat:10 {
                        do _step_;  
                }       
            }
            else {
                benchmark "Reflexes do the action" repeat:10 {
                        do _step_;  
                }               
            }
        }
    }
}

here are the results:

Manually call actions (over 10 iteration(s)): min = 4089.0007 ms (iteration #0) | max = 4134.1196 ms (iteration #9) | average = 4108.250980000001ms
Reflexes call actions (over 10 iteration(s)): min = 4096.4389 ms (iteration #7) | max = 4125.0502 ms (iteration #1) | average = 4106.92661ms
Reflexes do the action (over 10 iteration(s)): min = 4138.6298 ms (iteration #9) | max = 4182.6288 ms (iteration #6) | average = 4151.716270000001ms

Not much differences between the three methods. I already did other experiments and noticed that the operators on list could be more efficient than basic loops or ask, so that would be interesting to test a "foreach" kind of operator to see if it is also the case here

[lesquoyb]

Here is a model to test the best way to build a list out of properties of agents: I tested loop over vs loop from/to vs loop times vs ask vs collect With this model:

model accessinglistitems

global{
    int nb_agents <- 1000000;   
}

species b{

    int v;

    init {
        v <- rnd(0, 10);
    }
}

experiment e {

    reflex fill_list_from_agents {

        ask b{
            do die;
        }
        create b number:nb_agents;

        list<int> l1 <- [];
        benchmark "fill list loop over" repeat:10{
            loop obj over:b{
                l1 <+ obj.v;
            }
        }

        list<int> l2 <- [];
        benchmark "fill list loop from to" repeat:10{
            int to <- length(b)-1;
            loop i from:0 to:to{
                l2 <+ b[i].v;
            }
        }

        list<int> l3 <- [];
        benchmark "fill list loop times" repeat:10{
            int to <- length(b)-1;
            int i <- 0;
            loop times:to{
                l3 <+ b[i].v;
                i <- i + 1;
            }
        }

        list<int> l4 <- [];
        benchmark "fill list ask"  repeat:10{
            ask b{
                l4 <+ v;
            }
        }

        list<int> l5 <- [];
        benchmark "fill list collect" repeat:10{
            l5 <- b collect (each.v);
        }
    }
}

Which gave me those results

fill list loop over (over 10 iteration(s)): min = 246.7017 ms (iteration #4) | max = 524.0814 ms (iteration #0) | average = 288.12743ms
fill list loop from to (over 10 iteration(s)): min = 469.3644 ms (iteration #9) | max = 555.4155 ms (iteration #2) | average = 486.78100000000006ms
fill list loop times (over 10 iteration(s)): min = 634.147 ms (iteration #1) | max = 658.3801 ms (iteration #0) | average = 647.32994ms
fill list ask (over 10 iteration(s)): min = 227.0711 ms (iteration #4) | max = 297.658 ms (iteration #2) | average = 246.10171ms
fill list collect (over 10 iteration(s)): min = 97.022 ms (iteration #8) | max = 102.0031 ms (iteration #0) | average = 98.54660000000003ms

We can see that ask and loop over have basically the same execution time with a slight advantage for ask, but weirdly enough loop from to is far behind, it was expected that it performed worse but I wouldn't think it was that much. The loop times is even worse which once again was expected but not at that level either. Finally the best way was also the expected one: the operator collect as we are spared the list concatenation in gaml.

What this teaches us is that any operation (even a simple i <- i + 1;) that is done in gaml instead of in java can have a really big impact on performances.

[lesquoyb]

The same experiment but with a simple list of int instead of a list of agents raises similar results:

model accessinglistitems

global{
    int nb_agents <- 1000000;   
}

species b{

    int v;

    init {
        v <- rnd(0, 10);
    }
}

experiment e {

    reflex fill_list_from_other_list{

        list<int> base <- [];
        loop times:nb_agents{
            base <+ rnd(0,10);
        }
        list<int> l1 <- [];
        benchmark "fill list loop over" repeat:10{
            loop obj over:base{
                l1 <+ obj;
            }
        }

        list<int> l2 <- [];
        benchmark "fill list loop from to" repeat:10{
            int to <- length(base)-1;
            loop i from:0 to:to{
                l2 <+ base[i];
            }
        }

        list<int> l3 <- [];
        benchmark "fill list loop times" repeat:10{
            int to <- length(base)-1;
            int i <- 0;
            loop times:to{
                l3 <+ base[i];
                i <- i + 1;
            }
        }

        list<int> l5 <- [];
        benchmark "fill list collect" repeat:10{
            l5 <- base collect (each);
        }

    }

}

fill list loop over (over 10 iteration(s)): min = 137.9432 ms (iteration #8) | max = 230.8834 ms (iteration #1) | average = 159.16049000000004ms
fill list loop from to (over 10 iteration(s)): min = 356.9075 ms (iteration #3) | max = 631.9033 ms (iteration #5) | average = 417.74827000000005ms
fill list loop times (over 10 iteration(s)): min = 498.8026 ms (iteration #9) | max = 649.7151 ms (iteration #0) | average = 535.07749ms
fill list collect (over 10 iteration(s)): min = 17.6604 ms (iteration #7) | max = 20.3564 ms (iteration #4) | average = 18.880190000000002ms

Once again collect is impressively faster than the rest, then comes loop over, then way behind loop from to and even slower loop times.

As this test has been done with the same parameters and on the same computer as the previous one we can also conclude that gathering data is way way way faster once it's stored inside a list than extracting it from the list of agents (collect here being more than 5 times faster to compute than on the list of agents)