How to interpret the output results?

[SheperoMah]

I read the documentation, the books and some of the courses. But still I don't really get how to make use of the output files. I am interested in the plan with highest avg. score. Where can I find the result for this plan?

I interpret this plan to be the optimal simulation result (as regards the co-evolution algorithm), as regards the traffic flow in the city, car usage, travel times etc. Am I right?

In the output's main folder, the _outputplans.xml.gz contains the plans of the last iteration, most of the time the plan with highest score for the agent is not activated. In the file scorestats.txt I can see that the iteration 46 had the highest avg. score. But in the folder of this iteration, I cannot see the 46.plans.xml.gz. I presume that this is because of the option that specifies that only save the plans every 10th iteration. But the question is how can I save the plan with highest score, without saving all of them?

Another question, I believe that if I use the _outputplans.xml.gz to get the plan with highest score for each agent, this will not give me the plan with avg. highest score! As I understand it, agents' individual highest scores might not coincide with avg. highest score. Am I right here?

[Janekdererste]

The user guide might be a good starting point for you.

The output plans file contains the agents and all the plans they had memorized in the last iteration. The plans marked as "selected" (each agent has one selected plan) is the one which was executed in the last iteration.

There is no such thing as "plan with highest avg. score", since each plan has an individual score. The selected plan of a particular agent usually is the one which received the highest score within the set of plans this agent has tried. There are exceptions to this rule.

In the file scorestats.txt I can see that the iteration 46 had the highest avg. score.

You should work with the data from the last iteration. And again, I think "hightest avg. score" is not a usefull metric when analyzing MATSim.

[SheperoMah]

Thank you for your swift reply. I have already checked the user guide, I will check it further.

What you describe is not the case for me. I would expect that the last iteration will have the best score for each agent, for me it doesn't.

I have two agents for testing, the first one has the highest score plan selected in the last iteration, iteration nr. 100. Whereas, the second agent has a selected plan with score 136. All other plans for this agent have higher scores (260+), none of them is selected.

There are exceptions to this rule.

Where can I read about these exceptions?

There is no such thing as "plan with highest avg. score", since each plan has an individual score.

What I mean is that the plan with highest score for each agent might result in lower total score. If all agents chose the route with shortest distance (highest individual score), in the equil example for instance, they might cause traffic jam in the shortest link, which results in lower total score. As far as I see it, the optimal of the subproblem is not the optimal of the whole problem.

[Janekdererste]

What you describe is not the case for me. I would expect that the last iteration will have the best score for each agent, for me it doesn't.

That's indeed what should happen.

Where can I read about these exceptions?

This was probably phrased sloppily. The default strategy for plan selection contains a logit model. This allows agents to select a different plan than the absolute best with a certain probability. In your case, the plan with a score of 260 compared to 136 should be selected though.

If you could provided the config and the logfile of your run, I will have a look at them.

[SheperoMah]

I followed section 4.6 in the user guide when adjusting the replanning strategy. caseStudy.zip

[Janekdererste]

It might be the case that you never turn of your innovative strategies. This way agents keep producing new plans instead of re-trying and re-evaluating older ones. Most of your agents should still do that since you have the largest weight on SelectExpBeta. I don't know how the weighting works with a sample size of two agents though. What you can do to prevent agents inventing new plans until the end of the simulation is to include

<module name="strategy">
    <param name="fractionOfIterationsToDisableInnovation" value="0.9"/>
    .... other strategy stuff
</module>

This will turn off your innovative strategies (in your case re-planning and time shifting) after 90% of your iterations have finished.