search

matsim-org / matsim-code-examples

A repository containing code examples around MATSim
GNU General Public License v3.0
83 stars 179 forks source link

Performance issue for simple simulation (perhaps problem with input files) #1127

Closed Tim-Schwabe closed 5 months ago

Tim-Schwabe commented 5 months ago

Hello,

I am trying to test a simple simulation where I want to move 2 agents from A to B in a self created network (based on OSM data) without knowing which exact route to take. The problem now is that the simulations take concerningly long without any result after 2 hours. Are my input files wrong? I just added the facilities for testing but I assume it should be possible without them.

Additionally, are there any known perforance limitations when it comes to network file size?

config.xml:

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE config SYSTEM "./dtd/config_v2.dtd">

<config>

    <module name="network">
        <param name="inputNetworkFile" value="network_onlyTrucksMerged.xml" />
    </module>

    <module name="plans">
        <param name="inputPlansFile" value="plans.xml" />
    </module>

    <module name="facilities">
        <param name="inputFacilitiesFile" value="facilities.xml"/>
        <param name="facilitiesSource" value="fromFile"/>
    </module>

    <module name="controler">
        <param name="outputDirectory" value="./output" />
        <param name="overwriteFiles" value="deleteDirectoryIfExists"/>
        <param name="lastIteration" value="3" />
        <param name="mobsim" value="qsim" />
    </module>

    <module name="planCalcScore">
        <parameterset type="activityParams">
            <param name="activityType" value="AH" />
            <param name="typicalDuration" value="02:00:00" />
        </parameterset>
        <parameterset type="activityParams">
            <param name="activityType" value="AM" />
            <param name="typicalDuration" value="02:00:00" />
        </parameterset>
        <parameterset type="activityParams">
            <param name="activityType" value="AF" />
            <param name="typicalDuration" value="02:00:00" />
        </parameterset>
    </module>

    <module name="strategy">
        <param name="maxAgentPlanMemorySize" value="5" /> <!-- 0 means unlimited -->

        <parameterset type="strategysettings">
            <param name="strategyName" value="BestScore" />
            <param name="weight" value="0.9" />
        </parameterset>

        <parameterset type="strategysettings">
            <param name="strategyName" value="ReRoute" />
            <param name="weight" value="0.1" />
        </parameterset>
    </module>

</config>

facilities.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE facilities SYSTEM "http://www.matsim.org/files/dtd/facilities_v1.dtd">

<facilities>

    <facility id="1" x="8.2453535" y="50.0201239">
        <activity type="AM"/>
    </facility>

    <facility id="2" x="8.3786165" y="50.0220971">
        <activity type="AF"/>
    </facility>

    <facility id="3" x="8.6452802" y="50.1140535">
        <activity type="AH"/>
    </facility>

</facilities>

plans.xml:

<?xml version="1.0" ?>
<!DOCTYPE plans SYSTEM "http://www.matsim.org/files/dtd/plans_v4.dtd">
<plans>
    <person id="1">
        <plan>
            <act type="AH" facility="3" x="8.6452802" y="50.1140535" start_time="04:50:00.00" end_time="04:58:15.20" />
            <leg num="0" mode="car" dep_time="04:58:15.20"> </leg>
            <act type="AF" facility="2" x="8.3786165" y="50.0220971" end_time="05:52:30.70" />
            <leg num="1" mode="car" dep_time="05:52:30.70"> </leg>
            <act type="AH" facility="3" x="8.6452802" y="50.1140535" />
        </plan>
    </person>
    <person id="5">
        <plan>
            <act type="AH" facility="3" x="8.6452802" y="50.1140535" start_time="04:50:00.00" end_time="04:23:37.90" />
            <leg num="0" mode="car" dep_time="04:23:37.90"> </leg>
            <act type="AF" facility="2" x="8.3786165" y="50.0220971" end_time="05:13:39.82" />
            <leg num="1" mode="car" dep_time="05:13:39.82"> </leg>
            <act type="AH" facility="3" x="8.6452802" y="50.1140535" />
        </plan>
    </person>
</plans>

thank you in advance!

example logfile: logfile.log

Janekdererste commented 5 months ago

It looks like your simulation is stuck in the routing phase and does never start.

One thing I noticed is that your first activities have start times. Usually, we model things so that agents start the simulation at the first activity (no start_time) and then start the day at the end_time of the first activity. I don't know whether this causes your problem.

Additionally, are there any known perforance limitations when it comes to network file size?

Not really. You need more RAM if the network is larger. You can easily have more than a million links in your network, without the simulation slowing down. The runtime is determined by the number of agents being on the network.

Tim-Schwabe commented 5 months ago

Usually, we model things so that agents start the simulation at the first activity (no start_time) and then start the day at the end_time of the first activity. I don't know whether this causes your problem.

Unfortunately that did not resolve it. Since you guess that the problem is in the routing phase, could loose ends in the network be a problem?

kainagel commented 5 months ago

log warnings such as this

2024-06-03T09:22:34,830  WARN AStarEuclidean:172 finding totalCost=0.0; this will often (or always?) lead to a null pointer exception later.  In my own case, it was related to a network having freespeed infinity at places.  linkId=75530. kai, jan'18

point to a network error. Could you please post network.xml(.gz)?

Tim-Schwabe commented 5 months ago

network_onlyTrucksMerged.zip This is an edited version of network.zip, which is directly from OSM, but they produce the same problem I believe

kainagel commented 5 months ago

Something like this here:

        <link id="75530" from="410304" to="410302" length="0.000320410705451724" freespeed="13.88888888888889" capacity="1500.0" permlanes="1.0" oneway="1" modes="bus,car,pt,truck">

is a really really short link length, which seems to be rounded down to 0. It looks like the network uses coordinate system WGS84. This will not work. Easiest solution would be to

  1. declare that the network is in WGS84 coordinates (optimally in the file itself; can someone state the syntax?)
  2. declare some other (preferably metric) coordinate system in global in the config.

The userguide, in section "2.3.4.3 Coordinate Systems", says something about coordinate systems.
If everything works as intended, then the WGS84 coordinate system in network.xml will be converted into the "global" coordinate system before matsim starts.

Tim-Schwabe commented 5 months ago

Thank you, that seemed to have helped a lot, because now the simulation runs as expected, however the output in events.xml looks not as expedted:

<events version="1.0">
<event time="15818.0" type="actend" person="5" facility="3" link="10" x="8.6452802" y="50.1140535" actType="AH"/>
<event time="15818.0" type="departure" person="5" link="10" legMode="car" computationalRoutingMode="car"/>
<event time="15818.0" type="PersonEntersVehicle" person="5" vehicle="5"/>
<event time="15818.0" type="vehicle enters traffic" person="5" link="10" vehicle="5" networkMode="car" relativePosition="1.0"/>
<event time="15818.0" type="vehicle leaves traffic" person="5" link="10" vehicle="5" networkMode="car" relativePosition="1.0"/>
<event time="15818.0" type="PersonLeavesVehicle" person="5" vehicle="5"/>
<event time="15818.0" type="arrival" person="5" link="10" legMode="car"/>
<event time="15818.0" type="actstart" person="5" facility="2" link="10" x="8.3786165" y="50.0220971" actType="AF"/>
<event time="17896.0" type="actend" person="1" facility="3" link="10" x="8.6452802" y="50.1140535" actType="AH"/>
<event time="17896.0" type="departure" person="1" link="10" legMode="car" computationalRoutingMode="car"/>
<event time="17896.0" type="PersonEntersVehicle" person="1" vehicle="1"/>
<event time="17896.0" type="vehicle enters traffic" person="1" link="10" vehicle="1" networkMode="car" relativePosition="1.0"/>
<event time="17896.0" type="vehicle leaves traffic" person="1" link="10" vehicle="1" networkMode="car" relativePosition="1.0"/>
<event time="17896.0" type="PersonLeavesVehicle" person="1" vehicle="1"/>
<event time="17896.0" type="arrival" person="1" link="10" legMode="car"/>
<event time="17896.0" type="actstart" person="1" facility="2" link="10" x="8.3786165" y="50.0220971" actType="AF"/>
<event time="18820.0" type="actend" person="5" facility="2" link="10" x="8.3786165" y="50.0220971" actType="AF"/>
<event time="18820.0" type="departure" person="5" link="10" legMode="car" computationalRoutingMode="car"/>
<event time="18820.0" type="PersonEntersVehicle" person="5" vehicle="5"/>
<event time="18820.0" type="vehicle enters traffic" person="5" link="10" vehicle="5" networkMode="car" relativePosition="1.0"/>
<event time="18820.0" type="vehicle leaves traffic" person="5" link="10" vehicle="5" networkMode="car" relativePosition="1.0"/>
<event time="18820.0" type="PersonLeavesVehicle" person="5" vehicle="5"/>
<event time="18820.0" type="arrival" person="5" link="10" legMode="car"/>
<event time="18820.0" type="actstart" person="5" facility="3" link="10" x="8.6452802" y="50.1140535" actType="AH"/>
<event time="21151.0" type="actend" person="1" facility="2" link="10" x="8.3786165" y="50.0220971" actType="AF"/>
<event time="21151.0" type="departure" person="1" link="10" legMode="car" computationalRoutingMode="car"/>
<event time="21151.0" type="PersonEntersVehicle" person="1" vehicle="1"/>
<event time="21151.0" type="vehicle enters traffic" person="1" link="10" vehicle="1" networkMode="car" relativePosition="1.0"/>
<event time="21151.0" type="vehicle leaves traffic" person="1" link="10" vehicle="1" networkMode="car" relativePosition="1.0"/>
<event time="21151.0" type="PersonLeavesVehicle" person="1" vehicle="1"/>
<event time="21151.0" type="arrival" person="1" link="10" legMode="car"/>
<event time="21151.0" type="actstart" person="1" facility="3" link="10" x="8.6452802" y="50.1140535" actType="AH"/>
</events>

the link 10, which is being referenced every time, is never specified as far as I know.

I tried to follow this answer, but I am also really confused by all these coordinate codes. Does anyone know if this is correct? (network represents Frankfurt, Germany):

<module name="global">
        <param name="coordinateSystem" value="EPSG:5243" />
    </module>

    <module name="network">
        <param name="inputNetworkFile" value="network_onlyTrucksMerged.xml" />
        <param name="inputCRS" value="WGS84" />
    </module>

    <module name="plans">
        <param name="inputPlansFile" value="plans.xml" />
        <param name="inputCRS" value="WGS84" />
    </module>

logfile for reference: logfile.log

EDIT: I realized that I forgot to reference the facilities, but that did not change anything

Janekdererste commented 5 months ago

Does anyone know if this is correct

This looks correct. In case you are interested, WGS84 is a geographic coordinate system, but MATSim requires a projected coordinate system because it performs Euclidean math.

You do have a link "10" in your network.xml

<link id="10" from="641099" to="641102" length="0.003132823183265079" freespeed="13.88888888888889" capacity="1500.0" permlanes="1.0" oneway="1" modes="bus,car,pt,truck">
            <attributes>

You have set up your scenario, so that activities are referenced to facilities. The facilities carry the geographical location in the form of (x, y)-coordinates. On startup, the simulation takes these coordinates and assigns facilities to links so that the simulation knows on which link agents should start their trips. You can consider this link to be the postal address of the facility. By default, the simulation assigns facilities to the closest link. If you don't want that, you can assign facilities to links yourself, by adding a link="your-link-id" to facilities.

Now, looking at your events file, the agent leaves facitlity 3, which is at x="8.6452802" y="50.1140535" and travels to facility 2 which is at x="8.3786165" y="50.0220971". Those facilities are only 21m apart from each other. Probably both facilities were assigend to link 10.

Janekdererste commented 5 months ago

As a general recommendation, it is tricky to get coordinate reference systems right, especially, if you are not experienced with using them. Often it is easiest, to select a projected system and transform all your input files into that coordinate system. In your case, this would be EPSG:5243. A good choice for Frankfurt/Germany would also be EPSG:25832. This way you can directly see things like facilities being close to each other because all your x and y units are in meters, instead of degrees which is the case for WGS84.

Tim-Schwabe commented 5 months ago

Thank you all! It works now. It seems that the coordination system mapping was correct, but as you suggested the facilities were the problem. I have removed them now completely because I do not really need them, but for best practice purposes I might try to get them running with the hardcoding of link="link_id". A curious thing however is, that I got an error that MATSim could not find a link for the facilities, even though I located them on nodes that are already part of a link. Maybe that is forbidden.