Some tests running intermodal-access-egress-network which surfaced the problem:
PT2MATSim is our small test network in genet
Londinium and Londinium in docker is a smaller Londinium network
Londinium chonk is a denser Londinium (with detailed walk and bike networks)
Pre python 3.11:
PT2MATSim: 5.68s user 1.36s system 31% cpu 22.555 total
Londinium: 14.06s user 0.57s system 91% cpu 15.927 total
Londinium in docker: 0.01s user 0.03s system 0% cpu 29.674 total
Londinium chonk: 38.46s user 1.58s system 78% cpu 51.189 total
Post python 3.11:
PT2MATSim: 6.75s user 2.32s system 128% cpu 7.085 total
Londinium: 34.45s user 3.49s system 106% cpu 35.707 total
Londinium in docker: 0.01s user 0.01s system 0% cpu 52.099 total
Londinium chonk: 1483.67s user 19.35s system 98% cpu 25:26.03 total
After fix:
PT2MATSim: 9.19s user 3.28s system 33% cpu 37.537 total
Londinium: 9.31s user 2.81s system 114% cpu 10.598 total
Londinium in docker: 0.01s user 0.03s system 0% cpu 14.583 total
Londinium chonk: 21.48s user 2.51s system 103% cpu 23.143 total
The situation on AWS was even worse, the intermodal-access-egress-network process using Londinium chonk network was hanging for hours in a batch job.
These links need authentication to TCM dev AWS account:
this is the state machine execution that hung for a couple of hours yesterday due to this problem, before being aborted
this is an execution with the same inputs plus genet image from this branch
Resolves #199 by changing pandas version, see issue for more details. Some small changes to readme and setup to match with https://github.com/arup-group/genet/pull/192
Some tests running
intermodal-access-egress-network
which surfaced the problem:PT2MATSim
is our small test network in genetLondinium
andLondinium in docker
is a smaller Londinium networkLondinium chonk
is a denserLondinium
(with detailed walk and bike networks)Pre python 3.11:
Post python 3.11:
After fix:
The situation on AWS was even worse, the
intermodal-access-egress-network
process usingLondinium chonk
network was hanging for hours in a batch job. These links need authentication to TCM dev AWS account: