Urban-Analytics-Technology-Platform / acbm

activity-based modelling pipeline (for transport demand models)
https://hackmd.io/w-m_OKaDT3GGBfSqFPpBjA
Apache License 2.0
4 stars 1 forks source link

Speeding up the matching process #25

Closed BZ-BowenZhang closed 1 week ago

BZ-BowenZhang commented 5 months ago

The current matching process runs slowly because Python's for loop is inefficient. Thus, we need to consider how to speed up the matching process to deal with the bigger dataset.

Tools

  1. Using the multithreaded processing package to speed up (e.g. multiprocess)
  2. Partly rewrite the functions to reduce the use of for loops.
sgreenbury commented 4 months ago

As discussed, we'll revisit optimisation after updating to an alternative matching approach in #13 and continue to work with a sample of 15000 of the SPC population for now.