ActivitySim / activitysim

An Open Platform for Activity-Based Travel Modeling
https://activitysim.github.io
BSD 3-Clause "New" or "Revised" License
191 stars 99 forks source link

Multithreading (in addition to multiprocessing) #740

Open joecastiglione opened 1 year ago

joecastiglione commented 1 year ago

ActivitySim currently allows multiprocessing by breaking up the problem for individual components and then running these “chunks” in parallel. Doing this work requires the segmentation of the pipeline objects, such that each process can read from and write to the correct location. This process can be improved by leveraging the numba package’s multithreading capabilities, which takes care of the safe write problem. This can allow additional computing resources to be brought to bear to tasks with a multiprocessing segmentation.

lmz commented 12 months ago

I'd like to learn more about this one, to help me understand if this makes sense. Is there anything further I could read?