search

pzivich / MossSpider

Implemention of targeted maximum likelihood estimation for network-dependent data
https://mossspider.readthedocs.io/en/latest/
MIT License
1 stars 0 forks source link

Speed up NetworkTMLE #1

Open pzivich opened 2 years ago

pzivich commented 2 years ago

Right now the bottleneck is the creation of the data for the IPW numerator / Monte Carlo integration. This should be easy to speed up (via Pool) or using some clever tricks in generation. The real issue is the creation of all the summary measures for each copy (lots of matrix multiplications)

pzivich commented 2 years ago

So, the packing in Pool is expensive. However, when calculating all the different summary measures, this initial expense becomes worthwhile. I would like to find something that is quicker than Pool with the packing into a list. If I can find something that is, then everything will be far faster

pzivich commented 2 years ago

namespace looks like a promising alternative to access all the objects without copying

https://superfastpython.com/multiprocessing-manager-namespace/