search

GGraeler / My_Pysteps

Edited Pysteps for COMET Summer 2021 Research. Greta Graeler & Carter Humphreys 2021.
0 stars 0 forks source link

Add multiprocessing to pysteps functions #1

Open HumphreysCarter opened 3 years ago

HumphreysCarter commented 3 years ago

Reading in of MRMS files in pysteps.io.readers.py with read_timeseries() function.

https://github.com/GGraeler/My_Pystep/blob/02f26554d2b3c85f39a0f440a3b20b889262794d/pysteps/io/readers.py#L61-L73

HumphreysCarter commented 3 years ago

dating() function in pysteps.tracking.tdating.py

https://github.com/GGraeler/My_Pystep/blob/02f26554d2b3c85f39a0f440a3b20b889262794d/pysteps/tracking/tdating.py#L156-L181

HumphreysCarter commented 3 years ago

Reading in of MRMS files in pysteps.io.readers.py with read_timeseries() function.

https://github.com/GGraeler/My_Pystep/blob/02f26554d2b3c85f39a0f440a3b20b889262794d/pysteps/io/readers.py#L61-L73

Added support in commit ee04bf68fb87651130370ba29bb8ee13662eae44

HumphreysCarter commented 3 years ago

dating() function in pysteps.tracking.tdating.py

https://github.com/GGraeler/My_Pystep/blob/02f26554d2b3c85f39a0f440a3b20b889262794d/pysteps/tracking/tdating.py#L156-L181

Implement multiprocessing over cell detection, since it can be non-sequential over time. https://github.com/GGraeler/My_Pystep/blob/3eea8f610c28d3be749231eeddc93e99fd7f59c0/pysteps_modified/tracking/tdating.py#L157-L166

Cell advection does need to be done sequentially, due to the tracking() call. A list of cells from the above block could be sorted and then each time step pair (current, next) could be passed to the tracking() function. https://github.com/GGraeler/My_Pystep/blob/3eea8f610c28d3be749231eeddc93e99fd7f59c0/pysteps_modified/tracking/tdating.py#L167-L181

Performance should be improved since multiple time steps of cells could be identified at once, however long execution times can't be avoid with the cell advection in tracking() due to the sequential nature.