search

FireDynamics / fdsreader

Python reader for FDS data
GNU General Public License v3.0
45 stars 18 forks source link

Parallel reading #23

Open JanVogelsang opened 3 years ago

JanVogelsang commented 3 years ago

It should be possible to read in data for most classes using multiple threads or even processes (processes might use another data bus and therefore be more performant, will have to check if that really is the case).

JanVogelsang commented 3 years ago

For all cases that has been tested, read times have not been any problem.

If you experience enormously long loading times, please send your FDS case and we might think of further improving the reading performance.

JanVogelsang commented 3 years ago

Some simple tests didn't yield any noticeable speedup when reading in data using multiple threads. For small amounts of data (~30MB files), the time it took to read in and process all data even increased by about 5%.

For most use cases simple parallelization won't be applicable and more sophisticated approaches are needed. For HPC systems various approaches will yield great results, however the same approaches will most likely not work for common PC-systems due to only having a single memory and disk bus.