Closed galfthan closed 9 years ago
I checked with testpackage version of Magnetosphere that restarting with this patch is bitidentical compared to restarting with the old version
Performance test on Voima. Restarting AAS_remake_pqm2 from 1149s (350GB file) on 60 nodes.
Performance, old above and new below
# avgtime maxtime mintime
original: readBlockData 129.3 96.45 235.9 490 25.52 1 1
withfix: readBlockData 108.8 94.34 210.4 210 6.929 1 1
Some performance improvement, even if this is so small that it might also be random fluctuation. One can also see that we have bad load balance when reading. Max is almost 2x average
This is fixed
In _readBlockData in ioread.cpp the assignement of blockdata could be streamlined.
Now it is computing a velocity for each phase space cell, and adding that individually to the velocity mesh using the slow set_value operation (one hash map lookup and a lot of index math). Instead on can create all blocks in one operation and then copy all blockdata with one loop.
The number of phase space cells per process can be very large, up to 100 M, and thus this inefficiency may be visible in run-time. Most likely the actual file io is still the big bottleneck.
In https://github.com/galfthan/vlasiator/tree/faster_block_reads there is a patch, but it has not yet been tested. Once that has been done (performance and that it actually works) a pull request can be done.