On a dtrace data file of 2342965 lines and a perf map file of 19072 lines, processing of dtrace-perf-map.pl took well over 5 hours.
Investigation demonstrated that this was because of the inner loop over the @map_entries array; reducing the length of that array to 10 would give a 5s runtime, 20 a 10s, 30 a 15s, etc.
Perl is good for a lot of things but not for 4 billion array iterations.
It seemed to me that we could improve this by sorting the array and applying binary search (An extent map was also considered but thought too complex to build, though fun).
Processing the same file now takes roughly 20s on the same machine (11s on mine).
On a dtrace data file of 2342965 lines and a perf map file of 19072 lines, processing of dtrace-perf-map.pl took well over 5 hours.
Investigation demonstrated that this was because of the inner loop over the
@map_entriesarray; reducing the length of that array to 10 would give a 5s runtime, 20 a 10s, 30 a 15s, etc. Perl is good for a lot of things but not for 4 billion array iterations.It seemed to me that we could improve this by sorting the array and applying binary search (An extent map was also considered but thought too complex to build, though fun). Processing the same file now takes roughly 20s on the same machine (11s on mine).
Or, binary search go brrr.