Benchmark data reduction speeds

astrofle commented 5 days ago

We should be testing that dysh is at least as fast as GBTIDL.

teuben commented 5 days ago

I'll advocate we come up with some kind of "DDT" like they had in AIPS. CASA never wanted to commit to it. We had one in miriad, called mirbench, I have nemobench5 in NEMO, modeled after how geekbench5 works. Their 5 comes from version 5 (now they are at 6). My 5 comes from each benchmark program is normalized to take 5 seconds on a specific chip, so you take the average of them and turn it into a speed, 1000 being the speed on that specific architecture. This allows you (like geekbench6) to move along as architectures improve.

teuben commented 4 days ago

I took a look at the large (700MB+) PS file from example_data. These are 151 integrations of a single IF/PL, but the file itself has 5 IFs and 2 PLs.

filein,"TGBT21A_501_11.raw.vegas.fits"
getps,152
exit

and for dysh

s1 = GBTFITSLoad("TGBT21A_501_11.raw.vegas.fits")
p1 = s1.getps(scan=152,ifnum=0,plnum=0)

with /usr/bin/time outputs of shell type access

dysh:    6.55user 1.39system 0:08.47elapsed 93%CPU
gbtidl:  2.54user 0.36system 0:06.52elapsed 44%CPU 

dysh     2.07user 0.36system 0:03.01elapsed 81%CPU 
gbtidl   0.22user 0.14system 0:02.77elapsed 13%CPU

In the first pair the total shell time was measure with the benchmark, the second pair is a dummy loading and exiting dysh/gbtidl

Most of the dysh time went into calibration, 400ms without calibration, 3.5sec with. Those were low level %time results from getps().

I also have low level C and numpy examples, but have to dig this up. The numbers here were measured today on our lma machine.

GreenBankObservatory / dysh

Benchmark data reduction speeds #279