Testing on ciclad.ipsl.jussieu.fr

[eguil]

First test on ciclad

• install PCMDI metrics package + durolib + pytz
• run drive_ACCESS1-0.py in batch mode
• compare with perf on crunchy: 50% slower

12 months ACCESS1.0:

ciclad vs. crunchy (bining per month):

average cpu1  =  1.38083333333     average cpu1  =  1.23916666667
average cpu2  =  2.995             average cpu2  =  2.12666666667
average cpu3  =  1.26083333333     average cpu3  =  1.0475
average cpu4  =  31.4675           average cpu4  =  22.6633333333
average cpu40 =  0.313333333333    average cpu40 =  0.0825
average cpu5  =  1.40333333333     average cpu5  =  1.035

CPU read T/S  =  57.04             CPU read T/S  =  15.47
CPU comp. rho =  95.15             CPU comp. rho =  37.62

ciclad vs. crunchy (overall):

 CPU of chunk inits         = 179.13   CPU of chunk inits         = 67.49
 CPU of density bining      = 465.85   CPU of density bining      = 338.33
 CPU of masking and var def = 31.35    CPU of masking and var def = 29.15
 CPU of annual mean compute = 89.47    CPU of annual mean compute = 54.33
 CPU of interpolation       = 9.79     CPU of interpolation       = 4.88
 CPU of zonal mean          = 10.51    CPU of zonal mean          = 8.08
 CPU of persistence compute = 24.42    CPU of persistence compute = 23.51
 CPU of chunk               = 811.11   CPU of chunk               = 526.09

Ciclad perf:

 Max memory use 8.731316 GB
 Ratio to grid*nyears 1.61691037037 kB/unit(size*nyears)
 CPU use, elapsed 833.77 939.14218998
 Ratio to grid*nyears 12.8668209877 1.e-6 sec/unit(size*nyears)

Crunchy perf:

 Max memory use 8.77472 GB
 Ratio to grid*nyears 1.62494814815 kB/unit(size*nyears)
 CPU use, elapsed 552.96 884.007050991
 Ratio to grid*nyears 8.53333333333 1.e-6 sec/unit(size*nyears)

[durack1]

@eguil the numbers above are suggestive of I/O differences to me.. The memory seems to be almost identical, but the time taken to read the data could slow things right down.. Not sure this jives with the CPU timings though..?

[eguil]

@durack1 well, I/O is one issue but not the main one - look at the annual and zonal mean compute differences. cdutils.averager perfs. seem to be one issue here.

[durack1]

See #33, closing this as a dupe