search

lukelbd / climate-benchmarks

⏱ Various benchmark comparisons for data analysis with python, julia, CDO, NCO, and NCL
22 stars 2 forks source link

Add xgcm #2

Closed jbusecke closed 2 years ago

jbusecke commented 4 years ago

I built a prototype for doing the isobar2isentropic interpolation in xgcm. I have done some very preliminary output comparison, and the values seem to be similar. I don't expect them to be exactly the same as metpy, since they are using some sort of iterative refinement I believe.

I was not able to compare to NCL, because it is not installed locally on my computer.

I did a preliminary test on my imac (which seems to be slightly slower than the macbook used for the results in the README) and it seems like xgcm is actually quite performant (particular for larger datasets) 🚀:

Dataset: /Users/juliusbusecke/Desktop/atmos-benchmarks/60lev/dataN0009T200_3.nc (30M)
Logfile: /Users/juliusbusecke/Desktop/atmos-benchmarks/results/./isobars2isentropes_60lev_iMac.log
MetPy
3.667s
MetPy + Dask
2.662s
xgcm
7.304s
xgcm + Dask
7.186s
NCL
0.001s

Dataset: /Users/juliusbusecke/Desktop/atmos-benchmarks/60lev/dataN0018T200_3.nc (118M)
Logfile: /Users/juliusbusecke/Desktop/atmos-benchmarks/results/./isobars2isentropes_60lev_iMac.log
MetPy
6.834s
MetPy + Dask
5.927s
xgcm
7.998s
xgcm + Dask
7.234s
NCL
0.001s

Dataset: /Users/juliusbusecke/Desktop/atmos-benchmarks/60lev/dataN0036T200_3.nc (475M)
Logfile: /Users/juliusbusecke/Desktop/atmos-benchmarks/results/./isobars2isentropes_60lev_iMac.log
MetPy
23.288s
MetPy + Dask
19.225s
xgcm
10.709s
xgcm + Dask
7.447s
NCL
0.001s

Dataset: /Users/juliusbusecke/Desktop/atmos-benchmarks/60lev/dataN0060T200_3.nc (1.3G)
Logfile: /Users/juliusbusecke/Desktop/atmos-benchmarks/results/./isobars2isentropes_60lev_iMac.log
MetPy
61.566s
MetPy + Dask
62.851s
xgcm
17.900s
xgcm + Dask
8.379s
NCL
0.001s

Dataset: /Users/juliusbusecke/Desktop/atmos-benchmarks/60lev/dataN0090T200_3.nc (2.9G)
Logfile: /Users/juliusbusecke/Desktop/atmos-benchmarks/results/./isobars2isentropes_60lev_iMac.log
MetPy
154.227s
MetPy + Dask
148.069s
xgcm
34.491s
xgcm + Dask
9.925s
NCL
0.002s

To use xgcm, you need to install both xgcm and numba with conda install -c conda-forge xgcm numba.

Do you think you will be able to rerun these tests including xgcm?

jbusecke commented 3 years ago

Hi @lukelbd, would you be interested in adding xgcm to this package at all? Otherwise I'd be happy to close this.

jbusecke commented 2 years ago

I am going to close this, if there is some interest in this at some point, we can reopen.