Open anthony-meza opened 1 month ago
UPDATE:
I think this needs to be done in parallel. ATM it takes far too long (~2 hours) to fit harmonics on just 3 years of 6 hourly data.
UPDATE (again):
After some thinking, I think it would make the most sense to just do all matrix multiplications as arrays (rather than MeshArrays). This would allow us to take full advantage of the BLAS optimizations
suggestion: split mean problem and seasonal harmonics problem.
Seasonal cycle matrices can be computed using the following ECCOtour.jl code
Ecycle,Fcycle = seasonal_matrices(fcycle,t)
where t is your timegrid.The seasonal cycle of a variable
x
can be computed using the following code:βcycle = Fcycle*x xseasonal = Ecycle*βcycle
Alternatively, could finexseasonal
by using an equivalent code:E\hat = Ecycle*Fcycle xseasonal = E\hat*x
UsingE\hat
, we could probably usematrixfilter
to solve for the seasonal cycle of atmospheric variables. 26 years of atmospheric variables only requires ~26GB of RAM and ilko has more than that, so with proper garbage collect we would not reach any sorts of out of memory errors. Alternatively, we could operate on each face to save some memory. Either way, the following code would work.matrixfilter(E\hat,filein,years,γ) #where perhaps we only read in each face