search

EDmodel / ED2

Ecosystem Demography Model
78 stars 112 forks source link

Tower vs Fast File Output #97

Closed DanielNScott closed 9 years ago

DanielNScott commented 9 years ago

Disclaimer: I know this is a design issue, not really a science issue, and that nobody necessarily has or wants to or should make an investment of time in changing anything. However...

Is there a 'real' reason we have both?

It seems like "tower" output is really a more elegant solution to producing "fast" output, especially given that file i/o is one of the slowest operations one can perform, presumably on both the fortran output and the post-processing-input sides.

As such, shouldn't we just deprecate IFOUTPUT and remove it sometime?

One reason I bring this up is because I find myself adding stuff to the tower output...

mpaiao commented 9 years ago

@DanielNScott I think the only reason to keep both "fast" is that the tower output only accepts polygon-level variables, according to @mdietze (issue #76). I may be wrong but I think it doesn't work with polygon matrices either (e.g. soil variables aggregated to polygon).

mdietze commented 9 years ago

The -T- design does work with polygon-level depth-varying soil and snow variables. It deliberately doesn't work with site, patch, or cohort variables because they vary in dimension and I wanted something simple and predictable.

As the 'tower' name implies I first created these to make it easy to compare ED2 to eddy covariance towers. The reason we have both is that the -I- files came first but they required a ton of slow i/o intensive post-processing to generate simple NEE, ET, and heat flux time series. This post-processing time was a particularly annoying bottleneck since this occurred in concert with when I first started doing data assimilation through R, so I was doing thousands of runs. So I created the tower files as a solution.

In practical terms, if you don't turn the -I- files on it doesn't really cost you anything to keep the code around, while it does cost you time and effort to remove it, so I'm content either way so long as the 'tower' files stick around.

DanielNScott commented 9 years ago

Ah, yes great. I think I noticed that before (the lack of support for site, patch etc) and then forgot. That really answer's my question. And I agree with you mike on the cost-of-removal aspect obviously.