How to present results from multiple algos in paper

ocean-transport / scale-aware-air-sea

Repo for collaborative project on scale-aware air-sea fluxes

2 stars 0 forks source link

How to present results from multiple algos in paper #45

Open jbusecke opened 2 years ago

jbusecke commented 2 years ago

@ocean-transport/collab_team Our most recent results indicate that differences between algos are very minor and thus I suggest not focussing on those front and center in the paper.

I suggest the following:

For maps we show the averaged values across all algos we processed (this might change slightly) + Some in detail differences for the appendix
For histograms we could show multiple distributions?
For any aggregated metrics (e.g. global small scale heatflux) we could give the mean + std/var between algos as uncertainty?

Happy to make adjustments, just wanted to start a discussion

jbusecke commented 2 years ago

Here is a plot I made for the Appendix

Rows are latent and sensible heat flux. Columns are: (left) Small scale contribution averaged over algos; (middle) stdv between algos; (right) relative 'error' between algos with respect to the small scale contribution.

We can see that the difference between algos is not always negligible, but the high relative values are generally where the small scale contribution is small to begin with, so not that relevant to our results. Wonder how we could show that more conclusively.

jbusecke commented 2 years ago

We had a good discussion of this today, and there was an alternative suggestion to use the algo that makes most sense for the model (ecmwf?) and show results from that algo (instead of averaging across them). Either way the uncertainties would have to be discussed (in the Appendix presumably), but I would like to get @rabernat s opinion on this.