Open adele-morrison opened 2 months ago
Good point!
If I recall correctly, there are some comments on the utility of each method sprinkled in the recipe. But your question above suggests that either these comments are a bit hidden or perhaps it worths reconsidering about the alternative methods and whether they overall help or confuse?
@dhruvbhagtani may be able to shed some insight?
The way it’s written it seems to be encouraging people to use those methods and that they are ok. The only reason I could see for having those methods is to illustrate to people that they shouldn’t use them! But I’m not sure that’s worth the space.
Maybe I'm missing something, but I do not completely agree with the other two methods being incorrect. Could you tell me -- Is it just this plot (I agree the orange and green line are far off from observations, but so is the blue), or is there a physical reasoning behind this inference?
The reason I ask is because according to me, all three methods are theoretically correct: Method 1: Computes using an online diagnostic, Method 2: Assuming steady state, MHT(y) = surface heat flux integral from y = y_min to y Method 3: If the difference in temperature between northward and southward flow at a latitude is T and the flow volume is V, then. MHT(y) = \rho Cp multiplied by the zonal integral of VT.
Out of all the three methods, the first one is least prone to errors, since other methods have certain assumptions (steady state, or frequency of outputs of velocity and temperature). Maybe we can jumble the order of these methods to end with the best. Alternatively, we can suggest that methods 2 and 3 are great for when the online MHT diagnostic is not outputted, which is often the case?
From the comment above, I think @adele-morrison is taking as the ground truth or what she refers as "correct" the methods where the heat transport you get from the online diagnostic.
She was not intending to compare them to observations (otherwise we go into discussions whether ocean models are correct etc...)
Alternatively, we can suggest that methods 2 and 3 are great for when the online MHT diagnostic is not outputted, which is often the case?
I think this is the key point why we need the other methods.
Isn't it?
I don’t think they’re great. Method 2 could probably be made exact by including heat content change from restarts snapshots?
Ah, I see the point now!
Alternatively, we can suggest that methods 2 and 3 are great for when the online MHT diagnostic is not outputted, which is often the case?
I think this is the key point why we need the other methods.
Isn't it?
Yep, they are less accurate than the first one, and are good alternatives when we don't have the diagnostic handy.
I don’t think they’re great. Method 2 could probably be made exact by including heat content change from restarts snapshots?
Yes, that'll be a great addition! I can try my hand at it (I've never dealt with analysing restart data but hopefully it's not too complicated) next week.
Hmm I don’t think we should be encouraging people to use incorrect alternatives at all if they are wrong by a factor of 2. Method 2 is wrong because the assumption of steady state doesn’t hold, because we don’t run our models long enough to equilibrate. Method 3 is wrong because it misses out on high frequency correlations between T and v.
What about deleting method 3, and making method 2 exact by adding the heat content change?
(cc @rmholmes might be interested in this chat)
Umm, it depends on what we want for others when they read this notebook, and that could be subjective. I see your reasoning to update method 2 and remove 3. In my opinion, missing out on high-frequency oscillations isn't a method problem. For example, if we had daily outputs, then I would expect the errors to go down (see Fig. 2 of Yung and Holmes 2023) that shows a large fraction of 10 day-2 month MHT that we are probably missing. Hence, it's still a useful method. We could maybe recompute using daily outputs to reduce errors.
Before we go down any of these routes we discussed (all of which will anyways improve this example in some respect), I think it's worthwhile to see Ryan's opinion as well, and we can make a decision.
Yes, though even using daily output for method 3 misses a significant fraction of the heat transport across the Antarctic continental slope, where the overflows occur on sub-daily timescales in ACCESS-OM2-01.
I would tend to agree with Adele. Method 1 is correct up to numerical precision (+ I think the effect of the SSH smoother, which is not really MHT and is small but would be needed to close the longitudinally-integrated heat budget). If you want to keep the other two I would suggest indicating everywhere in the notebook, including in the figure caption, that they are approximate.
One reason to retain method 2 as is (i.e. without accounting for tendency) is that this is how some of the observational estimates are made and so that comparison is more comparable. This is noted in the notebook, although it would be good to include more detail about which observational estimates use which methods.
Also note that:
OK. From what I see then I think delete method 3 and add a clear(er) caveat for method 2 then. Perhaps don't even call it "method 2" but just "alternative approximate MHT calculation when X diagnostic is not available"?
How does this sound to all?
Sounds good. Plus showing how to add heat content change from snapshots to method 2 to show that method 1 could be reconstructed exactly if that was the aim (in addition to without this to reproduce the obs method).
That also sounds great. We can leave the issue open and see if anybody picks it up.
Even just deleting the Method 3 and adding a clearer caveat for Method 2 is a great contribution; what @adele-morrison suggests above it an even better enhancement but sounds more work.
The Meridional_heat_transport.ipynb notebook currently calculates heat transport by 3 methods. Two of those methods are clearly incorrect, as shown in the figure from the notebook, copied here. The blue line is the correct method using the online heat transport diagnostic, and the orange and green are incorrect methods (that are wrong by a factor of ~2 in the Southern Hemisphere!):
I suggest we simplify this notebook and delete the incorrect methods 2 and 3, because we don't want to encourage people to compute incorrect metrics. We should encourage people to output the correct diagnostics, not try to hash together incorrect answers after the fact.
Am I missing something obvious about why we need these?