Make movie of daily snapshots of relative effect of small scales

[paigem]

Make a video of daily snapshots of the relative effect of small scales so we can hopefully see what types of physical mechanisms are at play.

We are guessing that the systems may be synoptic scale atmospheric fronts, and we could verify this by overlaying the surface pressure contours.

This was based on discussions on Friday, Sep. 30th (notes here).

[paigem]

Here are 50-day movies of the relative small scale effect in both latent and sensible heat flux. These are made from daily snapshots.

Sensible heat flux

https://user-images.githubusercontent.com/26591824/194196600-c2d2e44d-2028-4c11-9ca1-4a4b8b081ff8.mp4

Latent heat flux

https://user-images.githubusercontent.com/26591824/194196656-fd260e38-896d-49c2-bf4f-bad242259fe7.mp4

Pressure contours

I haven't yet added pressure contours in the movies, but below is a single snapshot of the latent heat flux with the sea level pressure contours overlaid.

Take-aways so far

The movies show the small scales moving through time. To my eye, they do look like they are likely due to atmospheric fronts. The pressure contours don't line up perfectly with all of the patterns in the snapshot above, but adding contours to the movie may reveal more.

[jbusecke]

These are amazing @paigem! I agree that these structures (the local extrema) look VERY atmospheric (they move very fast, and they do not particularly confined by land boundaries. In terms of the pressure, I suggest to increase the amount of contours, so we could see 'bunching' which would indicates a frontal system?

[dhruvbalwada]

These look amazing! Could maybe look at some gradient quantity in the atmosphere too, like mapping atmospheric surface temperature gradients.

Also noticed that some of the features, like in the Agulhas, don't transit - indicating ocean signatures. :)

[jbusecke]

You are right! There are some "standing" features like the Gulf Stream too!

[paigem]

Good points @dhruvbalwada!! It's very cool to see some ocean signatures here! 😄

I have rerun the latent heat flux movie with pressure contours overlaid:

https://user-images.githubusercontent.com/26591824/194367468-521fd39d-22eb-4e8c-a732-2ce8c7da4a44.mp4

The frontal features here don't seem to align perfectly with the pressure contours, and often appear more perpendicular to the contours.

I've been reading a bit about how to detect atmospheric fronts, and it looks like surface temperature is a common indicator, though many metrics involving various surface fields are used. See for example these two papers:

• Automated detection of weather fronts using a deep learning neural network
• A simple diagnostic for the detection of atmospheric fronts

I'm currently running another movie with temperature contours to compare.

[jbusecke]

I think on a more basic level the suggestion to plot slp was more to confirm that stuff generally moves with pressure systems? Which I would say a lot of these definitely do! I wonder how deep we have to dive into 'front detection' here or if something like surface temperature gradient (which we can calculate relatively easy) is sufficient?

[paigem]

Two more movies for now:

Latent heat flux using ecmwf algorithm (as all previous plots) with contours of atmospheric reference temperature t_ref

https://user-images.githubusercontent.com/26591824/194466180-2779e76a-9261-4255-8d86-7cbe46c2ba44.mp4

Latent heat flux using ncar algorithm

https://user-images.githubusercontent.com/26591824/194465841-eb19a2c6-8e5e-402f-87b2-c49aae67ecce.mp4

[paigem]

Sensible heat flux using ncar method

https://user-images.githubusercontent.com/26591824/194957995-48dd4220-de85-425e-a8be-7bd8397f3f43.mp4

[paigem]

Latent heat with coare3p6 algorithm

https://user-images.githubusercontent.com/26591824/195194773-64526e18-56ac-4a23-8ba6-51564b1d46f2.mp4

Sensible heat with coare3p6 algorithm

https://user-images.githubusercontent.com/26591824/195194827-72ea69a0-d5ea-4c04-a95e-68a3951ab9a9.mp4

Latent heat with andreas algorithm

https://user-images.githubusercontent.com/26591824/195194951-9852fa1e-70d1-431a-9837-d99ff513e241.mp4

Sensible heat with andreas algorithm

https://user-images.githubusercontent.com/26591824/195194984-0966b6fc-5347-4f8b-a278-5352e2807306.mp4

Note the different colorbar scales for several of the above videos. andreas ranges from -40 to 40, the sensible heat using ncar ranges from -100 to 100, while all others range from -20 to 20.

[jbusecke]

Wow, I am actually very suprised by the difference seen in all the algorithms shown. To double check that there are no obvious errors, could we make some video of the total flux (and the large scale flux) with all algos? I would suggest 2 videos with 4 panels each, that show 4 algos side by side. The goal of this exercise for me would be to confirm that the overall sign/patterns are similar in all algos, and that the strong difference in behavior (even a change of sign) is indeed caused by the different response of each algo to the small scale structures.

[jbusecke]

@paigem let me know if that is something that I should help with. You have already done an amazing job with these movies!

[paigem]

Good idea @jbusecke to show several panels together! I'll run those today.

[paigem]

More movies! This time, I have all 4 algorithms together in the same video and these are for the full fields (i.e. with high-res input, and coarsened after computing the fluxes).

Preliminary observations

• All algorithms look very similar. There are slight differences in magnitude (so I know I am indeed plotting different algorithms here!), but it is very hard to see any differences. This is to be expected.

Next steps

• Compute the same movies for the large scale fields
• Would it make sense to put all the small-scale relative movies into one animation as below? I think probably yes, so I'll add this to my list unless others think it's not necessary.

Latent heat flux

https://user-images.githubusercontent.com/26591824/195717581-6744a926-6d2a-4048-bde6-08d22d8ba766.mp4

Sensible heat flux

https://user-images.githubusercontent.com/26591824/195717611-01131a0b-9b46-40b6-8b73-9b5e05f406a5.mp4

[paigem]

Here is a movie that @jbusecke and I discussed yesterday that includes large scales on the top row and small scales (absolute difference between full field and large scale) on the bottom row for all algorithms. After looking at the small scale relative figures, we are concerned that many of those regions we identified are due to near-zero values in the original field. So we will focus now on the small scales yielded from the absolute difference.

Sorry for the small sizing of the plots in the movie - will need to improve that moving forward. Also, the colorbars weren't included here, but the top row ranges from -50 to 50, and the bottom row from -10 to 10.

Sensible heat flux

https://user-images.githubusercontent.com/26591824/196428123-1ed839ee-6918-4c8c-a988-b2d262207e70.mp4

Preliminary observations

• ecmwf has minimal small scale effects
  • most differences are negative, indicating that small scales dampen the full field
  • with this colorbar, most patterns are short-lived "wisps", rather than large, coherent structures
  • most patterns are in the northern hemisphere midlatitudes, but some in the Southern Hemisphere midlatitudes as well
• NCAR shows a relatively large, "blob-like" pattern for the small scales
  • in both hemisphere's midlatitudes, there are negative patches
  • the higher magnitude negative patches appear to line up with positive values in the large-scale field --> small scales largely dampen the full field (by reducing the positive magnitude)
  • some lower magnitude negative patches align with negative values in the large-scale field --> small scales are reinforcing the full field (by increasing the negative magnitude)
  • overall pattern looks like waves/storm tracks across the globe
• coare3p6 shows both positive and negative impacts of the small scales
  • large magnitudes in the northern hemisphere midlatitudes that are a combination of positive and negative values
  • the positive small-scale values in the northern hemisphere roughly align with the negative patches in the large-scale field --> small scales reinforce the negative patches in the full field
  • in the Southern Hemisphere, there is not a clear pattern between the sign of the small scales and the large scale field
• andreas shows largely positive values of the small scales that align closely with the mostly negative patches in the large-scale field --> the small scales are damping the full field (by reducing the negative magnitude)
  • similarly, we see negative values of the small scales that align with the positive patches in the large-scale field --> small scales dampen the full field (by reducing the positive magnitude)

These observations in general do not match with our previous findings in #32 that the small scales universally reinforce the full field. I.e. the time average is not the same as the individual daily snapshots.

Note that these movies are run for 50 days starting on Jan. 1st. Thus the northern hemisphere is in winter and the southern is in summer.

[jbusecke]

Wow 🤯. The difference is really striking! This movie is really helpful.

These observations in general do not match with our previous findings in https://github.com/ocean-transport/scale-aware-air-sea/issues/32 that the small scales universally reinforce the full field. I.e. the time average is not the same as the individual daily snapshots.

I would agree with this, and even go as far as to say that this immediately qualifies as a new finding: Air sea fluxes due to small scales are very sensitive to the algorithm used. I think I will now prioritize getting results from the NCAR data, to confirm that this shows similar differences.

But I think it is not all lost of our old hypothesis: I still see mostly reinforcement of the large scale flux in the western boundary regions in all algos except andreas, just with very different amplitude. I have the hope that by degrading only wind/sst we will be able to find out what is causing these more persistent features (presumably the ocean).

[paigem]

Latent heat flux

Top row colorbar: -170 to 170 Bottom row colorbar: -30 to 30

https://user-images.githubusercontent.com/26591824/196718708-a993aab3-1c62-4c95-a1ff-7277ec0b3da9.mp4

Preliminary observations

• As for qh, ql shows that the small scales have the smallest magnitude (by a noticeable margin) for the ecmwf algorithm.
• Both ecmwf and NCAR appear to have mostly negative small scales that align with negative regions in the large scale patterns --> small scales are reinforcing the full field
• coare3p6 shows a pattern of both positive and negative values for the small scales
  • the positive small scales align with the large negative magnitude patches in the large-scale field --> the small scales are damping the full field in those regions
• andreas shows nearly all positive small scales --> the small scales are damping the full field globally

I think these results agree with your claim:

Air sea fluxes due to small scales are very sensitive to the algorithm used. However, here we see two methods showing mostly damping by small scales, particularly in the boundary regions. The other two methods show the opposite: the small scales reinforce the full fields.

@jbusecke I agree that running on a different dataset would be a great indicator if the algorithms behave similarly. That will help us determine if there are some algorithms tailored to certain model datasets.