Neutral density calculation example

[julia-neme]

I think people have started using pygamma to calculate neutral density surfaces. Would be great put an example notebook - with the caveat that I think pygamma needs to be installed by the user as it does not come in the conda environments right? At least I think that was the case at the beginning of the year but I'm not up to date.

[navidcy]

I presume you don't mean this package -> https://pypi.org/project/pygamma/ right?

[adele-morrison]

This would be great! Linking the discussion from the Hive including installation issues and example code for pygamma here.

NRI folks, is there some way we can get pygamma included in the conda environment? It seems like there's a lot of users interested in this. I guess the current conda environment is managed by CLEX? What's the plan for next year when there's no more CLEX?

[navidcy]

No, they can't add the package in the conda environment if it's not conda-installable...

[adele-morrison]

In that Hive post I linked @aidanheerdegen said "In some cases CMS have created a conda package in their channel."

[navidcy]

Sounds like good idea but unfortunately if it can't be installed via conda then we might need also in the example detailed installation instructions.

In that Hive post I linked @aidanheerdegen said "In some cases CMS have created a conda package in their channel."

Let's ask them to do that then...! If there is need to do that then let's do it.

I have this memory that there was another package by @geoffstanley? Was that conda-installable? Was it to do the same thing?...

[adele-morrison]

Read the Hive post. :) @geoffstanley recommends pygamma.

[navidcy]

I guess the current conda environment is managed by CLEX? What's the plan for next year when there's no more CLEX?

That's a more general discussion, not restricted to this issue here. But there is a new Centre of Excellence and we are discussing on how/whether the CMS infrastructure (eg the conda envs) would carry on in that Centre...

[navidcy]

Read the Hive post. :) @geoffstanley recommends pygamma.

OK -- sorry for posting prematurely :(

[anton-seaice]

In that Hive post I linked @aidanheerdegen said "In some cases CMS have created a conda package in their channel."

@dsroberts - Is it better/easier/more feasible longer term to put this pip package in the CMS channel or conda-forge ?

[adele-morrison]

That's a more general discussion, not restricted to this issue here. But there is a new Centre of Excellence and we are discussing on how/whether the CMS infrastructure (eg the conda envs) would carry on in that Centre...

The sense I am getting is that ~W21C~ 21st Century Weather Centre is not going to be as inclusive as CLEX was. Will their CMS team still be open to e.g. including new conda packages from people (like me) who may not be affiliated with ~W21C~ 21st Century Weather Centre? I wonder if an NRI-supported conda environment would be better for COSIMA? But yes, we should have this discussion on the Hive, not here... I will make a post.

[dsroberts]

@anton-seaice I think the best course of action long-term is get it on conda-forge. We don't know what is happening with the new centre, nor do we know what's happening with a lot of the the software we currently manage. My expectation is that the COECMS conda channel will not be maintained going forward.

[navidcy]

The sense I am getting is that 21st Century Weather Centre is not going to be as inclusive as CLEX was.

@adele-morrison I'm very keen for this chat but it does go beyond neutral surfaces!

where do you get this sense???! Not from me I hope..

[navidcy]

Why don't we open an issue in the pygamma package urging them to add it in conda-forge?

[navidcy]

Shall I open it? Could somebody point me to the GitHub repo of pygamma?

[julia-neme]

Yeah, I am trying to find it. But I don't think there is one for the package that Geoff pointed to in the hive (https://currents.soest.hawaii.edu/hgstage/pygamma/)

I found this other package: https://github.com/guidov/pygamman and this one https://github.com/geoffstanley/neutralocean

I don't think Geoff's neultraocean does neutral densities. And I'm not quite sure about the pygamman package since its not the one that was recommended in the hive forum. I might try to get Geoff's opinion before proceeding..

[navidcy]

tagging @geoffstanley

[julia-neme]

Are neutral surfaces the same as neutral densities? I.e., can we use a value of 28.3 to define AABW as per literature?

[geoffstanley]

Hello!

Neutral Surfaces are 2D surfaces (depth as a function of latitude and longitude) while Neutral Density is a 3D scalar field (a density value in kg/m3 as a function of latitude, longitude, and depth). An isosurface of Neutral Density is a Neutral Surface.

If you specifically need a 3D neutral density for global (or very large-scale) oceanography and you're ocean dataset has a similar climate to the Levitus climatology from the early 1980's, then currently McDougall and Jackett (1997)'s γⁿ is the one to use. Or, if you specifically need neutral density values for comparison with McDougall and Jackett (1997) values in published literature, then again their γⁿ is what you need. In that case, I believe Eric Firing's pygamma is the best Python wrapper for γⁿ.

Maybe @efiring could upload pygamma to conda-forge, which it sounds like is needed because of particularities of your machines?

Sometimes people think they need 3D neutral density, when in fact 2D neutral surfaces are needed. In this case, neutralocean provides much greater accuracy, isn't tied to the Levitus climatology, is fast, native-Python, and on conda-forge. The omega surfaces from neutralocean can be labelled to with neutral density labels in kg/m3 that are similar (based on the "Veronis density") but not identical to McDougall and Jacket (1997) γⁿ labels.

Eventually, neutralocean will include 3D neutral density, but that is some time off. For the moment, having both pygamma and neutralocean at your fingertips is a good plan!

[access-hive-bot]

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/post-clex-conda-analysis-environments-on-nci/2198/6

[julia-neme]

Thank you Geoff! That is a very clear explanation. In that case for the sake of this Documented Example, I will do the notebook with instructions on how to locally install pygamma, and when/if the package gets added to conda-forge I'll come back and modify accordingly.

[navidcy]

That sounds great @julia-neme!

[efiring]

This weekend @ocefpaf and I did a little updating of https://currents.soest.hawaii.edu/hgstage/pygamma to build and run with Python 3.12 and numpy 2.0, using conda-forge compilers and libraries. We will see how far we can take it over the next few weeks.

[anton-seaice]

Thankyou @efiring :)

[ocefpaf]

This weekend @ocefpaf and I did a little updating of https://currents.soest.hawaii.edu/hgstage/pygamma to build and run with Python 3.12 and numpy 2.0, using conda-forge compilers and libraries. We will see how far we can take it over the next few weeks.

BTW, the draft PR is passing: https://github.com/conda-forge/staged-recipes/pull/26814

Just waiting on a stable release.

[efiring]

Thanks to @ocefpaf, the pygamma wrapper of the original neutral density Fortran code by David Jackett is now available on conda-forge for MacOS and Linux. A Windows build likely will follow, but it's not working yet. The package is slightly renamed, with "_n" added to avoid conflict with the "pygamma" package on PyPI:

conda install -c conda-forge pygamma_n

[julia-neme]

That's great! Thank you both very much. @anton-seaice @dsroberts , could we add the package to the current/next conda environments?

[navidcy]

@efiring thanks a lot! How did you hear about this issue -- I'm wondering!

Also thanks for the clarification in respect to the pygamma package! I was wondering in the beginning why a magnetic resonance library computes neutral densities but everyone seems on board with it so I thought that somehow they are related! Turns out they are not -- mere coincidence!

[navidcy]

How did you hear about this issue -- I'm wondering!

Oh... I see that @efiring was tagged here.

[geoffstanley]

Thanks very much @efiring and @ocefpaf ! It's great to have this on conda-forge. Much appreciated.

[dsroberts]

@julia-neme yes, for sure. I'm on leave until the end of next week, I'll do this first thing when I'm back.

[navidcy]

@dsroberts is this included in the conda-env now?

[dsroberts]

@navidcy it is now. Sorry, should have been more precise, my first day back from leave is today.

[navidcy]

no worries at all!