search

JMMP-Group / CO_AMM15

Coastal Ocean (CO) configuration of the Atlantic Margin Model (1.5 km resolution)
4 stars 1 forks source link

Develop a safe AMM15 version: CO9p0_AMM15 at 4.0.x #13

Closed jpolton closed 3 years ago

jpolton commented 4 years ago

Make a base, or reference, NEMOv4 config that is as close as possible to JennyG v3.6 configuration.

This is so that: 1) there is a back-up plan for March incase some of the enhancements don't come off in time. 2) enhancements can be added incrementally

MO would call this "P-zero"

UPDATED a bit.

endaodea commented 4 years ago

Starting at 4.0.2 will update this if Dave Storkey advises that GO suites are say at 4.0.4

jpolton commented 4 years ago

Starting at 4.0.2 will update this if Dave Storkey advises that GO suites are say at 4.0.4

@endaodea following Catherine and Dave's email I understand that we will end up using v4.0.3 or (more probably) v4.0.4. Is that correct?

endaodea commented 4 years ago

I guess so. I'll look to make the base version 4.0.4 then and work from there. Hopefully not much will have changed form 4.0.2 to 4.0.4 in terms of namelist and new additional bugs!

jpolton commented 3 years ago

Update: targeting 4.0.3 to align with GO (ECMWF founds issues with SI3 at 4.0.4)

endaodea commented 3 years ago

Appears to be some further movement on what version GO are going to use, will update further this afternoon after meeting with GO team members.

jpolton commented 3 years ago

FYI @endaodea Here are snippets from the exchange I had with Sarah Keeley @ECMWF, it looks like a whispering gallery regarding what is good and not so good about 4.0.3 / 4.0.4 . I am not sure this is the same message you passed from Catherine. Note this unofficial email channel is for context rather than hard evidence to make decision on. Oldest is at the bottom.

From: Sarah Keeley Sarah.Keeley@ecmwf.int Date: Tuesday, 24 November 2020 at 10:27 To: "Polton, Jeffrey" jelt@noc.ac.uk

it was me. 4.0.4 has very different behaviour in SI3 in terms of errors so we will not get it working in time... 4.0.3 has interesting rare failures when we couple it...stand alone the model seems to behave as long as we don't switch on aEVP in the ice model. Unfortunately there are very few Sarahs at ECMWF.

From: Polton, Jeffrey jelt@noc.ac.uk Sent: 24 November 2020 09:51 To: Sarah Keeley Sarah.Keeley@ecmwf.int ... Regarding v4.0.3 the Met Office are using this for the Global Ocean series. Catherine said: “… following tests from Sarah at ECMWF, we have decided that we will chill GO8 at 4.0.3. Sarah has done some test with 4.0.4 and despite it was only supposed to have bugfixes compared to 4.0.3 she found that with 4.0.4 the performance was poor…” I wonder if that was you… if it wasn’t then their might be help from that effort? If it was you, then it looks like a mix up in communication about 4.0.4 and 4.0.3…

Jeff

From: Sarah Keeley Sarah.Keeley@ecmwf.int Date: Tuesday, 24 November 2020 at 09:40

Hi Jeff, ...Trying to hand over a semi-frozen version of NEMO4 by the middle of next month and since moving to 4.0.3 we are suddenly having unreproducible & reproducible failures...

endaodea commented 3 years ago

@jpolton @davbyr @jdha HI Guys, so the outcome of today's meeting is that the GO guys are of the opinion that there are basically no changes to the ocean between 4.02-4.04 (at least for the global ocean) and are going to try to set up a GO version at 4.0.4 and try it out. They say that will take the order of a week.

In the mean time I can set up a CO9 package branch based on 4.0.2 the exercise wil be pretty similar for 4.0.4. Messing around with git+sphinx today with a view to documenting that process and learning git instead of svn.

I'll have to do some sanity check runs repeating what I have already at 4.0.2 if it checks out then we can rubber stamps the 4.0.4 co9 package branch as the official p0.

Once it's there we can branch off it for our various trial experiments, e.g. including Nico's code, and if trial of that works out better, merge back in to the package branch at a later date for a p1 p2 etc.

jpolton commented 3 years ago

@endaodea said: "It looks like we will be going to 4.0.4 as Catherine has just created a GO package branch at 4.0.4 so I’ll attempt to migrate the CO package branch from 4.0.2 to 4.0.4 and recheck that all is as expected for the p0 suite."

Early investigation reveals possible issues with open boundaries.

jpolton commented 3 years ago

@endaodea : A boundary bug has been squashed and a ticket has been raised incase a better solution exists. Now doing a longer run to check diff with 4.0.2

endaodea commented 3 years ago

1 year of 4040 done to compare with 402 Tides look pretty much identical

SST SBT SSS SBS look very similar on shelf, some noise difference in filaments and eddies off shelf but nothing on a sacle that looks to be concerned about.

https://user-images.githubusercontent.com/40431846/102950354-63953880-44c2-11eb-8d8f-3daab9f5f286.mp4

Will migrate the 404 suite to monsoon next. 402 basic suite on monsoon also exits with archiving (see related issue) PHASE_1990_AMM15_M2 1990_AMM15_M2 For further plots see: https://drive.google.com/drive/folders/14plgl5kJMIEe3mSvzTxek7Kn4IiEek2C?usp=sharing

Once monsoon suite 404 in place will close this issue

endaodea commented 3 years ago

Created 404 Config branch for CO9p0 with 404 as the base revision branched orfm GO8 branch @404

See related ticket:

https://code.metoffice.gov.uk/trac/rmed/ticket/138#ticket

Also creating Monsoon rose suite:u-ca907
to extract build and run this package branch

Suite Extracts, builds and runs, also has the same slow initialization problem on monsoon that 402 has On internal cray initialization is much quicker. Maybe IO issue?

nominal data under::crum/xu-ca907/nonum_field.nc.file/ checks out as exactly the same as a run on internal cray.

jpolton commented 3 years ago

@endaodea Good job. Let's close this item and put any new emergent issues in new tickets