search

NOAA-GFDL / SIS2

NOAA-GFDL's Sea Ice Simulator version 2
Other
16 stars 43 forks source link

CM4_SIS2 crashes with Too many diagnostics. Make MAX_NUM_DIAGNOSTICS bigger! #12

Closed nikizadehgfdl closed 9 years ago

nikizadehgfdl commented 9 years ago

Tried CM4_c96L48_am4c3_2000_sis2 and crashed with:

FATAL from PE 100: SIS_diag_mediator, register_diag_field: Too many diagnostics. Make MAX_NUM_DIAGNOSTICS bigger! EXT

MAX_NUM_DIAGNOSTICS is hard coded to 200. For CM4 models we need to up this. There is no way to increase this limit other than modifying the code.

I upped it to 400 and it worked.

adcroft commented 9 years ago

Really? I thought we'd increased the max # of diagnostics by x2 last month. -A.

On 2/27/15, Niki Zadeh notifications@github.com wrote:

Reopened #12.

Reply to this email directly or view it on GitHub: https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#event-243599019

Dr Alistair Adcroft (Alistair.Adcroft@noaa.gov) Princeton University Tel: (609) 987-5073 NOAA/GFDL, 201 Forrestal Road, Princeton, NJ 08540

nikizadehgfdl commented 9 years ago

This issue surfaces up again for SERIAL runs of CM4 which CWG wants to make. For such runs I had to bump up MAX_NUM_DIAGNOSTICS to 800 in two places: MOM6/framework/MOM_diag_mediator.F90 SIS2/SIS_diag_mediator.F90

underwoo commented 9 years ago

Would it be better to make that a cpp macro?

On Friday, February 27, 2015, Niki Zadeh notifications@github.com wrote:

This issue surfaces up again for SERIAL runs of CM4 which CWG wants to make. For such runs I had to bump up MAX_NUM_DIAGNOSTICS to 800 in two places: MOM6/framework/MOM_diag_mediator.F90 SIS2/SIS_diag_mediator.F90

— Reply to this email directly or view it on GitHub https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76488663 .

Seth Underwood Engility High Technology Services Group

Modeling Services Group GFDL/NOAA/DOC 201 Forrestal Road Princeton, NJ 08540-6649

(609) 452-5847 Office (304) 376-9002 Cell (609) 987-5063 Fax Seth.Underwood@noaa.gov

Hallberg-NOAA commented 9 years ago

Hi,

I am reluctant to change this parameter to a CPP macro, as it would substantially alter how we compiled MOM6 or SIS2 based models. If we did add such a CPP macro, it would be the only one that would be have to change as a result of choices about diagnostics. (We do have some macros that allow us to use static or dynamic memory and symmetric or non-symmetric arrays, but we also have a "standard" version of the include file that will work in all cases.)

The particular reason for these arrays of diagnostic pointers having a fixed (large) size is that the total number of diagnostics is not know at the time the first diagnostic is set up. However, this issue only arises during initialization. My preference would be to change to using dynamic allocation for these arrays of pointers based on a first guess of the number of pointers, where we would make a copy, deallocate and reallocate an expanded block of such pointers if necessary. I would do this in increments of order 100 diagnostic pointers to minimize the number of times the expansion had to occur. A linked lists would not be a good option as it would probably degrade run-time performance.

On Sun, Mar 1, 2015 at 9:55 AM, Seth Underwood notifications@github.com wrote:

Would it be better to make that a cpp macro?

On Friday, February 27, 2015, Niki Zadeh notifications@github.com wrote:

This issue surfaces up again for SERIAL runs of CM4 which CWG wants to make. For such runs I had to bump up MAX_NUM_DIAGNOSTICS to 800 in two places: MOM6/framework/MOM_diag_mediator.F90 SIS2/SIS_diag_mediator.F90

— Reply to this email directly or view it on GitHub < https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76488663

.

Seth Underwood Engility High Technology Services Group

Modeling Services Group GFDL/NOAA/DOC 201 Forrestal Road Princeton, NJ 08540-6649

(609) 452-5847 Office (304) 376-9002 Cell (609) 987-5063 Fax Seth.Underwood@noaa.gov

— Reply to this email directly or view it on GitHub https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76600164 .

Dr. Robert Hallberg Oceanographer and Ocean Group Head NOAA Geophysical Fluid Dynamics Laboratory

NOAA GFDL || Phone: (609) 452-6508 Princeton University Forrestal Campus || Home: (732) 463-1274 201 Forrestal Road || Fax: (609) 987-5063 Princeton, New Jersey 08540-6649 || Email: Robert.Hallberg@noaa.gov

adcroft commented 9 years ago

I'd rather not start using macros that make the compilation of each component different.

-A.

Dr Alistair Adcroft (Alistair.Adcroft@noaa.gov) Princeton University Tel: (609) 987-5073 NOAA/GFDL, 201 Forrestal Road, Princeton, NJ 08540

On Sun, Mar 1, 2015 at 9:55 AM, Seth Underwood notifications@github.com wrote:

Would it be better to make that a cpp macro?

On Friday, February 27, 2015, Niki Zadeh notifications@github.com wrote:

This issue surfaces up again for SERIAL runs of CM4 which CWG wants to make. For such runs I had to bump up MAX_NUM_DIAGNOSTICS to 800 in two places: MOM6/framework/MOM_diag_mediator.F90 SIS2/SIS_diag_mediator.F90

— Reply to this email directly or view it on GitHub < https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76488663

.

Seth Underwood Engility High Technology Services Group

Modeling Services Group GFDL/NOAA/DOC 201 Forrestal Road Princeton, NJ 08540-6649

(609) 452-5847 Office (304) 376-9002 Cell (609) 987-5063 Fax Seth.Underwood@noaa.gov

— Reply to this email directly or view it on GitHub https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76600164 .

adcroft commented 9 years ago

I agree with Bob.

Dr Alistair Adcroft (Alistair.Adcroft@noaa.gov) Princeton University Tel: (609) 987-5073 NOAA/GFDL, 201 Forrestal Road, Princeton, NJ 08540

On Mon, Mar 2, 2015 at 10:11 AM, Alistair Adcroft - NOAA Affiliate < alistair.adcroft@noaa.gov> wrote:

I'd rather not start using macros that make the compilation of each component different.

-A.

Dr Alistair Adcroft (Alistair.Adcroft@noaa.gov) Princeton University Tel: (609) 987-5073 NOAA/GFDL, 201 Forrestal Road, Princeton, NJ 08540

On Sun, Mar 1, 2015 at 9:55 AM, Seth Underwood notifications@github.com wrote:

Would it be better to make that a cpp macro?

On Friday, February 27, 2015, Niki Zadeh notifications@github.com wrote:

This issue surfaces up again for SERIAL runs of CM4 which CWG wants to make. For such runs I had to bump up MAX_NUM_DIAGNOSTICS to 800 in two places: MOM6/framework/MOM_diag_mediator.F90 SIS2/SIS_diag_mediator.F90

— Reply to this email directly or view it on GitHub < https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76488663

.

Seth Underwood Engility High Technology Services Group

Modeling Services Group GFDL/NOAA/DOC 201 Forrestal Road Princeton, NJ 08540-6649

(609) 452-5847 Office (304) 376-9002 Cell (609) 987-5063 Fax Seth.Underwood@noaa.gov

— Reply to this email directly or view it on GitHub https://github.com/CommerceGov/NOAA-GFDL-SIS2/issues/12#issuecomment-76600164 .

adcroft commented 9 years ago

As part of another project in MOM6, @nicjhan is likely going to make the structures that allocate with MAX_NUM_DIAGNOSTICS dynamic. See CommerceGov/NOAA-GFDL-MOM6#171.

Posting this so @nicjhan knows to port relevant mods to SIS2's diag_mediator and then close this ticket.

nichannah commented 9 years ago

Yes CommerceGov/NOAA-GFDL-MOM6#171 will change the diagnostics allocation to be dynamic, currently planning on using increments of 100 as suggested by @Hallberg-NOAA above. There will still be a limit on the number of diagnostics at the FMS layer though, and this value may need to be changed in the input.nml occasionally.