search

pwolfram / MPAS-Model

Repository for MPAS models and shared framework releases.
Other
1 stars 1 forks source link

Adds scalar interpolation capability for LIGHT tracers #21

Closed pwolfram closed 5 years ago

pwolfram commented 5 years ago

Adds capability to do horizontal interpolation for LIGHT tracers. Particles nearest to a boundary vertex will use standard routine and just return cell-based scalar value.

pwolfram commented 5 years ago

@bradyrx, note, the commit history from your branch bgcParticles will need updating once I merge this code into particlePassiveFloatVerticalTreatmentFix.

pwolfram commented 5 years ago

@bradyrx, additional testing is needed. Something you could test in the meantime is whether this works for your current case (e.g., doesn't crash). If it crashes, we can figure out that issue simultaneously while we figure out any accuracy bugs.

bradyrx commented 5 years ago

@pwolfram, I'll rebase bgcParticles onto here and run a quick BGC-active test run. Also, the invitation link for collaboration on this repo didn't work for some reason. Please re-invite if you meant to send that out.

bradyrx commented 5 years ago

@pwolfram, forgot that we were discussing this here, so I'll duplicate my email I just sent for documentation here.

I ran a 60to30 case with BGC on (as well as temp, salinity, DIC, ALK sensors on) as a 5-day smoke test.

The run breaks due to an issue with summing barycentric weights:

Run directory is here: /lustre/scratch3/turquoise/rileybrady/ACME/cases/GMPAS-OECO-ODMS-IAF.T62_oEC60to30v3.grizzly.4096o.512i.5ParticleLayers/run

pwolfram commented 5 years ago

@bradyrx, this is related to the known bug I need to fix. The problem is that the right vertex isn't being selected and a more complicated or brute force selection method is needed to get the "anchor" vertex for the interpolation. I think I've sorted this through conceptually but need to make the time for the coding and have started. I've blocked out a huge chunk of time Monday Dec 3 to devote to this issue. Will this work?

bradyrx commented 5 years ago

@pwolfram, sounds good to me. I should have some time early next week to run tests, rebase, etc.

pwolfram commented 5 years ago

Testing

Verified that particle is spatially interpolating for surface layer:

image

Salinity on boundary cell was also tested and particle and cell value of salinity lines were identical (as expected). Code testing was on gfortran and still needs tested using intel compiler in full E3SM case.

pwolfram commented 5 years ago

Regression testing

Testing against ocean/develop commit fd71e64529b7231d5d13882126e996849efd0d43

┌─[pwolfram][weaklynonlinear][~/Documents/MPAS_development/MPAS-Model_LIGHT_fix_freefloating/testing_and_setup/compass/tmp_bb3a6b322bfd2936a08e3e6b64ed3b5fcf815b45][10:28][±][branch_LIGHT_adds_scalar_interp ✗]                               
└─▪ ./nightly_ocean_test_suite.py                                                                                                                                                                                                               
 ** Running case Periodic Planar 20km - LIGHT particle general test                                                                                                                                                                             
   ** FAIL (See case_outputs/Periodic_Planar_20km_-_LIGHT_particle_general_test for more information)                                                                                                                                           
 ** Running case Periodic Planar 20km - LIGHT particle region reset test                                                                                                                                                                        
   ** FAIL (See case_outputs/Periodic_Planar_20km_-_LIGHT_particle_region_reset_test for more information)                                                                                                                                      
 ** Running case Periodic Planar 20km - LIGHT particle time reset test                                                                                                                                                                          
^[^[   ** FAIL (See case_outputs/Periodic_Planar_20km_-_LIGHT_particle_time_reset_test for more information)                                                                                                                                    
 ** Running case SOMA 32km - LIGHT particle time general test                                                                                                                                                                                   
      PASS                                                                                                                                                                                                                                      
 ** Running case ZISO 20km - LIGHT particle time general test                                                                                                                                                                                   
      PASS                                                                                                                                                                                                                                      
TEST RUNTIMES:                                                                                                                                                                                                                                  
00:19 Periodic_Planar_20km_-_LIGHT_particle_general_test                                                                                                                                                                                        
00:18 Periodic_Planar_20km_-_LIGHT_particle_region_reset_test                                                                                                                                                                                   
00:31 Periodic_Planar_20km_-_LIGHT_particle_time_reset_test                                                                                                                                                                                     
01:24 SOMA_32km_-_LIGHT_particle_time_general_test                                                                                                                                                                                              
00:43 ZISO_20km_-_LIGHT_particle_time_general_test                                                                                                                                                                                              
Total runtime 03:15

Failure is due to zLevelParticle not being updated correctly in the previous version of the code, e.g., for surface constrained particles. This field was previously 0, but is no a number corresonding to zMid, which is an improvement in accuracy of the code but one that breaks the regression testing. The offending code in ocean/develop is at https://github.com/MPAS-Dev/MPAS-Model/blob/ocean/develop/src/core_ocean/analysis_members/mpas_ocn_lagrangian_particle_tracking.F#L700-L722-- note if verticalTreatment isn't 4, no assignment of zLevelParticle is made as it should be made.

The key thing is that the real world cases pass, which correspond to previous uses of LIGHT.

Double-checking, spot testing

This was also verified by using a spun-up version of SOMA corresponding to previous testing (not in regression suite) and particle movement was bfb following the refractors herein against fd71e64529b7231d5d13882126e996849efd0d43.

bradyrx commented 5 years ago

@pwolfram, the most recent commit is still segfaulting on grizzly for a BGC run with sensors turned on.

I ran a 60to30 case with BGC and all sensors turned on (T, S, ALK, DIC):

/lustre/scratch3/turquoise/rileybrady/ACME/cases/GMPAS-OECO-ODMS-IAF.T62_oEC60to30v3.grizzly.512o.128i.5ParticleLayers.Ton.Son.DICon.ALKon/run

However, note that in ocn.log.16017775.181213-120421, the sampling seems to proceed just fine for T and S, and the error happens within the interpolation call for DIC:

I suspect then the issue lies within code chunks like lines 830-852 in mpas_ocn_lagrangian_particle_tracking.F.

Note that the run goes smoothly with no sensors turned on. I've queued a run with just T and S sensors to see if what I suspect is true.

bradyrx commented 5 years ago

@pwolfram, note that a 60to30 physics-only case with T & S sensors ran fine on grizzly. So this is a BGC-sensor interpolation problem.

Run directory:

/lustre/scratch3/turquoise/rileybrady/ACME/cases/GMPAS-IAF.T62_oEC60to30v3.grizzly.512o.128i.5ParticleLayers.Ton.Son/run/

pwolfram commented 5 years ago

Verified that merge commit e4f2dc54 into particlePassiveFloatVerticalTreatmentFix was BFB against ocean/develop and BFB across processor decompositions and pushed the merge.

Will perform any additional fixes needed at https://github.com/MPAS-Dev/MPAS-Model/pull/56 in order to prepare for production runs.