search

GEOS-ESM / GFDL_atmos_cubed_sphere

The GFDL atmos_cubed_sphere dynamical core code
Other
0 stars 2 forks source link

fv_grid_tools MAPL_SHMEM #40

Open wmputman opened 3 years ago

wmputman commented 3 years ago

For DYAMONDv2 I had to update fv_grid_tools to use SHMEM, unfortunately I seem to have broken it for non SHMEM runs.

https://github.com/GEOS-ESM/GFDL_atmos_cubed_sphere/blob/feature/wmputman/DevDYAMONDv2_Merge03nhGF2020/tools/fv_grid_tools.F90

tclune commented 3 years ago

Took a quick look at that link just now. Other than some print statements and some USE statements, it does not look like that file actually exercises any SHMEM functionality. Is this the broken version?

wmputman commented 3 years ago

Look in the 'create_grid' routine, you'll see the allocates

tclune commented 3 years ago

Ah. I had though SHMEM was in all the procedure names. sorry. Looking

atrayano commented 3 years ago

@wmputman I'll take a look and try to fix it after lunch

atrayano commented 3 years ago

@wmputman I cannot see anything obviously wrong with the file tools/fv_grid_tools.F90. The allocations (and deallocations) are done correctly for both Shmem and non-shmem cases. The code though is somewhat different on the branch you pointed us to, compared, let say to feature/wputman/dyamond_v2. For example, this branch calls create_grid, the other does not and instead inlines most of it, but even these lines show some differences (like one does not call get_symmetry)

The shmem related changes are easy to implement (allocation/deallocation of 3 global arrays, and some extra care of who is writing to these), and I prefer to start from a known "good" version, and implement them on top. Could you point us to a "good" starting branch (for this file only)?

wmputman commented 3 years ago

I suggest we start from: feature/wmputman/DevDYAMONDv2_Merge03nhGF2020

atrayano commented 3 years ago

@wmputman this is the branch that already has the shmem logic. I am looking at the version of tools/fv_grid_tools.F90 prior to the shmem changes