Subscript #1 of the array FARRAYPTR1 has value 1503 which is greater than the upper bound of 1502

[platipodium]

On Strand using 2 PET for SCHISM in CoastalApp I get

forrtl: severe (408): fort: (2): Subscript #1 of the array FARRAYPTR1 has value 1503 which is greater than the upper bound of 1502
2:
2: Image              PC                Routine            Line        Source
2: NEMS-schism_atmes  00000000007CEE6F  Unknown               Unknown  Unknown
2: NEMS-schism_atmes  000000000052B2B2  schism_nuopc_cap_         427  schism_nuopc_cap.F90

[josephzhang8]

This might be related to how the compiler interprets the pointer arrays?

-Joseph

Y. Joseph Zhang Web: schism.wiki Office: 804 684 7466

From: Carsten Lemmen @.> Sent: Wednesday, February 23, 2022 4:05 PM To: schism-dev/schism-esmf @.> Cc: Subscribed @.***> Subject: [schism-dev/schism-esmf] Subscript #1 of the array FARRAYPTR1 has value 1503 which is greater than the upper bound of 1502 (Issue #5)

[EXTERNAL to VIMS received message]

On Strand using 2 PET for SCHISM in CoastalApp I get

forrtl: severe (408): fort: (2): Subscript #1 of the array FARRAYPTR1 has value 1503 which is greater than the upper bound of 1502

2:

2: Image PC Routine Line Source

2: NEMS-schism_atmes 00000000007CEE6F Unknown Unknown Unknown

2: NEMS-schism_atmes 000000000052B2B2 schism_nuopccap 427 schism_nuopc_cap.F90

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F5&data=04%7C01%7Cyjzhang%40vims.edu%7C4a1060692b744ab19e8c08d9f71025f4%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812471002078137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=fhgpY0L8Ig%2FMp8upHO1%2BkqldemTMa5Nql%2FYTcDRP%2F0o%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ56JIWIVHVZBAMX6MDU4VDXRANCNFSM5PFQJJAA&data=04%7C01%7Cyjzhang%40vims.edu%7C4a1060692b744ab19e8c08d9f71025f4%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812471002078137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2Fvsq2%2BCOPfkqsVkInLv0laNR%2FujP%2FVyOpCzPTB2h5Yo%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cyjzhang%40vims.edu%7C4a1060692b744ab19e8c08d9f71025f4%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812471002078137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aZbae2j%2FptjKkEG3JVobwa0KRDl7vd3Yec0KuVyyjQ8%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cyjzhang%40vims.edu%7C4a1060692b744ab19e8c08d9f71025f4%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812471002078137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A4MuReqXdgwFPi7buQvhpRDkRNcKJOn5AHeGM4idP8s%3D&reserved=0. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

[platipodium]

I fear this is more related to the ownership of nodes and np.

[nodalDistgrid] A Distgrid describing the distribution of the nodes across the PETs. Note that on each PET the distgrid will only contain entries for nodes owned by that PET. This is the DistGrid that would be used to construct the Array in a Field that is constructed on mesh.

So the size of the distgrid is < np, and we call in that line

farrayPtr1(1:np), where farrayPtr1 is only looking at the size of the distgrid. So back to coding ...

[platipodium]

call ESMF_MeshGet(mesh2d, nodalDistgrid=nodalDistgrid, rc=rc)
  _SCHISM_LOG_AND_FINALIZE_ON_ERROR_(rc)

  array = ESMF_ArrayCreate(nodalDistgrid, typekind=ESMF_TYPEKIND_R8, &
    name="inst_zonal_wind_height10m", rc=localrc)

One idea is to use elementDistgrid to get the elements, then use the mapping from elements to nodes to get to np ... or use a different construction of the array above

[josephzhang8]

I think ESMF should handle that with the info from nodeOwners?

There should be a way in ESMF to query the dim of each distributed arrays, then we can simply adjust windx2() etc to match that, b/c inside SCHISM I've added halo/interface exchanges for nodes.

-Joseph

Y. Joseph Zhang Web: schism.wiki Office: 804 684 7466

From: Carsten Lemmen @.> Sent: Wednesday, February 23, 2022 4:26 PM To: schism-dev/schism-esmf @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism-esmf] Subscript #1 of the array FARRAYPTR1 has value 1503 which is greater than the upper bound of 1502 (Issue #5)

[EXTERNAL to VIMS received message]

call ESMF_MeshGet(mesh2d, nodalDistgrid=nodalDistgrid, rc=rc)

_SCHISM_LOG_AND_FINALIZE_ONERROR(rc)

array = ESMF_ArrayCreate(nodalDistgrid, typekind=ESMF_TYPEKIND_R8, &

name="inst_zonal_wind_height10m", rc=localrc)

One idea is to use elementDistgrid to get the elements, then use the mapping from elements to nodes to get to np ... or use a different construction of the array above

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F5%23issuecomment-1049230484&data=04%7C01%7Cyjzhang%40vims.edu%7C1dc04a7af8a942b6f7ea08d9f71311f1%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812483561383904%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=S4%2Fj46aNcyFTK0FwZXdxqLK22wn50bI30cNJnKYOKak%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZYTWGMTTUP6D7BC223U4VGF7ANCNFSM5PFQJJAA&data=04%7C01%7Cyjzhang%40vims.edu%7C1dc04a7af8a942b6f7ea08d9f71311f1%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812483561383904%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zuteBPMEZS1G%2FQ9nDnC4xVxZcvA3H%2FjCnpkVXoUgAtc%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cyjzhang%40vims.edu%7C1dc04a7af8a942b6f7ea08d9f71311f1%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812483561383904%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GjcRfOMsXhWsOkHGv9BdlbqfcPNeW7LhQicb%2BW5QCro%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cyjzhang%40vims.edu%7C1dc04a7af8a942b6f7ea08d9f71311f1%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637812483561383904%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ot4RYt%2BI1JpfILXSja3AZL39RAvNyj%2BEp3iWhQ7OflU%3D&reserved=0. You are receiving this because you commented.Message ID: @.**@.>>

[platipodium]

We might want to try:

! Private name; call using ESMF_ArrayCreate()
   function ESMF_ArrayCreateFrmPtr<rank><type><kind>(distgrid, farrayPtr,

since we can specify

distgrid ESMF_DistGrid object that describes how the array is decomposed and distributed over DEs. The dim- Count of distgrid must be smaller or equal to the rank of farrayPtr.

[platipodium]

I think ESMF should handle that with the info from nodeOwners? There should be a way in ESMF to query the dim of each distributed arrays, then we can simply adjust windx2() etc to match that, b/c inside SCHISM I've added halo/interface exchanges for nodes. …

Ok, so we need to give to windx2 the dimension of the owned nodes ... what is that in SCHISM?

[josephzhang8]

There is no such thing inside SCHISM - not needed at all. It's best to get that from ESMF.

[platipodium]

We can generate that number in addMEsh

[josephzhang8]

But an additional question arises: what if EMSF internally re-number the owned nodes? That'd screw up everything...

[platipodium]

We give it coordinates in that order, and all other information, so why would you expect ESMF to reorder? I suggest to rely on this for now until we run into a problem that hints to an ordering issue.

[josephzhang8]

Well u said ESMF may drop some nodes not owned by a MPI process, so there has to be renumbering. Can u check from the error what is local np? U can find that from outputs/local_toglobal*

[josephzhang8]

Best confirm with Bob?

[platipodium]

A hotfix replacing 1:np with 1:ubound(farrayPtr1,1) reintroduces the problem with holes ...

[platipodium]

This is really again https://github.com/schism-dev/schism-esmf/issues/3