ufs-community / ufs-weather-model

UFS Weather Model
Other
141 stars 248 forks source link

submodules do not update with latest develop code #194

Closed JessicaMeixner-NOAA closed 4 years ago

JessicaMeixner-NOAA commented 4 years ago

Description

'ccpp/framework' returns fatal: reference is not a tree when updating submodules for ufs-weather-model commit fd5c310f306c001be12877ecd99f0026b38a8923

To Reproduce:

What compilers/machines are you seeing this with? WCOSS-Dell (I assume all) Give explicit steps to reproduce the behavior. git clone https://github.com/ufs-community/ufs-weather-model cd ufs-weather-model git submodule update --init --recursive

Output


$ git submodule update --init --recursive 
Submodule 'FMS' (https://github.com/NOAA-GFDL/FMS) registered for path 'FMS'
Submodule 'FV3' (https://github.com/NOAA-EMC/fv3atm) registered for path 'FV3'
Submodule 'NEMS' (https://github.com/NOAA-EMC/NEMS) registered for path 'NEMS'
Submodule 'WW3' (https://github.com/NOAA-EMC/WW3) registered for path 'WW3'
Submodule 'stochastic_physics' (https://github.com/noaa-psd/stochastic_physics) registered for path 'stochastic_physics'
Cloning into 'FMS'...
remote: Enumerating objects: 15444, done.
remote: Total 15444 (delta 0), reused 0 (delta 0), pack-reused 15444
Receiving objects: 100% (15444/15444), 17.56 MiB | 16.04 MiB/s, done.
Resolving deltas: 100% (10946/10946), done.
Submodule path 'FMS': checked out 'f61416fef691d9ba39a40df1ce72aa574f54c390'
Cloning into 'FV3'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 4149 (delta 0), reused 2 (delta 0), pack-reused 4138
Receiving objects: 100% (4149/4149), 7.58 MiB | 10.43 MiB/s, done.
Resolving deltas: 100% (2956/2956), done.
Submodule path 'FV3': checked out '0b2d3ec4ad172d718abdb05493e9fe6e5e9c272b'
Submodule 'atmos_cubed_sphere' (https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere) registered for path 'atmos_cubed_sphere'
Submodule 'ccpp/framework' (https://github.com/NCAR/ccpp-framework) registered for path 'ccpp/framework'
Submodule 'ccpp/physics' (https://github.com/NCAR/ccpp-physics) registered for path 'ccpp/physics'
Cloning into 'atmos_cubed_sphere'...
remote: Enumerating objects: 2784, done.
remote: Total 2784 (delta 0), reused 0 (delta 0), pack-reused 2784
Receiving objects: 100% (2784/2784), 2.88 MiB | 572.00 KiB/s, done.
Resolving deltas: 100% (1864/1864), done.
Submodule path 'FV3/atmos_cubed_sphere': checked out '8b59ebc039dafe1c20ed6dd21cb38ca564852b98'
Cloning into 'ccpp/framework'...
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 5577 (delta 13), reused 27 (delta 11), pack-reused 5533
Receiving objects: 100% (5577/5577), 10.96 MiB | 13.79 MiB/s, done.
Resolving deltas: 100% (3834/3834), done.
fatal: reference is not a tree: f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76```
junwang-noaa commented 4 years ago

Dom is going to fix this in his PR #191 https://github.com/ufs-community/ufs-weather-model/pull/191 (fv3 PR 164 https://github.com/NOAA-EMC/fv3atm/pull/164). We plan to get it committed today.

On Mon, Aug 31, 2020 at 2:22 PM Jessica Meixner notifications@github.com wrote:

Description

'ccpp/framework' returns fatal: reference is not a tree when updating submodules for ufs-weather-model commit fd5c310 https://github.com/ufs-community/ufs-weather-model/commit/fd5c310f306c001be12877ecd99f0026b38a8923 To Reproduce:

What compilers/machines are you seeing this with? WCOSS-Dell (I assume all) Give explicit steps to reproduce the behavior. git clone https://github.com/ufs-community/ufs-weather-model cd ufs-weather-model git submodule update --init --recursive Output

$ git submodule update --init --recursive Submodule 'FMS' (https://github.com/NOAA-GFDL/FMS) registered for path 'FMS' Submodule 'FV3' (https://github.com/NOAA-EMC/fv3atm) registered for path 'FV3' Submodule 'NEMS' (https://github.com/NOAA-EMC/NEMS) registered for path 'NEMS' Submodule 'WW3' (https://github.com/NOAA-EMC/WW3) registered for path 'WW3' Submodule 'stochastic_physics' (https://github.com/noaa-psd/stochastic_physics) registered for path 'stochastic_physics' Cloning into 'FMS'... remote: Enumerating objects: 15444, done. remote: Total 15444 (delta 0), reused 0 (delta 0), pack-reused 15444 Receiving objects: 100% (15444/15444), 17.56 MiB | 16.04 MiB/s, done. Resolving deltas: 100% (10946/10946), done. Submodule path 'FMS': checked out 'f61416fef691d9ba39a40df1ce72aa574f54c390' Cloning into 'FV3'... remote: Enumerating objects: 11, done. remote: Counting objects: 100% (11/11), done. remote: Compressing objects: 100% (11/11), done. remote: Total 4149 (delta 0), reused 2 (delta 0), pack-reused 4138 Receiving objects: 100% (4149/4149), 7.58 MiB | 10.43 MiB/s, done. Resolving deltas: 100% (2956/2956), done. Submodule path 'FV3': checked out '0b2d3ec4ad172d718abdb05493e9fe6e5e9c272b' Submodule 'atmos_cubed_sphere' (https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere) registered for path 'atmos_cubed_sphere' Submodule 'ccpp/framework' (https://github.com/NCAR/ccpp-framework) registered for path 'ccpp/framework' Submodule 'ccpp/physics' (https://github.com/NCAR/ccpp-physics) registered for path 'ccpp/physics' Cloning into 'atmos_cubed_sphere'... remote: Enumerating objects: 2784, done. remote: Total 2784 (delta 0), reused 0 (delta 0), pack-reused 2784 Receiving objects: 100% (2784/2784), 2.88 MiB | 572.00 KiB/s, done. Resolving deltas: 100% (1864/1864), done. Submodule path 'FV3/atmos_cubed_sphere': checked out '8b59ebc039dafe1c20ed6dd21cb38ca564852b98' Cloning into 'ccpp/framework'... remote: Enumerating objects: 44, done. remote: Counting objects: 100% (44/44), done. remote: Compressing objects: 100% (33/33), done. remote: Total 5577 (delta 13), reused 27 (delta 11), pack-reused 5533 Receiving objects: 100% (5577/5577), 10.96 MiB | 13.79 MiB/s, done. Resolving deltas: 100% (3834/3834), done. fatal: reference is not a tree: f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76```

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/issues/194, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI7D6TNDGITCQDRFBE5GVF3SDPS6TANCNFSM4QQZUAOQ .

DusanJovic-NOAA commented 4 years ago

Can not reproduce:

$ git clone https://github.com/ufs-community/ufs-weather-model
Cloning into 'ufs-weather-model'...
remote: Enumerating objects: 99, done.
remote: Counting objects: 100% (99/99), done.
remote: Compressing objects: 100% (94/94), done.
remote: Total 9959 (delta 26), reused 7 (delta 5), pack-reused 9860
Receiving objects: 100% (9959/9959), 54.23 MiB | 22.78 MiB/s, done.
Resolving deltas: 100% (7721/7721), done.

$ cd ufs-weather-model

$ git submodule update --init --recursive
Submodule 'FMS' (https://github.com/NOAA-GFDL/FMS) registered for path 'FMS'
Submodule 'FV3' (https://github.com/NOAA-EMC/fv3atm) registered for path 'FV3'
Submodule 'NEMS' (https://github.com/NOAA-EMC/NEMS) registered for path 'NEMS'
Submodule 'WW3' (https://github.com/NOAA-EMC/WW3) registered for path 'WW3'
Submodule 'stochastic_physics' (https://github.com/noaa-psd/stochastic_physics) registered for path 'stochastic_physics'
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FMS'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FV3'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/NEMS'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/WW3'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/stochastic_physics'...
Submodule path 'FMS': checked out 'f61416fef691d9ba39a40df1ce72aa574f54c390'
Submodule path 'FV3': checked out '0b2d3ec4ad172d718abdb05493e9fe6e5e9c272b'
Submodule 'atmos_cubed_sphere' (https://github.com/NOAA-EMC/GFDL_atmos_cubed_sphere) registered for path 'FV3/atmos_cubed_sphere'
Submodule 'ccpp/framework' (https://github.com/NCAR/ccpp-framework) registered for path 'FV3/ccpp/framework'
Submodule 'ccpp/physics' (https://github.com/NCAR/ccpp-physics) registered for path 'FV3/ccpp/physics'
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FV3/atmos_cubed_sphere'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FV3/ccpp/framework'...
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FV3/ccpp/physics'...
Submodule path 'FV3/atmos_cubed_sphere': checked out '8b59ebc039dafe1c20ed6dd21cb38ca564852b98'
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 6 (delta 3), reused 5 (delta 3), pack-reused 1
Unpacking objects: 100% (6/6), done.
From https://github.com/NCAR/ccpp-framework
 * branch            f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76 -> FETCH_HEAD
Submodule path 'FV3/ccpp/framework': checked out 'f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76'
Submodule path 'FV3/ccpp/physics': checked out '4c17ff716a92d8ac0261d6cc2365bbd7a752b74a'
Submodule 'physics/rte-rrtmgp' (https://github.com/RobertPincus/rte-rrtmgp) registered for path 'FV3/ccpp/physics/physics/rte-rrtmgp'
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/FV3/ccpp/physics/physics/rte-rrtmgp'...
Submodule path 'FV3/ccpp/physics/physics/rte-rrtmgp': checked out '6ee0b62c1ac6204a89a4e922382b611c16dd5fa7'
Submodule path 'NEMS': checked out '5a781b5692d21e9881e53a404343dda753a44b05'
Submodule 'tests/produtil/NCEPLIBS-pyprodutil' (https://github.com/NOAA-EMC/NCEPLIBS-pyprodutil) registered for path 'NEMS/tests/produtil/NCEPLIBS-pyprodutil'
Cloning into '/gpfs/dell2/emc/modeling/noscrub/Dusan.Jovic/ufs/develop/ufs-weather-model/NEMS/tests/produtil/NCEPLIBS-pyprodutil'...
Submodule path 'NEMS/tests/produtil/NCEPLIBS-pyprodutil': checked out 'ca171b95095db4fcd0fc7b01c23d073d90becd99'
Submodule path 'WW3': checked out '19f3126c832b26ae4c5b27bca6c1dcc676755602'
Submodule path 'stochastic_physics': checked out '370d8e6b50aaa52977f992d1574e13b998e2e974'

$ git submodule status --recursive
 f61416fef691d9ba39a40df1ce72aa574f54c390 FMS (2019.01.03)
 0b2d3ec4ad172d718abdb05493e9fe6e5e9c272b FV3 (heads/develop)
 8b59ebc039dafe1c20ed6dd21cb38ca564852b98 FV3/atmos_cubed_sphere (heads/dev/emc)
 f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76 FV3/ccpp/framework (ccpp_transition_to_vlab_master_20190705-204-gf5d4cd2)
 4c17ff716a92d8ac0261d6cc2365bbd7a752b74a FV3/ccpp/physics (ccpp_transition_to_vlab_master_20190705-746-g4c17ff7)
 6ee0b62c1ac6204a89a4e922382b611c16dd5fa7 FV3/ccpp/physics/physics/rte-rrtmgp (1.2.1-15-g6ee0b62)
 5a781b5692d21e9881e53a404343dda753a44b05 NEMS (heads/develop)
 ca171b95095db4fcd0fc7b01c23d073d90becd99 NEMS/tests/produtil/NCEPLIBS-pyprodutil (heads/develop)
 19f3126c832b26ae4c5b27bca6c1dcc676755602 WW3 (6.07.1-36-g19f3126c)
 370d8e6b50aaa52977f992d1574e13b998e2e974 stochastic_physics (ufs-v1.0.0-68-g370d8e6)
JessicaMeixner-NOAA commented 4 years ago

@DusanJovic-NOAA What version of git are you using? On Dell I was using git version 1.8.3.1 and got the error I showed above (I checked out again to confirm), but on hera I am using git version 2.18.0 and get a similar output to what you show.

Thanks for the info @junwang-noaa, I guess this issue can be closed then.

DusanJovic-NOAA commented 4 years ago

@JessicaMeixner-NOAA

$ git --version
git version 2.14.3

$ which git
/usrx/local/dev/packages/git/2.14.3/bin/git
climbfuji commented 4 years ago

The submodule pointer is definitely wrong, it's pointing to some hash in @ClaraDraper-NOAA's fork.

DusanJovic-NOAA commented 4 years ago

How did it work for me then? Did git fetch 'f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76' from Clara's fork? I do not see that.

climbfuji commented 4 years ago

I don't understand this at all either. Look here:

commit f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76 (HEAD)
Merge: 65f8ea1 209f1c9
Author: ClaraDraper-NOAA <Clara.Draper@noaa.gov>
Date:   Mon Jul 20 21:44:25 2020 +0000

    Merge commit '209f1c92d99b7d4cc63e0d41c652fcfd730bd9fa' into csd-cpf-sfcpert

commit 209f1c92d99b7d4cc63e0d41c652fcfd730bd9fa (origin/master, origin/HEAD, master)
Merge: b14e3e0 d71e0d5
Author: Dom Heinzeller <dom.heinzeller@icloud.com>
Date:   Wed Jul 15 13:32:35 2020 -0600

    Merge pull request #311 from climbfuji/add_degrees_to_radians_conversion

    Add unit conversion from degrees to radians, address isssue #309

this was a recursive checkout on hera. If you go to NCAR ccpp-framework and look at the commits for branch "master", the latest is 209f1c92d99b7d4cc63e0d41c652fcfd730bd9fa. The commit f5d4cd2bf7752ebf1e4ed16dfdfae71dbfabfb76 is from a closed (!) PR that was never merged in Clara's fork, see here: https://github.com/NCAR/ccpp-framework/pull/316/commits.

Maybe, but just maybe, newer git versions are able to grab it, because they can find it in the closed PR history. Sounds odd to me, though.

junwang-noaa commented 4 years ago

Jessica, would you please check again if you still have problem on git submodule update? I can't repeat the issue on dell either.

JessicaMeixner-NOAA commented 4 years ago

worked with the latest develop branch today.