search

schism-dev / schism-esmf

Earth System Modeling Framework cap for SCHISM
5 stars 6 forks source link

Make cap a submodule to SCHISM #23

Open platipodium opened 1 year ago

platipodium commented 1 year ago

Dan Rosen and Arun Chawla recommended to move the cap to within SCHISM, as a submodule

platipodium commented 1 year ago

Joseph does not really favour git submodules in SCHISM, and Carsten does not favour including the code hard into SCHISM as subdirectory ... so maybe just leave as is?

jreniel commented 1 year ago

Maybe what you want is a git subrepo?

Recap:

Git subrepo is the "newest" of the three. It definitely replaces git subtrees (which doesn't cleanly allow for 2 way comm between the repos) but submodules still have their purpose, when you want to include links to another code base, rather than the codebase itself.

Hope this helps.

[EDIT]: I strongly advice against subtrees, because your commit history will get messed up if you push from the subtree to the parent. If others have subtree cloned the repo, they'll get your history too. It's a mess if it's not going to be used for one-way only (e.g. schism repo pulling from cap repo is OK, but schism repo pushig to cap repo is bad). git subrepo fixes this problem.

jreniel commented 1 year ago

I've been using git subrepo and I highly recommend it. Git subtree was working well, until I tried 2-way push/pull. That's what took me to git subrepo, which finally solved the problem of 2-way comm between repos.

platipodium commented 1 year ago

Thanks @jreniel. I have not looked into git subrepo but would go with your recommendation. If SCHISM is s subrepo of schism-esmf (which is more logical), we can just go ahead, but if schism-esmf is a submodule of schism (which is what the community usually does), @josephzhang8 has to decide in the end.

platipodium commented 1 year ago

git-subrepo is a separate piece of software, so not many users will have it installed...

jreniel commented 1 year ago

You can make it either ways or even technically (maybe nonsensical), both. Each repo has their own build system, therefore, their own system tree to look for.

git-subrepo works without the end users having it installed, this is explicitly stated in the readme.md.

You can always test these things in your own workspace and explore the result without necessarily pushing. After installing subrepo, simply do git subrepo clone git@github.com:schism-dev/schism in your schism-esmf repo. You can do this on another branch or you can revert: git reset HEAD~1 ... Or you can even copy your workdir and try on a different dir. Either way, it will result in an opening of your options...

jreniel commented 1 year ago

To give you an example, git subtree also works without the users having it installed. One example is jigsaw-python which includes ext/jigsaw as a subtree. The users don't know this is a subtree, and don't need git-subtree installed. The same happens with git subrepo, (but not so much with git submodules, which kind-of requires to have submodules in PATH).

jreniel commented 1 year ago

Having given it some thought, I think in your case it makes more sense that schism-esmf integrates schism as a subrepo, rather than the other way around. The "child" module integrating the parent module will make more sense in the long-run. Otherwise, in a sense, you'd be handing off the maintenance of your package to the parent, which doesn't make a lot of sense.

This is similar to how jigsaw-python integrates jigsaw, rather than the other way around.

josephzhang8 commented 1 year ago

You mentioned one does not need to install subrepo but later implied we do.

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: Jaime R Calzada @.> Sent: Thursday, March 2, 2023 5:46 PM To: schism-dev/schism-esmf @.> Cc: Y. Joseph Zhang @.>; Mention @.> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

Having given it some thought, I think in your case it makes more sense that schism-esmf integrates schism as a subrepo, rather than the other way around. The "child" module integrating the parent module will make more sense in the long-run. Otherwise, in a sense, you'd be handing off the maintenance of your package to the parent, which doesn't make a lot of sense.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F23%23issuecomment-1452668059&data=05%7C01%7Cyjzhang%40vims.edu%7C6fdf3dc4fa4b4c9046c908db1b6fe6e2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638133939663328342%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LHzqgeCL6HYBG9%2BbS2ltL1ZumUo7NUxXNU786zSKPo0%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZYQP4O6F4RRG25G4GTW2EPKVANCNFSM6AAAAAAVNZH2II&data=05%7C01%7Cyjzhang%40vims.edu%7C6fdf3dc4fa4b4c9046c908db1b6fe6e2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638133939663328342%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eF9CIZg1P2%2Fg7KCam2DBLLg7DG6ncn%2FbpTfV54UFD4A%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

josephzhang8 commented 1 year ago

I tried and it did not recognize 'subrepo'

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: Y. Joseph Zhang Sent: Thursday, March 2, 2023 6:19 PM To: schism-dev/schism-esmf @.>; schism-dev/schism-esmf @.> Cc: Mention @.***> Subject: RE: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

You mentioned one does not need to install subrepo but later implied we do.

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: Jaime R Calzada @.**@.>> Sent: Thursday, March 2, 2023 5:46 PM To: schism-dev/schism-esmf @.**@.>> Cc: Y. Joseph Zhang @.**@.>>; Mention @.**@.>> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

Having given it some thought, I think in your case it makes more sense that schism-esmf integrates schism as a subrepo, rather than the other way around. The "child" module integrating the parent module will make more sense in the long-run. Otherwise, in a sense, you'd be handing off the maintenance of your package to the parent, which doesn't make a lot of sense.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F23%23issuecomment-1452668059&data=05%7C01%7Cyjzhang%40vims.edu%7C6fdf3dc4fa4b4c9046c908db1b6fe6e2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638133939663328342%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LHzqgeCL6HYBG9%2BbS2ltL1ZumUo7NUxXNU786zSKPo0%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZYQP4O6F4RRG25G4GTW2EPKVANCNFSM6AAAAAAVNZH2II&data=05%7C01%7Cyjzhang%40vims.edu%7C6fdf3dc4fa4b4c9046c908db1b6fe6e2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638133939663328342%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eF9CIZg1P2%2Fg7KCam2DBLLg7DG6ncn%2FbpTfV54UFD4A%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

jreniel commented 1 year ago

The projects maintainer (the person doing the subrepo adding) needs to install it. But once the surepo has be initialized, anyone that clones the repo does not need it installed. Plz read carefully the subrepo docs. It really does solve your problem.

jreniel commented 1 year ago

(It is also trivial to install it) https://github.com/ingydotnet/git-subrepo#installation

jreniel commented 1 year ago

In this case, Carsten would be the only one needing to do the subrepo install, no one else would need to install it, not even the SCHISM repo. Subrepo does not interfere with the parent, which is the problem with subtree, that can interfere with the parent/siblings if more there's 3-way push/pull. Subtree only work well in 1-way mode, which would still be valid for Carsten's case, however I'm sure he'd prefer a "clean 2-way mode", which is what subrepo provides and subtree lacks.

Anyone cloning schism-esmf also would not need it installed, just the people doing the direct maintenance on the subrepo itself would need to install it.

jreniel commented 1 year ago

Pretty much all the common arguments against using it are addressed here: https://github.com/ingydotnet/git-subrepo#benefits

platipodium commented 1 year ago

@jreniel what a fervent discussion :=). Although it sounds nonsensical, I could see in the long run that they include each other (I can only hope that won't be recursive). I can try on the schism-esmf side first and we see how it works for CoastalApp folks (and for ourselves).

josephzhang8 commented 1 year ago

We need to be careful so as not to jeopardize the hard won progress made in the last 2 years.

Sent from my iPhone

On Mar 3, 2023, at 3:13 PM, Carsten Lemmen @.***> wrote:

 [EXTERNAL to VIMS received message]

@jrenielhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjreniel&data=05%7C01%7Cyjzhang%40vims.edu%7Ca695ddb90a8a437cf96e08db1bb6c8f2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638134244107437940%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DKQJCXUhFE2B%2F7SPfT6Ag8Spee3GNufBlKT%2BAJTD6B4%3D&reserved=0 what a fervent discussion :=). Although it sounds nonsensical, I could see in the long run that they include each other (I can only hope that won't be recursive). I can try on the schism-esmf side first and we see how it works for CoastalApp folks (and for ourselves).

— Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F23%23issuecomment-1453081677&data=05%7C01%7Cyjzhang%40vims.edu%7Ca695ddb90a8a437cf96e08db1bb6c8f2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638134244107437940%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UnXHtzsXHC%2BR0959uYWHCsUuFN9Z19jTdWlrnZoS48U%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ25SUUTEMIQBORFK7TW2GKZNANCNFSM6AAAAAAVNZH2II&data=05%7C01%7Cyjzhang%40vims.edu%7Ca695ddb90a8a437cf96e08db1bb6c8f2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638134244107437940%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=u7c%2FNqin09PyIsP%2Bx%2BA9j7C4OtxXGomxw9s9EiDsrlI%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

platipodium commented 1 year ago

For compatibility with CMEPS, Marina Verteinstein suggested to make the cap a submodule to schism .... Just noting down this. There's her personal fork that did that.

My conclusions so far

josephzhang8 commented 1 year ago

Thank you Carsten for the summary. I tend to agree with your assessment; anything we do need to also consider Dan's work on PDAF. I need to look at these more closely.

Regards,

Joseph Zhang (804)684 7595 (office)

SCHISM web: http://ccrm.vims.edu/schism/ From: Carsten Lemmen @.> Sent: Saturday, March 4, 2023 6:35 AM To: schism-dev/schism-esmf @.> Cc: Y. Joseph Zhang @.>; Mention @.> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

For compatibility with CMEPS, Marina Verteinstein suggested to make the cap a submodule to schism .... Just noting down this. There's her personal fork that did that.

My conclusions so far

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F23%23issuecomment-1454708920&data=05%7C01%7Cyjzhang%40vims.edu%7C47961f3e7a264a708f7208db1ca48ea7%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638135265334409614%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LLPavVMnAJzqIV50RQurxq40ILDeCEpUNYsAZUjGLA4%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ5S4UBG45BTA5LCFUTW2MSIBANCNFSM6AAAAAAVNZH2II&data=05%7C01%7Cyjzhang%40vims.edu%7C47961f3e7a264a708f7208db1ca48ea7%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638135265334409614%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=20sOyLvRPVT7%2FYZ2wCskmi2XBknpj9VBkCeZw4DkfYU%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

josephzhang8 commented 1 year ago

I looked at the url Jaime shared. While the subrepo cmds are indeed much cleaner than submodule, my main concern is that 'collaborators' need to remember they are working in a subrepo instead of a regular repo and use appropriate cmds for commit etc. If I were the only committer, I'd always work in the original repo and then subrepo pull but this rule cannot be easily enforced. There is also the issue of PR - which repo and has anyone actually tested that?

I think the most sustainable solution is to combine the 2 repos, and then either delete or rename the obsolete repo. Schism-esmf repo has only 110MB of files while schism repo is much larger. I can go either way of combining. Carsten: would putting esmf inside schism repo cause make issues? I know the logic is a bit weird but this is the approach adopted by other models (and preferred by UFS).

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: Joseph Zhang @.> Sent: Saturday, March 4, 2023 3:56 PM To: schism-dev/schism-esmf @.> Cc: Y. Joseph Zhang @.>; Your activity @.> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

Thank you Carsten for the summary. I tend to agree with your assessment; anything we do need to also consider Dan's work on PDAF. I need to look at these more closely.

Regards,

Joseph Zhang (804)684 7595 (office)

SCHISM web: http://ccrm.vims.edu/schism/ From: Carsten Lemmen @.<mailto:@.>> Sent: Saturday, March 4, 2023 6:35 AM To: schism-dev/schism-esmf @.<mailto:@.>> Cc: Y. Joseph Zhang @.<mailto:@.>>; Mention @.<mailto:@.>> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

For compatibility with CMEPS, Marina Verteinstein suggested to make the cap a submodule to schism .... Just noting down this. There's her personal fork that did that.

My conclusions so far

platipodium commented 1 year ago

I don't see urgency to act on this.

josephzhang8 commented 1 year ago

Thx Carsten. I think UFS folks are pressuring us to do this? I'm concerned that subrepo won't be usable enough for developers.

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: Carsten Lemmen @.> Sent: Monday, March 20, 2023 3:41 PM To: schism-dev/schism-esmf @.> Cc: Y. Joseph Zhang @.>; Mention @.> Subject: Re: [schism-dev/schism-esmf] Make cap a submodule to SCHISM (Issue #23)

[EXTERNAL to VIMS received message]

I don't see urgency to act on this.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism-esmf%2Fissues%2F23%23issuecomment-1476830526&data=05%7C01%7Cyjzhang%40vims.edu%7Ce07d4fe36b664f64acaa08db297b06e8%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638149380623995729%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ewy%2BHdvW7d1REAPigKdVELclONrMC%2BAMZMTyiidUbfs%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ45PQBHGN4VF3VKGQLW5CXETANCNFSM6AAAAAAVNZH2II&data=05%7C01%7Cyjzhang%40vims.edu%7Ce07d4fe36b664f64acaa08db297b06e8%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638149380623995729%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FIA6T0%2BzuD2QBBT6NeDMA0cBt2IWIYZ8N9Pp%2FENJdoI%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>