search

schism-dev / schism

Semi-implicit Cross-scale Hydroscience Integrated System Model (SCHISM)
http://ccrm.vims.edu/schismweb/
Apache License 2.0
85 stars 86 forks source link

Install with conda #13

Closed brey closed 1 year ago

brey commented 4 years ago

Dear All, may I suggest to configure also a conda based installation option, preferably on conda-forge. One can have multiple builds to meet all the configurations of SCHISM.

I think this option will provide a flexible and portable way to use SCHISM.

I have played a bit with this and I have managed to have it up and running relatively easy. You can try it out with

conda install -c gbrey pschism

It works for linux-64 and os-x.

Let me know if I could be of any help.

George

platipodium commented 3 years ago

I would like a conda install possibility, but be aware that conda is not recommended on HPC systems (we recently removed it from ours), see e.g. https://docs.computecanada.ca/wiki/Anaconda/en.

Also, with the current strategy of most dependencies resolved in-tree, the dependency management is not (yet) too complicated (mostly netcdf). This could become a more pressing issue if we get Joseph to accept more relying on external dependencies .. :=).

josephzhang8 commented 3 years ago

I think George B. also mentioned the possibility of conda install.

Too bad conda is now blacklisted as we are moving in that direction.

jreniel commented 3 years ago

Not being able to use conda is inconvenient, but not a game changer, as you are still allowed to use virtualenv. The main difference would be that with with virtualenv you might need to satisfy some dependencies manually, something that we still have to do anyways when we provision a machine with the SCHISM binaries.

josephzhang8 commented 3 years ago

Good to know; thx Jaime.

brey commented 3 years ago

Dear All, I do agree that for a production HPC env it would make sense to have a module based installation for efficiency but, in my opinion, for development and prototyping nothing beats the simplicity of conda. Having the same configuration on the HPC and my laptop (where the dev is happening) within minutes without the need of Admin rights is very productive. I am using conda on our HPC since I couldn't get SCHISM compiled with the available modules and I am only doing test runs anyway. Your mileage may vary. The above referenced conda command installs the 4.8 tagged version and works for me (let me know if you have any problems). It would take some efforts to have all options configured though. I have compiled mine with mpich but we need to have one with OpenMP as well. Not to mention all the variants of SCHISM modules. I will make a pull request with the conda recipe files asap.

brey commented 2 years ago

I am working in finalising the conda recipe. May I reiterate the possibility of having the metis/parmetis dependency covered independently from a stand alone installation.

Both packages are available in conda.

Any reason why they need to be included in the build process?

josephzhang8 commented 2 years ago

Hi Geroge:

PM3 used to have an issue so we had to use our own copy. PM4 can be used out-of-box without any modifications. So I think it's ok to link to a pre-installed PM4 lib. Thx.

-Joseph

Joseph Zhang Office: (804) 684 7466 Web: schism.wiki

From: George Breyiannis @. Sent: Friday, November 12, 2021 9:27 AM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

I am working in finalising the conda recipe. May I reiterate the possibility of having the metis/parmetis dependency covered independently from a stand alone installation.

Both packages are available in conda.

Any reason why they need to be included in the build process?

- You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-967157643&data=04%7C01%7Cyjzhang%40vims.edu%7C8f9aa81cdf9a4d15854808d9a5e86d07%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723239963870226%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ueX4jtd093LGIEViKLw8cAOKF14uRe7WFAzkgq9GMJw%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ4VGL6R2HEO7DP5P23ULUPZRANCNFSM4L2WZZCQ&data=04%7C01%7Cyjzhang%40vims.edu%7C8f9aa81cdf9a4d15854808d9a5e86d07%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723239963880182%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=z3tLXykdt2sejM7JeoGVXhkBaKxfvNVNaaEEfS1mpHE%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cyjzhang%40vims.edu%7C8f9aa81cdf9a4d15854808d9a5e86d07%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723239963880182%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=LfM%2BV8LC6pxmguDVnjqygPERu9EPhjDMUeJuVc0oeGM%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cyjzhang%40vims.edu%7C8f9aa81cdf9a4d15854808d9a5e86d07%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723239963890143%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Q0NCo6kve8inH8QdoSqCuF4%2BgtZG3XRR2aEOvhReTwc%3D&reserved=0.

brey commented 2 years ago

In that case, any ideas on how to mask the compilation of the local Metis folder from CMake (not my strong point)?

josephzhang8 commented 2 years ago

Hi Eli:

Could you add that in cmake also, since you are the original architect? Thx.

-Joseph

Joseph Zhang Office: (804) 684 7466 Web: schism.wiki

From: George Breyiannis @. Sent: Friday, November 12, 2021 9:40 AM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

In that case, any ideas on how to mask the compilation of the local Metis folder from CMake (not my strong point)?

- You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-967167327&data=04%7C01%7Cyjzhang%40vims.edu%7Cc9a16b0e55874fd28daf08d9a5ea5384%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723248113110789%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mfUNhBBsd3F7ZcKKUPYEGj%2BAqd9UVyB4TpX97uVzAhk%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZY6RTQZTBNSNSSE6VTULURMPANCNFSM4L2WZZCQ&data=04%7C01%7Cyjzhang%40vims.edu%7Cc9a16b0e55874fd28daf08d9a5ea5384%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723248113120746%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=o4H2dLgCv9NkJDVTaNDj6UoYMp8z72uYh%2Brib8VO3kQ%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cyjzhang%40vims.edu%7Cc9a16b0e55874fd28daf08d9a5ea5384%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723248113120746%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4RjmjLxA1yFcdg4dCobdKzXKCK3Kdu%2BZHVHjJA68FiI%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cyjzhang%40vims.edu%7Cc9a16b0e55874fd28daf08d9a5ea5384%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723248113130705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ckUWLc6Qm4VFPRxuWkK2G0BE8AjWDHUVI6c3zk9wCdE%3D&reserved=0.

jreniel commented 2 years ago

Not my strong point either, but my intuition tells me that you probably still need the parmetis headers to compile schism, and of course, they have to match whatever binary version is being bootstrapped through conda. Since you still need the parmetis headers, and since the parmetis source code is bundled inside the schism source code, doing full parmetis compilation might actually be better than using a conda one. In any case, it's your choice, it depends how much you want to leverage "flexibility" against complexity.

In any case, I think it might be possible to bootstrap the conda metis lib, as long as the headers used match versions (I'm not sure if conda will distribute the headers along with the binaries though, which would technically be the safest route). I would just compile the provided metis as part of the schism-conda recipe, but that's just me...

jreniel commented 2 years ago

FYI, if you want to deploy SCHISM anywhere and make sure it uses with modules then you can use any of these links:

https://github.com/jreniel/schism_deploy_v5.8.0 https://github.com/jreniel/schism_deploy_v5.9.0 https://github.com/jreniel/schism_deploy_master

josephzhang8 commented 2 years ago

Sure. Again it will be next week before I have even a hope. To clarify (I'm a bit asleep during this exchange) is the subject of this to compile parmetis and not metis or is the subject to drop the whole library.

One thing we discussed in the burst of activity around moving to GitHub was the possibility of automatically downloading parmetis if its location isn't named. I believe this required a version bump. I guess it isn't a very powerful idea though while the library is housed inside the schism source code.

From: Y. Joseph Zhang @.> Sent: Friday, November 12, 2021 7:21 AM To: schism-dev/schism @.>; schism-dev/schism @.> Cc: Comment @.>; Ateljevich, @. @.> Subject: RE: [schism-dev/schism] Install with conda (#13)

Hi Eli:

Could you add that in cmake also, since you are the original architect? Thx.

-Joseph

Joseph Zhang

Office: (804) 684 7466

Web: schism.wiki

From: George Breyiannis @. Sent: Friday, November 12, 2021 9:40 AM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

In that case, any ideas on how to mask the compilation of the local Metis folder from CMake (not my strong point)?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-967167327&data=04%7C01%7C%7Cbd4df6d10d8b4c29e3c008d9a5f01fd6%7Cb71d56524b834257afcd7fd177884564%7C0%7C0%7C637723273440929744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RRNqW63VwDBKfPpZnJb%2Fj%2FEV6Yx%2FPKfJOUiaPPURSKU%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZY6RTQZTBNSNSSE6VTULURMPANCNFSM4L2WZZCQ&data=04%7C01%7C%7Cbd4df6d10d8b4c29e3c008d9a5f01fd6%7Cb71d56524b834257afcd7fd177884564%7C0%7C0%7C637723273440929744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sqePSvPAilb0gouE6%2FvYijsh4irzkeQm9vzsad4qSng%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7Cbd4df6d10d8b4c29e3c008d9a5f01fd6%7Cb71d56524b834257afcd7fd177884564%7C0%7C0%7C637723273440939695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3ST8owiAmZeMPeZpm6xUi2wLG%2FV4e8VlMRlK6ew%2Fl%2FU%3D&reserved=0 or Androidhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7Cbd4df6d10d8b4c29e3c008d9a5f01fd6%7Cb71d56524b834257afcd7fd177884564%7C0%7C0%7C637723273440939695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=25YTNftQZK1P58%2BK3gTo6%2F6ST5xTGv0htiXzTBMiKRA%3D&reserved=0.

jreniel commented 2 years ago

is the subject of this to compile parmetis and not metis or is the subject to drop the whole library.

The subject is whether the conda recipe should use the precompiled version that exists in the conda repo, or should it compile the one included with SCHISM. Technically you can do either, the question is what you gain and lose from chose one or the other. I don't think dropping the whole library is part of the conversation (I don't think that's even possible, since parmetis obviosuly plays a critical role!)

Easy and safe is to compiled the one bundled. Using the conda one would probably require to find and match the correct version of the headers, so while doable, might be more complicated than simply compiling the one provided.

One thing we discussed in the burst of activity around moving to GitHub was the possibility of automatically downloading parmetis if its location isn't named. I believe this required a version bump. I guess it isn't a very powerful idea though while the library is housed inside the schism source code.

I remember these conversations, and while we flirted with the idea of downloading on the fly, now that I have more experience, I think it is better to just keep it in the source code (for simplicity). But there are other formal alternatives, most notably git subtrees, which I would suggest to talk about in the event that we need to change the Parmetis version in the future.

josephzhang8 commented 2 years ago

Thx Eli for your help. I just want to add that compiling PM each time takes trivial amount of time, so as an average user I'd always do that. This also avoided any dependency. I do understand George's point of linking to an existing PM lib though.

-Joseph

Y. Joseph Zhang Web: schism.wiki Office: 804 684 7466

From: Ateljevich, @. @.> Sent: Friday, November 12, 2021 12:50 PM To: Y. Joseph Zhang @.>; schism-dev/schism @.>; schism-dev/schism @.> Cc: Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

Sure. Again it will be next week before I have even a hope. To clarify (I'm a bit asleep during this exchange) is the subject of this to compile parmetis and not metis or is the subject to drop the whole library.

One thing we discussed in the burst of activity around moving to GitHub was the possibility of automatically downloading parmetis if its location isn't named. I believe this required a version bump. I guess it isn't a very powerful idea though while the library is housed inside the schism source code.

From: Y. Joseph Zhang @.**@.>> Sent: Friday, November 12, 2021 7:21 AM To: schism-dev/schism @.**@.>>; schism-dev/schism @.**@.>> Cc: Comment @.**@.>>; Ateljevich, @. @*.**@*.***>> Subject: RE: [schism-dev/schism] Install with conda (#13)

Hi Eli:

Could you add that in cmake also, since you are the original architect? Thx.

-Joseph

Joseph Zhang

Office: (804) 684 7466

Web: schism.wiki

From: George Breyiannis @. Sent: Friday, November 12, 2021 9:40 AM To: schism-dev/schism @*.**@*.>> Cc: Y. Joseph Zhang @*.**@*.>>; Comment @*.**@*.***>> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

In that case, any ideas on how to mask the compilation of the local Metis folder from CMake (not my strong point)?

- You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-967167327&data=04%7C01%7Cyjzhang%40vims.edu%7C578480c0ca174387bcb408d9a604db19%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723362075298180%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=LB5LwHCBkx06WHi1cBM98pmfg0uZFrR5n15d5RGGy%2Fo%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZY6RTQZTBNSNSSE6VTULURMPANCNFSM4L2WZZCQ&data=04%7C01%7Cyjzhang%40vims.edu%7C578480c0ca174387bcb408d9a604db19%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723362075308133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eBfh5rLEmmmNAjND8HkCx1m5NTpdJAMyGGfyRziQ8hg%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cyjzhang%40vims.edu%7C578480c0ca174387bcb408d9a604db19%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723362075308133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NPUmhe2qCvKYha2yWdv0fZAXeurCaFOVtUHGkLazjPY%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cyjzhang%40vims.edu%7C578480c0ca174387bcb408d9a604db19%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C637723362075318096%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RNaRVtpt7hZo6aZXpjHqbZe9FIvIeFKcihZYPodik6A%3D&reserved=0.

jreniel commented 2 years ago

FYI This isn't Eli talking. He hasn't actually responded. Eli probably knows by memory which CMake variables need to be overridden for using conda parmetis. I would have to dig a little to find them, but through CMake variables you certainly can override which Parmetis gets used.

brey commented 2 years ago

Hi All. I have sorted out the issues with CMAKE and METIS/PARMETIS. The conda recipe avoids compiling the local library and uses the conda-forge version.

You can get both pschism and pschism-wwm with

conda install -c gbrey pschism=5.9

or

conda install -c gbrey pschism-wwm=5.9

However, I am having an issue with the wwm flavor on Linux (it compiles nicely on osx). I have traced the problem to the definition of constants in OPENMP. See https://stackoverflow.com/questions/63016641/cant-support-both-openmp-v4-and-v5-in-gcc.

Apparently such an issue doesn't affect clang that is why it works on osx. Let me know if you've seen this issue and how, in your opinion, should be addressed.

platipodium commented 2 years ago

Thanks for the update. I think that we should deprecate the use of gcc < 9 / OpenMP < 5. The current stable version is 11, with version 12 expected in April 2021.

jamal919 commented 2 years ago

However, I am having an issue with the wwm flavor on Linux (it compiles nicely on osx). I have traced the problem to the definition of constants in OPENMP. See https://stackoverflow.com/questions/63016641/cant-support-both-openmp-v4-and-v5-in-gcc.

Do you think #57 relates to this problem?

platipodium commented 1 year ago

@gbrey what is the status of the conda deployment?

brey commented 1 year ago

@platipodium I think it's done. I was wondering if there is a need to have the OLD IO option even with >5.10 but I think that having the 5.9 version should be enough. You can test both versions on my channel with 

conda install -c gbrey pschism=5.10*=mpi_openmpi*
conda install -c gbrey pschism=5.10*=mpi_mpich*
conda install -c gbrey pschism=5.9*=mpi_openmpi*
conda install -c gbrey pschism=5.9*=mpi_mpich*

and 

conda install -c gbrey pschism-wwm=5.10*=mpi_openmpi*
conda install -c gbrey pschism-wwm=5.10*=mpi_mpich*
conda install -c gbrey pschism-wwm=5.9*=mpi_openmpi*
conda install -c gbrey pschism-wwm=5.9*=mpi_mpich*

depending on your MPI flavour. I will do a last check and then I will push a PL to the repo and also one in the conda-forge channel if no one has an objection.

In fact someone should also be admin/maintainer on conda-forge besides me. Any volunteers?

platipodium commented 1 year ago

@gbrey Thanks for the update.

josephzhang8 commented 1 year ago

Thank you George, Carsten.

Regards,

Joseph Zhang (804)684 7595 (office)

SCHISM web: http://ccrm.vims.edu/schism/ From: Carsten Lemmen @.> Sent: Sunday, March 5, 2023 6:43 AM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

@gbreyhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbrey&data=05%7C01%7Cyjzhang%40vims.edu%7Cc8722faef3c545b27b2c08db1d6ec8d2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136133902774301%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eoJJwxZCIiFDBqKfjt%2Ft4Tge%2BtPELxHdbcipGwdLev4%3D&reserved=0 Thanks for the update.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-1455066925&data=05%7C01%7Cyjzhang%40vims.edu%7Cc8722faef3c545b27b2c08db1d6ec8d2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136133902774301%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qA4gsYLmy1IY3mc38hTvY%2FllGecl9Rg5Q5%2FrBdT6zuw%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ4AKSQQPV5ZQD524WDW2R34RANCNFSM4L2WZZCQ&data=05%7C01%7Cyjzhang%40vims.edu%7Cc8722faef3c545b27b2c08db1d6ec8d2%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136133902774301%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WmXQtLe7PgAkiiMsVQ1tQxFIP6X92VSAeyCiKOO0fRA%3D&reserved=0. You are receiving this because you commented.Message ID: @.**@.>>

brey commented 1 year ago

@platipodium Keeping the OLDIO option creates a problem.

Currently there are 2 binaries (TVD-LD and WWM) and 2 MPI options (mpich and openmpi). Thus the 4 total versions/release above. However, if we include the OLDIO as an option it would create 8 versions and most importantly a naming issue. How are we going to call the OLDIO packages?

pschism, pschism-oldio
pschism-wwm, pschism-wwm-oldio

This will create some confusion. Is running on one core a long term requirement?

In fact, this is relevant to a more general question: How many conda versions we want to release (in terms of modules)? If there are many, the OLDIO option will increase the number. Ideally, the best option would be to have one binary and invoke the modules in runtime. Is this doable?

Also, maybe there is a way to speed up --oversubscribe.

josephzhang8 commented 1 year ago

HI George:

Invoking modules at runtime is not possible; most modules use their own 'private' arrays for memory conservation. There are other considerations as well (e.g. TVD for efficiency).

Regards,

Joseph Zhang (804)684 7595 (office)

SCHISM web: http://ccrm.vims.edu/schism/ From: George Breyiannis @.> Sent: Sunday, March 5, 2023 1:51 PM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

@platipodiumhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fplatipodium&data=05%7C01%7Cyjzhang%40vims.edu%7C0610a42ba8f04b57e03008db1daa867a%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136390557461924%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QxHsP09bnqDIjodP8hzxQeIipa656QPMfLIoQMyUTCs%3D&reserved=0 Keeping the OLDIO option creates a problem.

Currently there are 2 binaries (TVD-LD and WWM) and 2 MPI options (mpich and openmpi). Thus the 4 total versions/release above. However, if we include the OLDIO as an option it would create 8 versions and most importantly a naming issue. How are we going to call the OLDIO packages?

pschism, pschism-oldio

pschism-wwm, pschism-wwm-oldio

This will create some confusion. Is running on one core a long term requirement?

In fact, this is relevant to a more general question: How many conda versions we want to release (in terms of modules)? If there are many, the OLDIO option will increase the number. Ideally, the best option would be to have one binary and invoke the modules in runtime. Is this doable?

Also, maybe there is a way to speed up --oversubscribe.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-1455171947&data=05%7C01%7Cyjzhang%40vims.edu%7C0610a42ba8f04b57e03008db1daa867a%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136390557618165%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=inYEL2QyAceAOVuPJ59RL1QYfTut5HDsuBOEJFzVSZ4%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ2KIJWNVAFEK34HQNLW2TOAHANCNFSM4L2WZZCQ&data=05%7C01%7Cyjzhang%40vims.edu%7C0610a42ba8f04b57e03008db1daa867a%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136390557618165%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=I5Hw7ekrKBlWMnbSAGlP%2FWlTqvzWKIC9coRZiHhrP34%3D&reserved=0. You are receiving this because you commented.Message ID: @.**@.>>

brey commented 1 year ago

Thanks Joseph. How about using only OpenMP and multithreading for running on one core without oversubscribing?

platipodium commented 1 year ago

1) Hmm, I think we need to find a way to deal with multiple options. OLDIO is only one of them, WWM, TVD-VL, FABM, GEN and any of the others could also be relevant. In principle conda should be able to handle this, but I am no expert on the variants.

There could be a default variant pschism (using openmpi, OLDIO=OFF and TVD-VL=SB, or any other)

And then many variants with positive selection in any way conceivable.... pschism+FABM+OLDIO+WWM+SB pschism+VL+WWM+GEN+ICM+MPICH

We can use CD to create all of them and the respective conda packages.

2) Not providing MPI (in both flavours openmpi/mpich) would deter users who need that capability, not offering single-processor options would deter first-timers or people who'd like to try the system or who have simple test cases.

josephzhang8 commented 1 year ago

I assume you mean hyperthreading? This is possible as long as the system allows it, either under openMP or MPI or both. Our experience is that HT with openMP works best. You can run with 1 core and openMP threads also.

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: George Breyiannis @.> Sent: Sunday, March 5, 2023 4:58 PM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; Comment @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

Thanks Joseph. How about using only OpenMP and multithreading for running on one core without oversubscribing?

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-1455219365&data=05%7C01%7Cyjzhang%40vims.edu%7C6b41d4e95e6940e610a608db1dc4c12c%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136503118149359%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2F3dsfW4VAdM2IgYFG8Fv%2FU5wY%2FhGvdzMR2fpJ5pqRK0%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ4AZHW2AJEA7UF2JOLW2UEAJANCNFSM4L2WZZCQ&data=05%7C01%7Cyjzhang%40vims.edu%7C6b41d4e95e6940e610a608db1dc4c12c%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638136503118149359%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=r7dZ%2BoESvrVi%2F41a%2Bq%2B7BiiAVRFFbPbOinnoAuw9Gl4%3D&reserved=0. You are receiving this because you commented.Message ID: @.**@.>>

brey commented 1 year ago 
not offering single-processor options would deter first-timers or people who'd like to try the system or who have simple test cases.

I am not sure I get this. Every computer now is multicore. So testing the code on a simple case, even with MPI, and at minimum 2 cores shouldn't be a problem. If you output a large number of vectors this could be an issue for, say, an 8 core laptop but this gets very specific.

Note that coda installation is good enough for prototyping but not necessarily for production (side note, @pmav99 achieved extremely good latency with a conda implementation on Azure HPC with InfiniBand) .

In addition, conda provides automatically the MPI without any issues for the user. I remember you've mentioned multiple 1 core experiments with FABM. Those become now 2+ cores experiments which is an issue but I would say poses no big constraint for a new user testing stuff. It would, however, require more resources for scaling up.

Did I miss something?

Another option would be to update the code on the 5.9 branch differentiating between OLDIO and the new one on the semantic versioning. I am not sure though if this is the right approach going forward.

josephzhang8 commented 1 year ago

Issue resolved per @brey

brey commented 1 year ago

Dear All, I am trying to finalise the matrix of binaries that will be offered via conda. From the cmake files, I have:

#   Algorithm choices
# TVD_LIM must be one of SB, VL, MM or OS for Superbee, Van Leer, Minmod, or Osher.")
set (TVD_LIM VL CACHE STRING "Flux limiter")
#Turn OLDIO off to use the new scribe based I/O
set (OLDIO OFF CACHE BOOLEAN "Old nc output (each rank dumps its own data)")
set (PREC_EVAP OFF CACHE BOOLEAN "Include precipitation and evaporation calculation")
set (USE_GOTM OFF CACHE BOOLEAN "Use GOTM turbulence model. This just enables the build -- GOTM must still be selected in param.nml")
set (USE_HA OFF CACHE BOOLEAN "Enable harmonic analysis output modules")   # Not implemented yet
set( USE_MARSH OFF CACHE BOOLEAN "Use marsh module")

#   Enable/Disable Modules
set( USE_SED2D OFF CACHE BOOLEAN "Use 2D sediment module")
set( USE_WWM OFF CACHE BOOLEAN "Use wind-wave module")
set( USE_ICE OFF CACHE BOOLEAN "Use ICE module")

#Tracer models
set( USE_GEN OFF  CACHE BOOLEAN "Use generic tracer module")
set( USE_AGE OFF  CACHE BOOLEAN "Use age module")
set( USE_ECO OFF   CACHE BOOLEAN "Use ECO-SIM module")
set( USE_ICM OFF   CACHE BOOLEAN "Use ICM module")
##pH model is active only if ICM is on
set( ICM_PH OFF   CACHE BOOLEAN "Use pH module inside ICM")
set( USE_COSINE OFF   CACHE BOOLEAN "Use CoSiNE module")
set( USE_FIB OFF   CACHE BOOLEAN "Use fecal indicating bacteria module")
set( USE_SED OFF    CACHE BOOLEAN "Use sediment module")
set( USE_FABM OFF  CACHE BOOLEAN "FABM BGC model interface")

set (DEBUG OFF CACHE BOOLEAN "Enable diagnostic output")
set (USE_ANALYSIS OFF CACHE BOOLEAN "Enable (somewhat costly) derviation of derived flow/stress quantities")

Q1. What are the options for TVD_LIM? I am bit confused with OS. Q2. Are all module options compatible between them? That is, do all possible combinations of SED2, WWM & ICE work and therefore should be offered? Q3. Should there be only one tracer model available per binary or more tracer modules can be present? Q4. Same as Q2 for PH, COSINE, FIB, SED (different from SED2?) and FABM.

In a nutshell, if you can provide a matrix of the combinations we should provide, I can finalise the conda recipe.

platipodium commented 1 year ago

Hmm, I see how this could be very confusing for users who want to "Just download SCHISM". How about providing those that we (whoever) frequently use. My personal list is as matrix of all combinations of [SB, OLDIO, FABM, GEN]

josephzhang8 commented 1 year ago

Q1. What are the options for TVD_LIM? I am bit confused with OS.

This flag sets the TVD schemes. OS stands for 'Osher' scheme. We typically used VL.

Q2. Are all module options compatible between them? That is, do all possible combinations of SED2, WWM & ICE work and therefore should be offered?

Q4. Same as Q2 for PH, COSINE, FIB, SED (different from SED2?) and FABM.

Most modules are orthogonal to each other, so mix-and-match is fine, but some combinations may not make sense.

Exceptions: (1) SED2D is kinda obsolete with polymorphism and SED (3D sediment transport module), and may be removed in the future; (2) PH is a sub-module of ICM only.

Q3. Should there be only one tracer model available per binary or more tracer modules can be present?

Normally, yes but sometimes it makes sense to have more, e.g. GEN+AGE+ICM+SED is used in our project. FABM is supposed to be the most flexible that would ultimately replace all BGC modules, but this is on-going work.

It's up to you how many combinations you want to offer with conda. It's hard to define 'popular' combinations as the community is diverse. Thx.

Regards,

Joseph Zhang (804)684 7595 (office) SCHISM web: http://ccrm.vims.edu/schism/ From: George Breyiannis @.> Sent: Wednesday, March 8, 2023 8:31 AM To: schism-dev/schism @.> Cc: Y. Joseph Zhang @.>; State change @.> Subject: Re: [schism-dev/schism] Install with conda (#13)

[EXTERNAL to VIMS received message]

Dear All, I am trying to finalise the matrix of binaries that will be offered via conda. From the cmake files, I have:

Algorithm choices

TVD_LIM must be one of SB, VL, MM or OS for Superbee, Van Leer, Minmod, or Osher.")

set (TVD_LIM VL CACHE STRING "Flux limiter")

Turn OLDIO off to use the new scribe based I/O

set (OLDIO OFF CACHE BOOLEAN "Old nc output (each rank dumps its own data)")

set (PREC_EVAP OFF CACHE BOOLEAN "Include precipitation and evaporation calculation")

set (USE_GOTM OFF CACHE BOOLEAN "Use GOTM turbulence model. This just enables the build -- GOTM must still be selected in param.nml")

set (USE_HA OFF CACHE BOOLEAN "Enable harmonic analysis output modules") # Not implemented yet

set( USE_MARSH OFF CACHE BOOLEAN "Use marsh module")

Enable/Disable Modules

set( USE_SED2D OFF CACHE BOOLEAN "Use 2D sediment module")

set( USE_WWM OFF CACHE BOOLEAN "Use wind-wave module")

set( USE_ICE OFF CACHE BOOLEAN "Use ICE module")

Tracer models

set( USE_GEN OFF CACHE BOOLEAN "Use generic tracer module")

set( USE_AGE OFF CACHE BOOLEAN "Use age module")

set( USE_ECO OFF CACHE BOOLEAN "Use ECO-SIM module")

set( USE_ICM OFF CACHE BOOLEAN "Use ICM module")

pH model is active only if ICM is on

set( ICM_PH OFF CACHE BOOLEAN "Use pH module inside ICM")

set( USE_COSINE OFF CACHE BOOLEAN "Use CoSiNE module")

set( USE_FIB OFF CACHE BOOLEAN "Use fecal indicating bacteria module")

set( USE_SED OFF CACHE BOOLEAN "Use sediment module")

set( USE_FABM OFF CACHE BOOLEAN "FABM BGC model interface")

set (DEBUG OFF CACHE BOOLEAN "Enable diagnostic output")

set (USE_ANALYSIS OFF CACHE BOOLEAN "Enable (somewhat costly) derviation of derived flow/stress quantities")

Q1. What are the options for TVD_LIM? I am bit confused with OS. Q2. Are all module options compatible between them? That is, do all possible combinations of SED2, WWM & ICE work and therefore should be offered? Q3. Should there be only one tracer model available per binary or more tracer modules can be present? Q4. Same as Q2 for PH, COSINE, FIB, SED (different from SED2?) and FABM.

In a nutshell, if you can provide a matrix of the combinations we should provide, I can finalise the conda recipe.

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fschism%2Fissues%2F13%23issuecomment-1460159137&data=05%7C01%7Cyjzhang%40vims.edu%7C552579636cb54e8a9e7708db1fd956b7%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638138790547119550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YfaMbWIclDGS2sIJXwPpmwGUokxFB6wkwEQEtyhi5fw%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ5UOXZQL23LOYXSCG3W3CCYXANCNFSM4L2WZZCQ&data=05%7C01%7Cyjzhang%40vims.edu%7C552579636cb54e8a9e7708db1fd956b7%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638138790547275754%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MfzhR4tU%2B5v9MT6SUGwBAVVxlmCLRo0g1wLPaEUiw44%3D&reserved=0. You are receiving this because you modified the open/close state.Message ID: @.**@.>>

brey commented 1 year ago

I have pushed the final conda recipe. It includes instructions on how to build ones own package. At this point, I think it's better if the interested users build their own package with their own configuration. If the complexity of the modules/models improves or a way to initiate them in run time is developed we can consider conda-forge.

I don't believe there is a graceful way to build all the variants in conda-forge currently. Maybe also conda will offer some new ways going forward.

platipodium commented 1 year ago

Out of curiosity, how do you provide your own channel -c gbrey ?

brey commented 1 year ago

https://docs.anaconda.com/anacondaorg/user-guide/tasks/work-with-packages/#uploading-packages