search

guimou / odh-highlander

Tools to leverage Lmod in the ODH ecosystem
MIT License
13 stars 5 forks source link

Can Highlander solve OS-Climate library loading issues? #4

Open MichaelTiemannOSC opened 2 years ago

MichaelTiemannOSC commented 2 years ago

After discussing https://github.com/os-climate/physrisk/issues/86 we were pointed to this project as providing a more elegant solution than building infinitely many slightly differently customized notebooks. Is the Highlander approach ready and able to address the problems identified in the above issue?

guimou commented 2 years ago

Hi @MichaelTiemannOSC ! Yes, this is definitely a good use case for ODH-Highlander. Even more, it's one I use when presenting the project: on large scale environment, with many users, each of them need/want their own particular library, with more than often this one, or a dependancy, being something that cannot be installed with pip inside a notebook. We talked in May with Marcel about installing Highlander on OperateFirst as a PoC. Now that we have a good use case to validate the real-world viability, I guess it's time for action! Let me ping him and see what we can do.

MichaelTiemannOSC commented 2 years ago

Great news. Please note that we have onboarded some LinuxFoundation staff to manage and support OperateFirst for OS-Climate. They are copied on the initial issue. Because they are still ramping up, they will definitely need guidance if this touches a lot of unfamiliar OperateFirst territory, but if it's just a question of adding a line to https://github.com/operate-first/apps/blob/master/kfdefs/overlays/osc/osc-cl2/kustomization.yaml then we don't necessarily need to wait for Marcel.

guimou commented 2 years ago

Lol! If only it was a line to add... No, there is tons of work to do, I need shared storage setup in a very specific way, we need to build specific libraries and images depending on what you want to do... It's a least a couple of months of work (elapse) before being able to test anything for real.

MichaelTiemann commented 2 years ago

Just checking in...this still looks like best approach among many I've seen.

guimou commented 2 years ago

If you want to follow-up onboarding: https://github.com/operate-first/support/issues/617