search

oceanmodeling / ufs-weather-model

This repo is forked from ufs-weather-model, and contains the model code and external links needed to build the UFS coastal model executable and model components, including the ROMS, FVCOM, ADCIRC and SCHISM plus WaveWatch III model components.
https://github.com/oceanmodeling/ufs-coastal-app
Other
2 stars 3 forks source link

Connect with Unified Worklow people #51

Closed janahaddad closed 1 month ago

janahaddad commented 3 months ago

Christina Holt

Need better understanding of their roadmap & their level of investment in the tool

dpsnowden commented 3 months ago

@janahaddad I'm glad to see this issue. I came to the repo today with exactly this question. I'm looking for a reference(s) that describe what's in use today, and what the prospects are for the future. We have a bunch of different approaches to orchestrating a model run across university partners, NWS operations on WCOSS, NOS operations on WCOSS etc. Who do you know that's working on this problem?

I'm curious to know if the cloud sandbox is an option to test out the pros/cons of various workflow engines.

I gather, since this issue is on the ufs-coastal roadmap, that you see it as within scope of the ufs-coastal project. I'm still trying to understand what's in/out of the current project and why.

janahaddad commented 3 months ago

Hi Derrick! Good to see you here! Yes the UFS-Coastal team intends on exploring different workflow tools very soon.

The most basic planned capability of the workflow implementation is enabling future ufs-coastal uses to deploy their own coupling configuration/application without needing to configure everything manually. To that end we're planning on first exploring the Unified Workflow tools over the next few weeks. We'd love to hear about other solutions too that you feel would be worth testing out.

I was just looking at Cloud Sandbox documentation today and noticed they are using CloudFlow -- curious to know if @uturuncoglu and/or @pvelissariou1 think this is viable for ufs-coastal to try out at some point

Re: using the sandbox to test running ufs-coastal on the cloud generally speaking, will definitely bring that up for team discussion next week.

Also have been meaning to reply to your comments/questions on the ufs-coastal architecture (#47)... I'll add my reply there!

dpsnowden commented 3 months ago

Thanks for your comments @janahaddad. Is this topic worth a deeper dive before your next sprint planning meeting? "Workflow" is one of those confusing terms many people use differently. I'm far from an expert. This is an area where we should define the requirements better, IMHO. Here are a couple considerations I'd like to throw into the mix:

Our workflow choice should:

  1. make it easier for us (i.e. NOS staff) to develop the parts of the UFS we're responsible for
  2. consider cross-office needs and collaboration (esp OCS <-> CO-OPS but increasingly involving ORR, and NCCOS)
  3. be clear to external partners
  4. be designed for multiple computing architectures (on-premises RDHPC, cloud, WCOSS2, etc)
  5. consider the existing operational suite (STOFS 2d/3d AND all of the OFS currently using COMF)
  6. be consistent (or perhaps interoperable with) what EMC uses for the larger UFS.

On a more technical note, I need guidance when considering all the options and their functions. Some focus on orchestrating containers, and others on provisioning infrastructure in the cloud. ???? ¯(°_o)/¯ Perhaps we can start with a landscape survey and get some opinions of which one works. Here are some that come to mind.

Anyway, I'd appreciate learning what you all think about this. I'm interested in this from a general UFS curiosity perspective, but I'd also like to understand if the BIL cloud sandbox can be part of this work soon. At the least, I hope those two projects pursue the SAME path.

janahaddad commented 2 months ago

@Armaghan-NOAA pointing you to Derrick's insights and suggestions above... some things to keep in mind as we move forward with workflow decisions.

janahaddad commented 1 month ago

closing this issue with #78 as Ufuk is testing