OSC-Platform-023: Create a single application dashboard access

[caldeirav]

We need a simple link consolidating access to all tools under Data Commons platform, similar to https://odh.operate-first.cloud/

Right now it should ideally link to: JupyterHub, SuperSet, CloudBeaver, Trino. The page should be maintained under OS-Climate repository and easily maintainable so we can add new tools as they become available.

[MichaelTiemannOSC]

Can the links to JupyterHub drill into specific Jupyter notebooks within the JupyterHub? If so, users can directly access ITR, PhysRisk, etc.

[HumairAK]

@MichaelTiemannOSC as in a link to another JupyterHub user notebook server? No that's not supported by Jupyterhub or the ODH dashboard. There is no way for one user to access another user's notebook.

[MichaelTiemannOSC]

There are notebooks that are used for ingestion, they are all separate, and that's fine.

Maybe what I'm thinking about is that there should be a "Data Scientist's Laucher" which, from a single (Jupyter) environment is able to launch the different analytic tools that can independently access the PROD data, do various analysis, give the data scientists the opportunity to upload and download data as permitted, and that's the unified interface that these analysts see. Is that possible?

[caldeirav]

Following on this item: can we just proceed with having at least the existing dashboard used by Operate First environment (which would show links to Jupyter, CloudBeaver, Superset, Trino), maybe with an OS-Climate Data Commons logo? We raised this to make the access across easier during the demo flow and had agreed with this approach in our sprint call. @HumairAK

[caldeirav]

@HumairAK is there a pattern / example I can follow to prepare such a page consolidating access to the different tools please? I'd like to build one for the Summit demo if possible.

[HumairAK]

@caldeirav when do you need this available? We have an open pullrequest right now that we can populate with projects deployed on all operate-first clusters (sectioned by clusters). You can see a preview of it here. Right now the app routes aren't listed there, but those could be easily added. Would something like this be sufficient?

If so, we have another active PR that would make this possible for the osc-clusters here to also be listed here (if it's okay with @redmikhail).

The pr's above are ready to be merged, and we could have all this available within the coming days.

cc @4n4nd

[caldeirav]

This looks very nice for cluster monitoring but I was more thinking about something a bit more "visual" like a dedicated landing page for OS-Climate community with the OS-Climate logo, access to key systems, token retrieval service, key documentation links from one single page. It should not prevent us from populating the info for OS-Climate in the operate-first cluster page though, i guess there's value in showing this when explaining in the demo how we leverage operate first.

[eoriorda]

Humair will try and modify the ODH dashboard

[durandom]

@HumairAK can we open an issue in the ODH community and figure out why we can't use the ODH dashboard?

[HumairAK]

@durandom we have brought this up sometime ago, and did so again in their slack recently. the reason is because we deploy components in different namespaces and this is not supported (it will be in a much later release). We also want to be able to show services not in ODH, which is also not currently supported (and I'm not sure if it will be).

[eoriorda]

@HumairAK Did we get a response from the ODH slack channel. Did they ack this a feature.

[eoriorda]

If we upgrade to ODH we get the standard dashboard . We are on Version 1.2 need to be on 1.3 . @HumairAK will check with ODF group to see if this feature is available on 1.3 and the LOE .

[HeatherAck]

pending update from ODH

[HumairAK]

reached out to ODH slack here

Looks like the feature for adding custom links is not in yet, but will be available after this pr is merged.

As per @cfchase

[Humair Khan] any idea if this is on a milestone for any release? [Chris Chase] it's supposed to be reviewed and merged this week

If that's the case, I'd imagine by the time we updade odh we can probably expect this feature to be available.

[eoriorda]

Humair reached out to ODH . There is no PR that is ready for merging but its queued up for 1st Week of September.

Dependency : Patch needs to be merged 1st week of Sep ODH needs to be upgraded. 1.2 is our current version . Humair needs to to figure out the upgrade path Deploy the dashboard.

[HeatherAck]

still pending on ODH upgrade - revisit task in mid Oct, Not blocker for COP27 - lower priority

[HeatherAck]

wait for next operator version - date TBD, check w/ODH team - perform after COP27 (this will be operations centric - separate from user front end).

[HeatherAck]

Gregory from ODH team working on this. Humair will also get Ryan and Eric B up to speed on this.

[HeatherAck]

@MightyNerdEric to follow-up with Gregory to understand timeline for ODH upgrade; sync up with Humair on dashboard

[HeatherAck]

@MightyNerdEric to ping the OSC-Ops or support channel (Harshad, Marcel Hild)

[HeatherAck]

Discuss ODH implementation on Thurs 15-Dec (need to move from jupyter hub to notebook controller - must make sure users can still run their existing notebooks)

[HeatherAck]

Will discuss general use ODH implementation on 10-Jan (fork that OS-C will have to maintain). Decide if use for 1) mgmt components; 2) latest version of operator w/out Trino. Preference to start with latest version, but need to remove existing implementation from CL1.