Extending catalogs organization

[nitreb]

Hi, We would like to be able to display multiple lists of catalogs in the Onyxia sidebar. The idea is to provide an extra dimension to organize our catalogs. In our case for example, we could have themes like "IDE" catalogs, which would include tabs like "Jupyter Notebooks", "VSCodes", etc...

Would it be conceivable ton enhance the onyxia-values.yaml keys to allow such a feature? It is just an idea tho, maybe you already have something better in mind.

[odysseu]

You need to be in the onyxia chart v7. (see how to migrate from version 6 to version 7)

You would have as currently available in your onyxia.values something like this :

onyxia:
  web: ...
  api:
    region: ...
    catalogs:
      [
        {
          "id": "ide",
          "name": "Interactive services",
          "description": "IDE for datascientists.",
          "location": "https://inseefrlab.github.io/helm-charts-interactive-services",
          "status": "PROD",
          "highlightedCharts":
            [
              "jupyter-python",
              "rstudio",
              ...
              "vscode-tensorflow",
            ],
          "type": "helm",
        },
        {
          "id": "databases",
          "name": "Databases",
          "description": "Databases services.",
          "location": "https://inseefrlab.github.io/helm-charts-databases",
          "status": "PROD",
          "highlightedCharts":
            [
              "postgresql",
              "elastic",
              ...
              "metabase",
            ],
          "type": "helm",
        },
        {
          "id": "inseefrlab-helm-charts-datascience",
          "name": "Deprecated",
          "description": "Services for datascientists.",
          "location": "https://inseefrlab.github.io/helm-charts-datascience",
          "status": "PROD",
          "highlightedCharts": ["ubuntu", "openrefine"],
          "excludedCharts":
            [
              "rstudio",
              "jupyter",
              ...
              "delta-sharing-server",
            ],
          "type": "helm",
        },
        {
          "id": "automation",
          "name": "Automation",
          "description": "AIOps & MLOps services.",
          "location": "https://inseefrlab.github.io/helm-charts-automation",
          "status": "PROD",
          "highlightedCharts":
            ["argo-workflows", "mlflow", "metaflow", "argo-cd", "label-studio"],
          "type": "helm",
        },
        {
          "id": "divers",
          "name": "Experimental",
          "description": "Services divers.",
          "location": "https://inseefrlab.github.io/helm-charts-miscellaneous",
          "status": "PROD",
          "type": "helm",
        },
       ...
     ]

which renders like so :

Then you can use the LEFTBAR_LINK like in this doc : https://github.com/InseeFrLab/onyxia/blob/v7.0.0/web/.env#L309-L316

To use it like so:

onyxia:
  web:
    ...,
    HEADER_LINKS: |
     [
       {
         "label": "IDE"
         "icon": "https://.../someLogo.png", 
         "url": "/catalog/ide" 
       },
       {
         "label": "automation"
         "icon": "https://.../someLogo2.png", 
         "url": "/catalog/automation" 
       },
       {
         "label": "other charts"
         "icon": "https://.../someLogo3.png", 
         "url": "/catalog/others" 
       },
 ]
  api: ...

and have it render something like :

Then you can have each link point to a given catalog.

Bonus: you can always use the "status": "DEV", in this api:catalogs: ... to hide the catalog in the "Catalogs page"'s tabs, and only have them in your left vertical bar.

[garronej]

Thanks for the issue @nitreb and for the guidance you providied @odysseu.

And yes! It's possible in v7! And with client side routing enabled on top of that!

I'm currently producing a video tutorial about Onyxia instance configuration.

I encourage you to wait until monday to start customizing 🙏

[nitreb]

Thank you for your responses @odysseu and @garronej! As your solution is good to access catalogs via separated links, it does not offer the possibility to spread different types of running services between multiples pages.

What we were trying to express (and maybe the example in #660 is better suited), is the ability to have enough customization options to add another pair of service catalog/running services links. Those 'services' might be completely different from the ones already existing, and thus have not their place among them.

Here is a concrete example, having:

• one catalog exposing generic data science tools, categorized by IDE, databases, automation, etc.
• another catalog exposing ocean-oriented services, categorized by ocean modelling, ocean analysis, ocean forecasting, earth observation assimilation, etc.. Each of these categories having multiple kind of services, that can be concurrent (a bit like Jupyterlab is concurrent to R-Studio).

I hope it clarifies our needs!

[fcomte]

In the #660 , I understand the need of separating Jobs and Services. This kind of separation can be worth it. In this issue It's less clear and It seem that you want to have as many "simulated namespace" as the number of catalogs. This can obviously be developped but I am not very sure about the usecase and the UX.

Don't you want to filter in the "My services" the ability to filter by calatog ?

[garronej]

@nitreb What you are thinking can be acheive via the new feature that enables to override CSS or inject custom code.
We can schedule an hours to implement a POC if you are interested.

[nitreb]

First, thank you for the solution proposal.

As stated in this comment in #660, never mind the link between the two issues.

We are going to try to give your more details about our intention and use cases.

We want our community to use our datalab to build end-user applications with their own tools. While we have started proposing in the service catalog the official Onyxia service repositories and add some new services for our community, we want them to contribute by adding their own tools. So far everything is happening under "Service catalog", where we added new tabs pointing to new repositories. But as our community will also build end-user applications of different kind, we are afraid using only the current tab segregation won't scale and hence being to messy.

So basically we think we need a new level of hierarchy to discover and organize the services.

So far in our context we though about three high-level categories:

• one for datascience tools
• another one for oceaonographic tools
• and one for end-user applications We though about duplicating the two "Service catalog" and "My services" pages into several and customizable " catalog" and "My " would be a good approach but maybe it isn't. What do you think?

Links pointing to URLs with search parameters can be a good solution if we can filter by keywords/tags. Maybe using the keywords field in the Chart.yaml? But we were wondering if it's actually enough and if we don't want a solution for categorizing/ordering the running services as well.

Rationale: we (our development team and our community) think using the self services is a good place to build end-user applications before deploying them to production permanently. To this end, the versioning system and inherent ephemeral nature of services fit the need perfectly.