kubeflow / notebooks

Kubeflow Notebooks lets you run web-based development environments on your Kubernetes cluster by running them inside Pods.
Apache License 2.0
18 stars 11 forks source link

Notebooks 2.0 // Backend // List WorkspaceKinds #51

Closed ederign closed 1 month ago

ederign commented 2 months ago

GET /v1/workspacekinds should return all workspace kinds in the cluster.

We don't have the front end yet, so we don't know the complete payload that would be required. On this task, let's add some primary fields like I did here. We can always improve this later after the basic infra is in place.

Here is the primary crd and also some crd samples for this task.

backend git:(issue40fup) kubectl describe workspacekinds.kubeflow.org jupyterlab -n workspace-test                                    (kind-nbv2-1/default)
Name:         jupyterlab
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  kubeflow.org/v1beta1
Kind:         WorkspaceKind
Metadata:
  Creation Timestamp:  2024-09-03T19:20:05Z
  Finalizers:
    notebooks.kubeflow.org/workspacekind-protection
  Generation:        5
  Resource Version:  3301836
  UID:               27022019-2fa4-4890-add4-6c0abb9cded4
Spec:
  Pod Template:
    Container Security Context:
      Allow Privilege Escalation:  false
      Capabilities:
        Drop:
          ALL
      Run As Non Root:  true
    Culling:
      Activity Probe:
        Jupyter:
          Last Activity:     true
      Enabled:               true
      Max Inactive Seconds:  86400
    Extra Env:
      Name:   NB_PREFIX
      Value:  {{ httpPathPrefix "jupyterlab" }}
    Extra Volume Mounts:
      Mount Path:  /dev/shm
      Name:        dshm
    Extra Volumes:
      Empty Dir:
        Medium:  Memory
      Name:      dshm
    Http Proxy:
      Remove Path Prefix:  false
      Request Headers:
    Options:
      Image Config:
        Spawner:
          Default:  jupyterlab_scipy_190
        Values:
          Id:  jupyterlab_scipy_190
          Spawner:
            Description:   JupyterLab, with SciPy Packages
            Display Name:  jupyter-scipy:v1.9.0
            Hidden:        false
            Labels:
              Key:    python_version
              Value:  3.11
          Spec:
            Image:              docker.io/kubeflownotebookswg/jupyter-scipy:v1.9.0
            Image Pull Policy:  IfNotPresent
            Ports:
              Display Name:  JupyterLab
              Id:            jupyterlab
              Port:          8888
              Protocol:      HTTP
      Pod Config:
        Spawner:
          Default:  tiny_cpu
        Values:
          Id:  tiny_cpu
          Spawner:
            Description:   Pod with 0.1 CPU, 128 Mb RAM
            Display Name:  Tiny CPU
            Hidden:        false
            Labels:
              Key:    cpu
              Value:  100m
              Key:    memory
              Value:  128Mi
          Spec:
            Resources:
              Requests:
                Cpu:     100m
                Memory:  128Mi
          Id:            big_gpu
          Spawner:
            Description:   Pod with 4 CPU, 16 GB RAM, and 1 GPU
            Display Name:  Big GPU
            Hidden:        false
            Labels:
              Key:    cpu
              Value:  4000m
              Key:    memory
              Value:  16Gi
              Key:    gpu
              Value:  1
          Spec:
            Affinity:
            Node Selector:
            Resources:
              Limits:
                nvidia.com/gpu:  1
              Requests:
                Cpu:     4000m
                Memory:  16Gi
            Tolerations:
              Effect:    NoSchedule
              Key:       nvidia.com/gpu
              Operator:  Exists
    Pod Metadata:
      Annotations:
        My - Workspace - Kind - Annotation:  my-value
      Labels:
        My - Workspace - Kind - Label:  my-value
    Security Context:
      Fs Group:  100
    Service Account:
      Name:  default-editor
    Volume Mounts:
      Home:  /home/jovyan
  Spawner:
    Deprecated:           false
    Deprecation Message:  This WorkspaceKind will be removed on 20XX-XX-XX, please use another WorkspaceKind.
    Description:          A Workspace which runs JupyterLab in a Pod
    Display Name:         JupyterLab Notebook
    Hidden:               false
    Icon:
      URL:  https://jupyter.org/assets/favicons/apple-touch-icon-152x152.png
    Logo:
      URL:  https://upload.wikimedia.org/wikipedia/commons/3/38/Jupyter_logo.svg
Status:
  Pod Template Options:
    Image Config:
      Id:          jupyterlab_scipy_190
      Workspaces:  3
    Pod Config:
      Id:          tiny_cpu
      Workspaces:  3
      Id:          big_gpu
      Workspaces:  0
  Workspaces:      3
Events:            <none>
alexcreasy commented 2 months ago

Hey @ederign, I'd like to help out with this project - I'm happy to help with frontend, backend, devops stuff as is needed. Do you have some idea where I could begin?

ederign commented 1 month ago

@alexcreasy it would be awesome! Just give me a couple of days to wrap up https://github.com/kubeflow/notebooks/pull/60 . This could be a good starting point for you to work on this issue.

ederign commented 1 month ago

Hi @alexcreasy thank you so much for the interest!

I suggest you to start reading https://docs.google.com/document/d/1_zk06zebbaTBdJ8TdU07Ibky25hqHGARXjVcsp2qEnU/edit#heading=h.qt6x5supdfo4

On my local, I've run the controller end-to-end tests in the cluster to install/set up everything and commented on the lines for teardown (delete). Maybe @thesuperzapper has better instructions on how to better set up it. Please let me know if you have any questions.

After that, you can probably base your contribution on https://github.com/kubeflow/notebooks/pull/60 and https://github.com/kubeflow/notebooks/pull/61 !

ederign commented 1 month ago

@thesuperzapper can you please assign this to @alexcreasy ?

alexcreasy commented 1 month ago

@ederign thanks for the info, I haven't had a chance to take a look yet, but I should be able to put in a bit of time on this over the weekend, I'll shout if I get stuck.

ederign commented 1 month ago

Done: https://github.com/kubeflow/notebooks/pull/66/commits