Eclipse che jupyter notebook

[capacman]

Hi, We are trying to use eclipse che with jupyter but when i tried it i saw that che-plugin-metadata-broker and che-plugin-artifacts-broker was pulled as well. From the documentation i understand their roles a little bit but for jupyter there is no theia plugin so is there a way to disable metadata-broker and artifact broker when we use a custom ide? Or should i configure my jupyter as dockerimage inside devfile and set editorFree option to true to have same behaviour? Thanks in advance.

Best Regards..

[l0rd]

@capacman plugins brokers are there to process the components in your devfile and the editor is one of those (cc @sleshchenko @amisevsk correct me if I am wrong).

Anyway you may be interested in the Jupyter support within Theia with the python VS Code extension #16818. This new approach provides a much better UX than the approach running Jupyter as a separate editor (that was just a PoC really).

@vzhukovskii when removing the status/need-triage label you should at least assign the area/ label otherwise no team will look at it.

[vzhukovs]

@vzhukovskii when removing the status/need-triage label you should at least assign the area/ label otherwise no team will look at it.

Got it. Took a note about it.

[amisevsk]

@l0rd is correct in terms of the brokers purpose; che-plugin-metadata-broker is needed to convert plugin meta.yamls into a format used by the Che server. The artifacts broker is started for every workspace (since generally the workspace uses Che-Theia) but it should do nothing as long as there are no Theia/VS Code extensions in the workspace.

[capacman]

@l0rd we talked with our data scientist and they still prefer jupyter. May be it can change in the future.

Actually i am trying to minimize workspace provision time. This is why i am asking whether disabling metadata and artifact broker is possible since they take some time. Currently i am using devfile like below and it is working fine except with a problem

metadata:
  name: jupyter-gbm
components:
  - type: dockerimage
    image: "registry.192.168.39.247.nip.io/gbm/gbmjc:0.34"
    alias: jupyter-editor
    env:
      - name: JUPYTER_NOTEBOOK_DIR
        value: /projects
      - name: JUPYTER_GATEWAY_URL
        value: http://127.0.0.1:8887
      - name: JUPYTER_GATEWAY_HTTP_USER
        value: guest
      - name: JUPYTER_GATEWAY_HTTP_PWD
        value: guest-password
      - name: JUPYTER_GATEWAY_VALIDATE_CERT
        value: "false"
      - name: LOG_LEVEL
        value: DEBUG
    endpoints:
      - name: "jupyterc"
        port: 8888
        attributes:
          type: ide
          protocol: http
          secure: 'true'
          cookiesAuthEnabled: 'true'
          discoverable: 'false'
    mountSources: true
    memoryLimit: "512M"
  - memoryLimit: 512Mi
    type: dockerimage
    volumes:
      - name: connectionfiles
        containerPath: /home/jovyan/.local/share/jupyter/runtime
    image: 'registry.192.168.39.247.nip.io/gbm/gbmeg:0.6'
    alias: enterprise-gateway
    env:
      - name: GM_KERNEL_SPEC
        value: python
  - memoryLimit: 1024Mi
    type: dockerimage
    volumes:
      - name: connectionfiles
        containerPath: /home/jovyan/.local/share/jupyter/runtime
    image: 'registry.192.168.39.247.nip.io/gbm/gbmpython:0.1'
    alias: kernel
apiVersion: 1.0.0
attributes:
  persistVolumes: "false"
  editorFree: "true"

But jupyter port 8888 is not protected by jwt proxy. I mean i opened private window to ensure there is no cookie set and pasted address from workspace response like "https://servera1kfcqa0-jwtproxy-server-4400.192.168.39.247.nip.io" and jupyter opened. Also workspace api response says cookiesAuthEnabled should be true but che jwt proxy config says otherwise:

{"links":{"self":"https://che-che.192.168.39.247.nip.io/api/workspace/workspace19f5udw4h6sd9k3w","ide":"https://che-che.192.168.39.247.nip.io/admin/jupyter-gbm","environment/statusChannel":"wss://che-che.192.168.39.247.nip.io/api/websocket","environment/outputChannel":"wss://che-che.192.168.39.247.nip.io/api/websocket"},"attributes":{"stackName":"","org.eclipse.che.runtimes_id":"runtimes6mk7semph9kh25wp","infrastructureNamespace":"che","updated":"1599753269434","created":"1599745283918"},"namespace":"admin","id":"workspace19f5udw4h6sd9k3w","temporary":false,"status":"RUNNING","runtime":{"machines":{"che-jwtproxy":{"attributes":{"memoryLimitBytes":"134217728","memoryRequestBytes":"134217728","source":"tool","cpuRequestCores":"0.0","cpuLimitCores":"0.5"},"status":"RUNNING"},"jupyter-editor":{"attributes":{"component":"jupyter-editor","memoryRequestBytes":"209715200","memoryLimitBytes":"512000000","source":"recipe","cpuLimitCores":"0.0","cpuRequestCores":"0.0"},"servers":{"jupyterc":{"url":"https://servera1kfcqa0-jwtproxy-server-4400.192.168.39.247.nip.io/","attributes":{"cookiesAuthEnabled":"true","type":"ide","secure":"true","port":"8888","discoverable":"false"},"status":"UNKNOWN"}},"status":"RUNNING"},"kernel":{"attributes":{"component":"kernel","memoryRequestBytes":"209715200","memoryLimitBytes":"1073741824","source":"recipe","cpuLimitCores":"0.0","cpuRequestCores":"0.0"},"status":"RUNNING"},"enterprise-gateway":{"attributes":{"component":"enterprise-gateway","memoryRequestBytes":"209715200","memoryLimitBytes":"536870912","source":"recipe","cpuLimitCores":"0.0","cpuRequestCores":"0.0"},"status":"RUNNING"}},"activeEnv":"default","machineToken":"eyJhbGciOiJSUzI1NiIsImtpbmQiOiJtYWNoaW5lX3Rva2VuIiwia2lkIjoid29ya3NwYWNlMTlmNXVkdzRoNnNkOWszdyJ9.eyJ3c2lkIjoid29ya3NwYWNlMTlmNXVkdzRoNnNkOWszdyIsInVpZCI6IjI0YzlmMTdkLTUxYzAtNGI0ZS1iNDk2LTUzMWQ1YzNjMWI1ZCIsImF1ZCI6IndvcmtzcGFjZTE5ZjV1ZHc0aDZzZDlrM3ciLCJuYmYiOi0xLCJ1bmFtZSI6ImFkbWluIiwiaXNzIjoid3NtYXN0ZXIiLCJleHAiOjE2MzEyODkyNjEsImlhdCI6MTU5OTc1MzI2MSwianRpIjoiODJhMWRmMzEtOWI2YS00Y2NkLThhYTAtZGQ2MzBkMjkzM2M1In0.sWJJpNkYEA2C-ZVtw_5eCanvTI_geHj4Y67XHpe3BsGK4RI43htCLH4WkGtII8rV6O-Pb02A3YAr64uc0naacJf2hJ4wI7JlBQ_kMX2XtSjQXvAqHkSoJBDfDDxmDtXVIOYRF62IARqT7ldtKCaVhuN0_IxnHqUn7qf9rfW5flYcJno4vf2ZN5llxYaau6D7zlWkXWqyE3Tp1pZk0a1oKlJQNMHjlelVGrR9NwV_yYOmKCE1zhJhgtg-0KOPkrU_9qET5hOByVyeamxPZ1liox6OYKqcIzoM3halVzZu0RJT-X-WGe-mtlc0Yy9P3Ga0_ocqUCWPsookJOLKNcG7oA"},"devfile":{"metadata":{"name":"jupyter-gbm"},"attributes":{"editorFree":"true","persistVolumes":"false"},"components":[{"mountSources":true,"endpoints":[{"name":"jupyterc","port":8888,"attributes":{"cookiesAuthEnabled":"true","type":"ide","discoverable":"false","secure":"true","protocol":"http"}}],"memoryLimit":"512M","type":"dockerimage","image":"registry.192.168.39.247.nip.io/gbm/gbmjc:0.34","alias":"jupyter-editor","env":[{"value":"/projects","name":"JUPYTER_NOTEBOOK_DIR"},{"value":"http://127.0.0.1:8887","name":"JUPYTER_GATEWAY_URL"},{"value":"guest","name":"JUPYTER_GATEWAY_HTTP_USER"},{"value":"guest-password","name":"JUPYTER_GATEWAY_HTTP_PWD"},{"value":"false","name":"JUPYTER_GATEWAY_VALIDATE_CERT"},{"value":"DEBUG","name":"LOG_LEVEL"}]},{"memoryLimit":"512Mi","type":"dockerimage","volumes":[{"name":"connectionfiles","containerPath":"/home/jovyan/.local/share/jupyter/runtime"}],"image":"registry.192.168.39.247.nip.io/gbm/gbmeg:0.6","alias":"enterprise-gateway","env":[{"value":"python","name":"GM_KERNEL_SPEC"}]},{"memoryLimit":"1024Mi","type":"dockerimage","volumes":[{"name":"connectionfiles","containerPath":"/home/jovyan/.local/share/jupyter/runtime"}],"image":"registry.192.168.39.247.nip.io/gbm/gbmpython:0.1","alias":"kernel"}],"apiVersion":"1.0.0"}}

jwtproxy:
  signer_proxy:
    enabled: false
  verifier_proxies:
  - listen_addr: ":4400"
    verifier:
      audience: "workspace19f5udw4h6sd9k3w"
      auth_cookies_enabled: false
      claims_verifiers:
      - options:
          iss: "wsmaster"
        type: "static"
      cookie_path: "/"
      excludes:
      - "/"
      key_server:
        options:
          issuer: "wsmaster"
          key_id: "workspace19f5udw4h6sd9k3w"
          public_key_path: "/che-jwtproxy-config/mykey.pub"
        type: "preshared"
      max_skew: "1m"
      max_ttl: "8800h"
      nonce_storage:
        type: "void"
      public_base_path: "/"
      upstream: "http://127.0.0.1:8888"

shouldnt auth_cookies_enabled should be true? In the mean time che is in multiuser mode with version 7.17.0 on minikube.

[sleshchenko]

@capacman there is a dedicated issue register to fix secure servers of dockerimage components https://github.com/eclipse/che/issues/17852

[capacman]

Thank you @sleshchenko i will follow it. Since initial question was about using jupyter without che-plugin-metadata-broker and che-plugin-artifacts-broker and above devfile definition make it possible i close this issue. Thank you for your efforts.