claimSize from CR is not taken into account during workspace startup

ibuziuk commented 1 year ago

Describe the bug

claimSize from CR is not taken into account during workspace startup (the default value is used)

Che version

7.54@latest

Steps to reproduce

Set the following configuration in the CR and start a workspace

storage:
      pvcStrategy: per-workspace
      perWorkspaceStrategyPvcConfig:
        claimSize: 4Gi

ERROR: PVC is provisioned with the default size 5Gi, not 4Gi as defined in the config

Expected behavior

PVC with 4Gi is provisioned

Runtime

OpenShift

Screenshots

No response

Installation method

OperatorHub

Environment

Amazon

Eclipse Che Logs

In the operator logs after update I can see the following:

I1006 13:04:38.242291 1 request.go:668] Waited for 1.048214451s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/oauth.openshift.io/v1?timeout=32s
time="2022-10-06T13:04:40Z" level=info msg="Updating existing object: v1alpha1.DevWorkspaceOperatorConfig, name: devworkspace-config"
I1006 13:04:48.290808 1 request.go:668] Waited for 2.998730784s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/apps.openshift.io/v1?timeout=32s
time="2022-10-06T13:04:54Z" level=info msg="Successfully reconciled."

Additional context

No response

dmytro-ndp commented 1 year ago

@ibuziuk : thanks for reporting!

The issue is reproduced with per-user PVC strategy as well in DS 3.2.0 built on top of Che 7.52.x, and DWO 0.16:

devEnvironments:   
  storage:
      pvcStrategy: per-user
      perUserStrategyPvcConfig:
        claimSize: 8Gi

https://issues.redhat.com/browse/CRW-3291?focusedCommentId=21052339&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-21052339

ibuziuk commented 1 year ago

Some investigation details:

DWOC is created and updated correctly e.g. after updating CR properties the resulting object has the correct values

apiVersion: controller.devfile.io/v1alpha1
config:
  workspace:
    defaultStorageSize:
      perWorkspace: 4Gi
kind: DevWorkspaceOperatorConfig
metadata:
  creationTimestamp: '2022-09-04T01:14:03Z'
  generation: 5
  managedFields:
    - apiVersion: controller.devfile.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:config':
          .: {}
          'f:workspace':
            .: {}
            'f:defaultStorageSize':
              .: {}
              'f:perWorkspace': {}
        'f:metadata':
          'f:ownerReferences':
            .: {}
            'k:{"uid":"dbdad220-cf53-4c03-a54d-efb6084c5b72"}': {}
      manager: manager
      operation: Update
      time: '2022-10-06T10:42:12Z'
  name: devworkspace-config
  namespace: dogfooding
  ownerReferences:
    - apiVersion: org.eclipse.che/v2
      blockOwnerDeletion: true
      controller: true
      kind: CheCluster
      name: eclipse-che

However, looks like there is one more DWOC in the openshift-operators namespace:

it has a pretty old creation date, and it is not clear atm why it was created:

apiVersion: controller.devfile.io/v1alpha1
config:
  enableExperimentalFeatures: true
kind: DevWorkspaceOperatorConfig
metadata:
  creationTimestamp: '2022-02-01T18:19:56Z'
  generation: 6
  managedFields:
    - apiVersion: controller.devfile.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:config': {}
      manager: Mozilla
      operation: Update
      time: '2022-02-01T18:19:56Z'
    - apiVersion: controller.devfile.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:config':
          'f:enableExperimentalFeatures': {}
      manager: kubectl-edit
      operation: Update
      time: '2022-02-04T15:38:26Z'
  name: devworkspace-operator-config
  namespace: openshift-operators

@dmytro-ndp could you clarify. on the DS 3.2.0 instance how many DWOC objects are on the cluster? Is there only one in the namespace where CR leaves or there is also one in the openshift-operators?

dmytro-ndp commented 1 year ago

@ibuziuk : there are currently 2 DWOC existed on the cluster: Screenshot from 2022-10-06 17-01-16

amisevsk commented 1 year ago

Two DevWorkspaceOperatorConfigs is expected; one is created by Eclipse Che and the other is the default DevWorkspace Operator DWOC -- someone created the main DWOC (probably me, to be honest) to enable experimental features in the cluster. One thing this does is enable logging diffs when objects need to be updated.

The devworkspace-operator-config dwoc can be removed if nobody needs the additional debug information, but it's useful to have available if there are issues starting workspaces.

AObuchow commented 1 year ago

apiVersion: controller.devfile.io/v1alpha1
config:
  workspace:
    defaultStorageSize:
      perWorkspace: 4Gi
kind: DevWorkspaceOperatorConfig
metadata:
  creationTimestamp: '2022-09-04T01:14:03Z'
  generation: 5
  managedFields:
    - apiVersion: controller.devfile.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:config':
          .: {}
          'f:workspace':
            .: {}
            'f:defaultStorageSize':
              .: {}
              'f:perWorkspace': {}
        'f:metadata':
          'f:ownerReferences':
            .: {}
            'k:{"uid":"dbdad220-cf53-4c03-a54d-efb6084c5b72"}': {}
      manager: manager
      operation: Update
      time: '2022-10-06T10:42:12Z'
  name: devworkspace-config
  namespace: dogfooding
  ownerReferences:
    - apiVersion: org.eclipse.che/v2
      blockOwnerDeletion: true
      controller: true
      kind: CheCluster
      name: eclipse-che

One thing I immediately noticed is that the DWOC is in the dogfooding namespace, whereas in the following picture we see that it is in the openshift-devspaces:

amisevsk commented 1 year ago

Testing this on our dogfooding cluster:

:heavy_check_mark: New workspaces are created with per-workspace strategy as intended.
:heavy_check_mark: New workspaces from factory URL (pasting a github repo in the create workspace box) are created with the controller.devfile.io/devworkspace-config attribute to specify a custom config. These workspaces respect the storage size configured in the CheCluster (currently 2Gi). This works for both the existing devfile and no devfile case.
:x: Workspaces created by clicking one of the sample tiles do not get the controller.devfile.io/devworkspace-config attribute and use the global defaults for storage size (5Gi)

This appears to be an issue in how the dashboard is handling workspaces. To verify if a workspace is configured to use the dwoc created by Che:

kubectl get dw <workspace-name> -o json | jq '.spec.template.attributes."controller.devfile.io/devworkspace-config"'

in addition, checking the devworkspace operator logs shows any non default values in the config used for a particular workspace:

$ kubectl logs -n openshift-operators deploy/devworkspace-controller-manager -c devworkspace-controller \
     | grep '^{' \
     | jq 'select(.devworkspace_id == "workspace1df3e7b4be9944bb")'
//...
{
  "level": "info",
  "ts": 1665069048.7958415,
  "logger": "controllers.DevWorkspace",
  "msg": "Reconciling Workspace",
  "Request.Namespace": "amisevsk-che",
  "Request.Name": "che-docs",
  "devworkspace_id": "workspace1df3e7b4be9944bb",
  "resolvedConfig": "workspace.defaultStorageSize.perWorkspace=2Gi,enableExperimentalFeatures=true"
}

ibuziuk commented 1 year ago

@olexii4 @akurinnoy folks, could you please take a look ^ seems to be the UD issue with attributes setup (I guess we use the samples as-is without pathing them with the correct attributes)

ibuziuk commented 1 year ago

yet another PR is pending merge https://github.com/eclipse-che/che-dashboard/pull/645

The following PRs should be backported to 7.52.x and 7.54.x before closing the issue:

eclipse-che / che