Node Selector doesn't get applied on K8s platforms

[juanpablo-santos]

Description: per title, Node Selector configuration option doesn't get applied on K8s platforms, neither by setting spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector nor by applying the configuration on task execution, through the UI (the deployer.XXXXXX.kubernetes.node-selector=NAME:VALUE)

Release versions:

{
  "versions": {
    "implementation": {
      "name": "spring-cloud-dataflow-server",
      "version": "2.11.2"
    },
    "core": {
      "name": "Spring Cloud Data Flow Core",
      "version": "2.11.2"
    },
    "dashboard": {
      "name": "Spring Cloud Dataflow UI",
      "version": "3.4.2"
    },
    "shell": {
      "name": "Spring Cloud Data Flow Shell",
      "version": "2.11.2",
      "url": "https://repo.maven.apache.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.11.2/spring-cloud-dataflow-shell-2.11.2.jar"
    }
  },
  "features": {
    "streams": true,
    "tasks": true,
    "schedules": true,
    "monitoringDashboardType": "GRAFANA"
  },
  "runtimeEnvironment": {
    "appDeployer": {
      "deployerImplementationVersion": "2.11.2",
      "deployerName": "Spring Cloud Skipper Server",
      "deployerSpiVersion": "2.11.2",
      "javaVersion": "17.0.9",
      "platformApiVersion": "",
      "platformClientVersion": "",
      "platformHostVersion": "",
      "platformSpecificInfo": {
        "default": "kubernetes"
      },
      "platformType": "Skipper Managed",
      "springBootVersion": "2.7.18",
      "springVersion": "5.3.31"
    },
    "taskLaunchers": [
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      }
    ]
  },
  "monitoringDashboardInfo": {
    "url": "https://grafana.XXXXXXXXX.dom",
    "source": "default-scdf-source",
    "refreshInterval": 15
  },
  "security": {
    "isAuthentication": true,
    "isAuthenticated": true,
    "username": "YYYYYY",
    "roles": [
      "ROLE_CREATE",
      "ROLE_DEPLOY",
      "ROLE_DESTROY",
      "ROLE_MANAGE",
      "ROLE_MODIFY",
      "ROLE_SCHEDULE",
      "ROLE_VIEW"
    ]
  },
  "git": {
    "commit": "86b53e0"
  }
}

Custom apps: N/A, issue is reproduced with any task execution.

Steps to reproduce: Neither setting spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector: "NAME:VALUE" nor by applying the configuration on task execution, through the UI (the deployer.XXXXXX.kubernetes.node-selector=NAME:VALUE.

Screenshots:

Additional context: Saw PR spring-cloud/spring-cloud-dataflow#301, the configuration provided there also doesn't work

[cppwfs]

@juanpablo-santos Thank you for reporting this issue: A work around is to set the following property for node selectors: deployer.<appname>.kubernetes.deployment.nodeSelector=foo:bar

We will work on resolving this bug.

[juanpablo-santos]

Hi @cppwfs! thanks a lot for the workaround :-)

I did try the property, but instead of using nodeSelector I did use node-selector (as generated by the UI), thought they were interchangeable and didn't thought on using nodeSelector. In any case, the workaround is more than enough for us until the issue is fixed.

cheers, juan pablo

[corneil]

Thanks @juanpablo-santos for doing the legwork. We will be fixing that one since was expecting a property named with nodeSelector instead of allowing node-selector.

[juanpablo-santos]

Hi @corneil! saw the associated PR, would that cover the platform property as well? i.e., spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector also doesn't get assigned

thanks for looking into this so fast :-)