Incomplete information on "Workloads > kubecf" (no pods)

[svollath]

Frontend Deployment type

• [ ] Cloud Foundry Application (cf push)
• [x] Kubernetes, using a helm chart
• [ ] Docker, single container deploying all components
• [ ] npm run start
• [ ] Other (please specify below)

Backend (Jet Stream) Deployment type

• [ ] Cloud Foundry Application (cf push)
• [x] Kubernetes, using a helm chart
• [ ] Docker, single container deploying all components
• [ ] Other (please specify below)

SUSE CaaSP4 added as "CaaSP Endpoint" (Name: CaaSP4-tf4)

Expected behaviour

When I visit "Workloads > kubecf", status for pods is shown like for other workloads.

Actual behaviour

While information for kubecf is filled, it's incomplete - e.g. for pods it reports "0".

Steps to reproduce the behavior

• Use helm-3.1.3
• Install CAP-2.0-RC3 (kubecf-2.2.2) to CaaSP4
• Install Console-3.2.1-RC1
• Add CaaSP as endpoint to Stratos (CaaSP-endpoint, kubeconfig upload)
• Navigate to "Workloads > kubecf" and find pods report: "0 pods, 0 containers, 0 ready, 0 not ready"

Log output covering before error and any error statements

kubectl logs stratos-0 proxy -n stratos snippet

...
    stemcells:
    - alias: default
      os: ubuntu-xenial
      version: "621.59"
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: did not find expected comment or line break 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: line 11: could not find expected ':' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: did not find expected comment or line break 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/eirini.yaml
# Create the bits secret
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/ingress.yaml
# If the ingress feature (ingress-controller) is not enabled, create public services to allow
# public ingress to the cluster.
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/ops.yaml
# This template creates a ConfigMap for each ops file under assets/operations.
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/properties.yaml
# This ConfigMap creates an ops-file containing the user provided properties.
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/sync-integration-tests.yaml
# SITS only makes sense when using Diego, for this reason, we only enable it if Eirini is not
# enabled.
' 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "BOSHDeployment" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksJob" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksJob" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101" 
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksStatefulSet" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
WARN[Mon Jun  8 13:56:59 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174 
WARN[Mon Jun  8 13:56:59 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404} 
WARN[Mon Jun  8 13:57:00 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174 
WARN[Mon Jun  8 13:57:00 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404} 
WARN[Mon Jun  8 13:57:11 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174 
WARN[Mon Jun  8 13:57:11 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}

[richard-cox]

I had a quick look into this and saw similar failures in the backend. It looks like there's multiple issues when trying to pass helm's manfiest for KubeCf's.

• Err was: Object 'Kind' is missing in - These are issues with KubeCf, they have objects without kinds. We'd need to chase them and see if this is expected
• Err was: no kind "QuarksStatefulSet" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery - These are due to non-standard resources, not sure we can do anything about these.. would need to know at build time and add them to the schema we use to pass the manifest (see https://github.com/kubernetes/client-go/issues/193#issuecomment-594440015)
• Other errors - These could be errors with how we chop up the manifest or non-standard text in the manifest itself

To solve this we'd probably need to manually chop up the kubecf manifest returned by helm and parse in ever smaller chunks to narrow down on the issue.