failed to unmarshal extension properties: invalid character ':' after top-level value (404: Not Found)

wrdls commented 3 years ago

When running it against an AWS EKS 1.19 cluster I get the following error:

❯ ./bin/kubedd --target-kubernetes-version 1.20
ERR  - failed to unmarshal extension properties: invalid character ':' after top-level value (404: Not Found)

Results for cluster at version 1.19+ to 1.20
-------------------------------------------
>>>> Deprecated API Version's <<<<
...

The tool seems to work though, so I'm not sure what the error actually means.

Built from source with make build from commit 086224d8d4ec6fe2ddf753bc872250e5afcd6cb5.

pghildiyal commented 3 years ago

hi @wrdls does it have access to internet? this error comes when it is unable to download open-api schema from kubernetes github repo.

wrdls commented 3 years ago

I added some debugging in https://github.com/devtron-labs/silver-surfer/blob/main/pkg/K8sSchemaParser.go#L109 and it seems to get an incorrect version from the AWS EKS cluster.

❯ ./bin/kubedd --target-kubernetes-version 1.20
DEBUG - https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.20/api/openapi-spec/swagger.json <nil>
DEBUG - https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.19+/api/openapi-spec/swagger.json <nil>
ERR  - failed to unmarshal extension properties: invalid character ':' after top-level value (404: Not Found)
❯ curl https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.19+/api/openapi-spec/swagger.json                                                                                                              
404: Not Found%

wrdls commented 3 years ago

Instead of just trimming the suffix, it might also be an option to parse the version from GitVersion?

❯ kubectl version      
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.13-eks-8df270", GitCommit:"8df2700a72a2598fa3a67c05126fa158fd839620", GitTreeState:"clean", BuildDate:"2021-07-31T01:36:57Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.21) and server (1.19) exceeds the supported minor version skew of +/-1

ssarbadh commented 10 months ago

Just in case someone else faces - I seemed to face the same issue when doing a check for files by not specifying --source-kubernetes-version 1.26 --target-kubernetes-version 1.27

skoenig commented 8 months ago

I can reproduce the error with kubedd built from 6de142b452f36d0114b87c666a858f0812d78651 (v0.1.2). I have following outdated HPA manifest in fixtures/:

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
  labels:
    app.kubernetes.io/instance: cross-products-currency-service
  name: cross-products-currency-service-hpa
  namespace: example
spec:
  maxReplicas: 10
  metrics:
  - resource:
      name: memory
      target:
        averageUtilization: 70
        type: Utilization
    type: Resource
  - resource:
      name: cpu
      target:
        averageUtilization: 70
        type: Utilization
    type: Resource
  minReplicas: 2
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: cross-products-currency-service-deployment

If I omit --source-kubernetes-version flag like so:

./bin/kubedd --target-kubernetes-version 1.29 -d fixtures/

I also get the mentioned unmarshall error without any further output.

In contrast to this, adding the --source-kubernetes-version flag (although the help text states: "In case of directory defaults to same as target-kubernetes-version." ) yields the expected output:

./bin/kubedd --source-kubernetes-version 1.29 --target-kubernetes-version 1.29 -d fixtures

Results for file fixtures/hpa.yaml
-------------------------------------------
>>>> Newer Versions available <<<<
 Namespace   Name                                  Kind                      API Version (Current Available)   Replace With API Version (Latest Available)   Migration Status                             
 example     cross-products-currency-service-hpa   HorizontalPodAutoscaler   autoscaling/v2beta1               autoscaling/v2                                can be migrated with just apiVersion change

@pghildiyal To me, this looks like a bug, should I open a new issue?

pghildiyal commented 8 months ago

@skoenig Yes please open a new issue and thanks for debugging it

devtron-labs / silver-surfer

failed to unmarshal extension properties: invalid character ':' after top-level value (404: Not Found) #5