solo-io / wasm

Web Assembly tools and SDKs for extending cloud-native infrastructure
Apache License 2.0
305 stars 39 forks source link

wasme-operator #236

Closed axelardu closed 3 years ago

axelardu commented 3 years ago

Describe the bug Wasme-operator is not picking up filterDeployment CRD.

To Reproduce Steps to reproduce the behavior: Following the Running filters in Production tutorial:

  1. https://docs.solo.io/web-assembly-hub/latest/tutorial_code/wasme_operator/

  2. cat <<EOF | kubectl apply -f -
    apiVersion: wasme.io/v1
    kind: FilterDeployment
    metadata:
    name: bookinfo-custom-filter
    namespace: bookinfo
    spec:
    deployment:
    istio:
      kind: Deployment
    filter:
    config: '{"name":"hello","value":"world"}'
    image: webassemblyhub.io/ilackarms/istio-test:1.5.0-0
    EOF
  3. check the wasm-operator logs

{"level":"info","ts":"2021-02-09T17:00:06.467Z","caller":"operator/operator.go:71","msg":"started wasme version dev"} {"level":"info","ts":1612890007.2205408,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":9091"} {"level":"info","ts":1612890007.22161,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"} {"level":"info","ts":1612890007.2218785,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"wasme","source":"kind source: /, Kind="} {"level":"info","ts":1612890196.9221773,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"wasme"} {"level":"info","ts":1612890196.9222035,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"wasme","worker count":1}

Expected behavior wasme-operator picks up the filterDeployment CRD and shows this in the logs.

Additional context

Sodman commented 3 years ago

This does indeed appear to be a bug. I've reproduced on my end with Istio 1.8.2 and a fresh cpp filter created from wasme 0.0.32 targeting Istio 1.8.x. My gut says that Istio has updated their underlying version of Envoy to a newer version, and the "config" field has moved places in the protobuf - which is likely what's causing the failure here. I'm thinking this is the source of the issue because if you remove the "config" field from your FilterDeployment you'll notice suddenly a lot more action in your cache and operator pods. (You might need to bounce cache if it's given up on retries).

Need to confirm and then if true, figure out exactly which version of Istio this changed in and potentially add separate logic for versions before/after that change.

Sodman commented 3 years ago

After further investigation, this appears to be an issue with our docs being out of date.

The notable change here is that the config field has changed from a string to a google.protobuf.Any.

The correct way to configure a FilterDeployment looks like this:

apiVersion: wasme.io/v1
kind: FilterDeployment
metadata:
  labels:
    app: wasme-test-app
    app.kubernetes.io/name: wasme-test-app
  name: myfilter
  namespace: bookinfo
spec:
  deployment:
    istio:
      kind: Deployment
  filter:
    config:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: world
    image: webassemblyhub.io/sodman/assemblyscript-example:v0.4

Context: I ran our e2e suite, which tests the wasme operator with istio (among other things), and it passed for multiple versions of Istio, see here - https://github.com/solo-io/wasm/pull/237/checks?check_run_id=1866206727 where we've tested 1.8.0-1.8.3 inclusive.

Our e2e tests use this example filter: https://github.com/solo-io/wasm/blob/master/tools/wasme/cli/test/e2e/operator/test_filter.yaml#L18

axelardu commented 3 years ago

Thanks for the quick investigation and update. I see that the docs are already updated!