search

openfaas / nats-queue-worker

Queue-worker for OpenFaaS with NATS Streaming
https://docs.openfaas.com/reference/async/
MIT License
128 stars 59 forks source link

Path needs transformation before invoking via gateway_invoke #85

Closed alexellis closed 4 years ago

alexellis commented 4 years ago

The path for a service needs transformation before invoking via gateway_invoke

Expected Behaviour

If env is invoked as per:

curl gateway/async-function/env -d "test"

The path received by the function should be / whatever the value for gateway_invoke - true or false.

Current Behaviour

When in gateway_invoke true and with direct_functions set to false the whole path is pre-pended:

curl gateway/async-function/env -d "test"

The function receives a request for /async-function/env3 instead of /

Possible Solution

There are several areas where changes could be made:

Steps to Reproduce (for bugs)

1) Deploy with --set gateway.directFunctions=false and --set queueWorker.gatewayInvoke=true 2) Deploy faas-cli deploy --name env --image functions/alpine:latest --fprocess=env 3) Invoke, and use an async receiver URL to view the result i.e. run nc -l 8888 on your laptop and then add the address in the header: -H "X-Callback-Url: http://192.168.0.28:8888" to your invoke

You'll then see this incorrect path:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
fprocess=env
HOME=/home/app
Http_Accept=*/*
Http_X_Call_Id=b3aa50ae-a31d-4eb9-9987-915ef97f0ca8
Http_X_Start_Time=1577136342524082049
Http_X_Callback_Url=http://192.168.0.28:8888
Http_X_Forwarded_For=172.19.0.1:47744
Http_X_Forwarded_Host=gateway:8080
Http_User_Agent=curl/7.47.0
Http_Content_Length=0
Http_Accept_Encoding=gzip
Http_Content_Type=application/x-www-form-urlencoded
Http_Method=POST
Http_ContentLength=0
Http_Path=/async-function/env3
Http_Host=172.19.0.78:8080

Now set deploy with --set gateway.directFunctions=true and --set queueWorker.gatewayInvoke=false

Perform the same invocation, and you should see the Path as Http_Path=/

Context

This will break some functions and services, for instance figlet did not seem to work with this change.

Pinging @LucasRoesler @ewilde

LucasRoesler commented 4 years ago

/assign me

alexellis commented 4 years ago

My gut feeling is that this needs changing in the gateway code since the provider is bypassed for the publish message.

alexellis commented 4 years ago

See what you think once you can repro

LucasRoesler commented 4 years ago

After a couple of side excursions, I reproduced using 

$ kind version
kind v0.6.1 go1.13.4 darwin/amd64
$ kind create cluster
$ helm repo add openfaas https://openfaas.github.io/faas-netes/ && helm repo update
$ kubectl apply -f https://raw.githubusercontent.com/openfaas/faas-netes/master/namespaces.yml
$ helm upgrade openfaas --install openfaas/openfaas \
    --namespace openfaas  \
    --set functionNamespace=openfaas-fn \
    --set generateBasicAuth=true --set gateway.directFunctions=false --set queueWorker.gatewayInvoke=true
$ kubectl port-forward -n openfaas svc/gateway 31112:8080 &
$ faas-cli login -u admin --password $(kubectl -n openfaas get secret basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode)
$ faas-cli deploy --name pycho --image theaxer/pycho:latest --fprocess='python index.py'
$ faas-cli deploy --name env --image functions/alpine:latest --fprocess=env
$ curl http://127.0.0.1:31112/async-function/env -H "X-Callback-Url: http://gateway.openfaas:8080/function/pycho"
$ faas-cli logs pycho

2019-12-25T13:24:31Z 2019/12/25 13:24:31 stdout: 2019-12-25 13:24:31,961:DEBUG:body=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2019-12-25T13:24:31Z HOSTNAME=env-67b4b6b5fc-p5d5c
2019-12-25T13:24:31Z fprocess=env
2019-12-25T13:24:31Z KUBERNETES_PORT_443_TCP_PORT=443
2019-12-25T13:24:31Z KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
2019-12-25T13:24:31Z ENV_SERVICE_HOST=10.98.222.110
2019-12-25T13:24:31Z KUBERNETE
2019-12-25T13:24:31Z 2019/12/25 13:24:31 stdout: S_SERVICE_HOST=10.96.0.1
2019-12-25T13:24:31Z KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
2019-12-25T13:24:31Z KUBERNETES_SERVICE_PORT_HTTPS=443
2019-12-25T13:24:31Z ENV_PORT_8080_TCP=tcp://10.98.222.110:8080
2019-12-25T13:24:31Z ENV_PORT_8080_TCP_PORT=8080
2019-12-25T13:24:31Z ENV_PORT=tcp://10.98.222.110:8080
2019-12-25T13:24:31Z ENV_PORT_8080_TCP_PROTO=tcp
2019-12-25T13:24:31Z ENV_PORT_8080_TCP_AD
2019-12-25T13:24:31Z 2019/12/25 13:24:31 stdout: DR=10.98.222.110
2019-12-25T13:24:31Z KUBERNETES_SERVICE_PORT=443
2019-12-25T13:24:31Z KUBERNETES_PORT=tcp://10.96.0.1:443
2019-12-25T13:24:31Z KUBERNETES_PORT_443_TCP_PROTO=tcp
2019-12-25T13:24:31Z ENV_SERVICE_PORT=8080
2019-12-25T13:24:31Z ENV_SERVICE_PORT_HTTP=8080
2019-12-25T13:24:31Z HOME=/home/app
2019-12-25T13:24:31Z Http_User_Agent=curl/7.64.1
2019-12-25T13:24:31Z Http_Content_Length=0
2019-12-25T13:24:31Z Http_X_Forwarded_For=10.244
2019-12-25T13:24:31Z 2019/12/25 13:24:31 stdout: .0.10:54370
2019-12-25T13:24:31Z Http_X_Forwarded_Host=gateway.openfaas.svc.cluster.local:8080
2019-12-25T13:24:31Z Http_X_Start_Time=1577280271780310400
2019-12-25T13:24:31Z Http_Accept=*/*
2019-12-25T13:24:31Z Http_Accept_Encoding=gzip
2019-12-25T13:24:31Z Http_X_Call_Id=abf39f8e-60f2-4dc1-84af-ed7d25f46fd8
2019-12-25T13:24:31Z Http_X_Callback_Url=http://gateway.openfaas:8080/f
2019-12-25T13:24:31Z 2019/12/25 13:24:31 stdout: unction/pycho
2019-12-25T13:24:31Z Http_Method=POST
2019-12-25T13:24:31Z Http_ContentLength=0
2019-12-25T13:24:31Z Http_Path=/async-function/env
2019-12-25T13:24:31Z Http_Host=10.244.0.14:8080

$ helm upgrade openfaas --install openfaas/openfaas \
    --namespace openfaas  \
    --set functionNamespace=openfaas-fn \
    --set generateBasicAuth=true --set gateway.directFunctions=true --set queueWorker.gatewayInvoke=false
$ curl http://127.0.0.1:31112/async-function/env -H "X-Callback-Url: http://gateway.openfaas:8080/function/pycho"
$ faas-cli logs pycho
...
2019-12-25T14:04:20Z Http_ContentLength=0
2019-12-25T14:04:20Z Http_Path=/
2019-12-25T14:04:20Z Http_Host=env.openfaas-fn.svc.cluster.local:8080
2019-12-25T14:04:20Z
2019-12-25T14:04:20Z 2019-12-25 14:04:20,310:INFO:127.0.0.1 - - [25/Dec/2019 14:04:20] "POST / HTTP/1.1" 200 -
2019-12-25T14:04:20Z
2019-12-25T14:04:20Z 2019/12/25 14:04:20 POST / - 200 OK - ContentLength: 1023

The relevant line being 2019-12-25T13:24:31Z Http_Path=/async-function/env and 2019-12-25T14:04:20Z Http_Path=/

LucasRoesler commented 4 years ago

@alexellis per your comment about the gateway implementing this, it seems like the QueuedProxy should just always use handlers.FunctionPrefixTrimmingURLPathTransformer{}. Is there a case where the function should receive the entire url that the gateway received?

LucasRoesler commented 4 years ago

@alexellis if we always apply the function prefix transform to the queue endpoint, it seems to work correctly


$ helm upgrade openfaas --install openfaas/openfaas \
    --namespace openfaas  \
    --set functionNamespace=openfaas-fn \
    --set generateBasicAuth=true \
    --set gateway.directFunctions=true \
    --set queueWorker.gatewayInvoke=false \
    --set gateway.image=theaxer/gateway:fix-85
$ curl -XPOST http://127.0.0.1:31112/async-function/env -H "X-Callback-Url: http://gateway.openfaas:8080/function/pycho"
$ faas-cli logs pycho
....
2019-12-25T14:31:07Z 2019/12/25 14:31:07 stdout: ttp_Content_Length=0
2019-12-25T14:31:07Z Http_Accept=*/*
2019-12-25T14:31:07Z Http_X_Call_Id=075eee9f-2787-41e7-8b26-ee78a0b1f6c5
2019-12-25T14:31:07Z Http_X_Callback_Url=http://gateway.openfaas:8080/function/pycho
2019-12-25T14:31:07Z Http_X_Start_Time=1577284267024974300
2019-12-25T14:31:07Z Http_Method=POST
2019-12-25T14:31:07Z Http_ContentLength=0
2019-12-25T14:31:07Z Http_Path=/
2019-12-25T14:31:07Z Http_Host=env.o
2019-12-25T14:31:07Z 2019/12/25 14:31:07 stdout: penfaas-fn.svc.cluster.local:8080

$ helm upgrade openfaas --install openfaas/openfaas \
    --namespace openfaas  \
    --set functionNamespace=openfaas-fn \
    --set generateBasicAuth=true \
    --set gateway.directFunctions=false \
    --set queueWorker.gatewayInvoke=true \
    --set gateway.image=theaxer/gateway:fix-85

$ curl -XPOST http://127.0.0.1:31112/async-function/env -H "X-Callback-Url: http://gateway.openfaas:8080/function/pycho"
$ faas-cli logs pycho
...
2019-12-25T14:39:09Z 2019/12/25 14:39:09 stdout: 4749653810100
2019-12-25T14:39:09Z Http_Method=POST
2019-12-25T14:39:09Z Http_ContentLength=0
2019-12-25T14:39:09Z Http_Path=/
2019-12-25T14:39:09Z Http_Host=10.244.0.14:8080