"Content-Type" environment variable not working as expected with classic watchdog

[spekulatius]

Hey @LucasRoesler

Function using the PHP template. The template doesn't allow to return a content type. As a work around it's been set in the yml file:

functions:
  php7-json:
    lang: php7
    handler: ./php7-json
    image: spekulatius/php7-json:latest
    environment:
      write_debug: true
      content_type: application/json

But this doesn't work when data is posted. It ignores the environment config and uses the header instead.

Expected Behaviour

All requests should use the environment variable and return Content-Type: application/json as a header.

Current Behaviour

Note the content type:

peter@x1:/tmp/php7-json$ curl -i http://faas1:8080/function/php7-json
HTTP/1.1 200 OK
Content-Length: 11
Content-Type: application/json
Date: Mon, 07 Dec 2020 14:21:36 GMT
X-Duration-Seconds: 0.021615

{"data":""}

peter@x1:/tmp/php7-json$ curl -i http://faas1:8080/function/php7-json --data-ascii "test"
HTTP/1.1 200 OK
Content-Length: 15
Content-Type: application/x-www-form-urlencoded
Date: Mon, 07 Dec 2020 14:21:53 GMT
X-Duration-Seconds: 0.024064

{"data":"test"}

peter@x1:/tmp/php7-json$ curl -i http://faas1:8080/function/php7-json -d "test=test"
HTTP/1.1 200 OK
Content-Length: 20
Content-Type: application/x-www-form-urlencoded
Date: Mon, 07 Dec 2020 14:22:00 GMT
X-Duration-Seconds: 0.020691

{"data":"test=test"}

Possible Solution

Not sure.

Steps to Reproduce (for bugs)

See https://github.com/spekulatius/faasd-content-type-example

Context

Building an API based on existing PHP library.

Your Environment

• OS and architecture:

Ubuntu 20.04

• Versions:

Faasd instance recently started. Version was initial 0.9.5 and later upgraded to 0.9.8-4-g0d9c846.

21:38 $ faas-cli version --gateway faas1:8080
  ___                   _____           ____
 / _ \ _ __   ___ _ __ |  ___|_ _  __ _/ ___|
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \
| |_| | |_) |  __/ | | |  _| (_| | (_| |___) |
 \___/| .__/ \___|_| |_|_|  \__,_|\__,_|____/
      |_|

CLI:
 commit:  c9d284d0c5bd90415baf9130fdce55af8e5a506b
 version: 0.12.19

Gateway
 uri:     http://faas1:8080
 version: 0.19.1
 sha:     f612947df76faea0132415151f9765518f443123
 commit:  Remove deprecated Angular/material versions

Provider
 name:          faasd
 orchestration: containerd
 version:       0.9.8-4-g0d9c846
 sha:           0d9c846117cc5db7b224e990fc61575fd793ce1c
✔ /tmp/php7-json [master L|✔]

[alexellis]

The quickest repro for this is faas-cli store deploy figlet -e content_type=application/json

On Kubernetes we get a hard-coded application/json response for every request. On faasd it's whatever the input content-type was, echoed back.

I was able to rule out the gateway and watchdog by running the watchdog in Docker and by calling the provider directly on port 8081 instead of calling the gateway port 8080 on the faasd host.

The command sudo ctr -n openfaas-fn task ls / info also showed the env-var on the container as expected.

I'm suspicious of this code - https://github.com/openfaas/faas-provider/blob/ed1533b814e5dfe4a90542adff1dba992f616288/proxy/proxy.go#L228

[alexellis]

0.9.10 still has the issue (as expected):

pi@faasd-pi:~ $ curl -i http://127.0.0.1:8080/function/figlet
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: application/json
Date: Tue, 08 Dec 2020 08:33:34 GMT
X-Duration-Seconds: 0.034296

pi@faasd-pi:~ $ curl -i http://127.0.0.1:8080/function/figlet -d hi
HTTP/1.1 200 OK
Content-Length: 60
Content-Type: application/x-www-form-urlencoded
Date: Tue, 08 Dec 2020 08:33:40 GMT
X-Duration-Seconds: 0.006218

 _     _ 
| |__ (_)
| '_ \| |
| | | | |
|_| |_|_|

pi@faasd-pi:~ $ 

sudo ctr -n openfaas-fn container info figlet

 "Spec": {
        "ociVersion": "1.0.2",
        "process": {
...
            "args": [
                "fwatchdog"
            ],
            "env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "fprocess=figlet",
                "content_type=application/json"
            ],
            "cwd": "/",
....

[alexellis]

Deploying env to see what the container sees:

faas-cli deploy --name env \
  -e content_type=application/json \
  --image functions/alpine:latest-armhf \
  --fprocess env
WARNING! Communication is not secure, please consider using HTTPS. Letsencrypt.org offers free SSL/TLS certificates.

Deployed. 200 OK.
URL: http://127.0.0.1:8080/function/env

pi@faasd-pi:~ $ curl -i http://127.0.0.1:8080/function/env
HTTP/1.1 200 OK
Content-Length: 362
Content-Type: application/json
Date: Tue, 08 Dec 2020 08:35:55 GMT
X-Duration-Seconds: 0.038320

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
content_type=application/json
fprocess=env
HOME=/root
Http_Accept_Encoding=gzip
Http_X_Forwarded_For=10.62.0.1:38330
Http_X_Forwarded_Host=127.0.0.1:8080
Http_User_Agent=curl/7.64.0
Http_Accept=*/*
Http_Method=GET
Http_ContentLength=0
Http_Content_Length=0
Http_Path=/
Http_Host=10.62.1.151:8080
pi@faasd-pi:~ $ 

Curling with data from curl:

pi@faasd-pi:~ $ curl -i http://127.0.0.1:8080/function/env -d test
HTTP/1.1 200 OK
Content-Length: 417
Content-Type: application/x-www-form-urlencoded
Date: Tue, 08 Dec 2020 08:36:48 GMT
X-Duration-Seconds: 0.003313

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
content_type=application/json
fprocess=env
HOME=/root
Http_X_Forwarded_For=10.62.0.1:38368
Http_X_Forwarded_Host=127.0.0.1:8080
Http_User_Agent=curl/7.64.0
Http_Accept=*/*
Http_Accept_Encoding=gzip
Http_Content_Type=application/x-www-form-urlencoded
Http_Method=POST
Http_ContentLength=-1
Http_Content_Length=-1
Http_Path=/
Http_Host=10.62.1.151:8080

[alexellis]

Curling the container directly, bypassing the provider and gateway:

pi@faasd-pi:~ $ curl -i http://10.62.1.151:8080 -d test
HTTP/1.1 200 OK
Content-Type: application/json
X-Duration-Seconds: 0.005527
Date: Tue, 08 Dec 2020 08:37:21 GMT
Content-Length: 315

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
content_type=application/json
fprocess=env
HOME=/root
Http_User_Agent=curl/7.64.0
Http_Accept=*/*
Http_Content_Length=4
Http_Content_Type=application/x-www-form-urlencoded
Http_Method=POST
Http_ContentLength=4
Http_Path=/
Http_Host=10.62.1.151:8080

[alexellis]

/set title: "Content-Type" environment variable not working as expected with classic watchdog