openfaas / nats-queue-worker

Queue-worker for OpenFaaS with NATS Streaming
https://docs.openfaas.com/reference/async/
MIT License
129 stars 59 forks source link

queue-worker crashes if callback happens and function call failed #90

Closed andeplane closed 4 years ago

andeplane commented 4 years ago

If function call ends up giving 503, nats-queue-worker crashes with panic: runtime error: invalid memory address or nil pointer dereference when trying to do callback if X-Callback-Url is set.

Expected Behaviour

It doesn't crash.

Current Behaviour

It crashes and pod dies.

Possible Solution

See https://github.com/openfaas/nats-queue-worker/pull/89

Steps to Reproduce (for bugs)

1) Deploy OpenFaaS in GCP using marketplace (default timeout settings). 2) Create function with the following handle, based on python3 template

import time
def handle(req):
    t = 60
    time.sleep(t)
    print(f"Slept for {t} seconds")

3) Deploy function using default timeout settings faas up -f sleep.yml 4) Do an async function call with callback to e.g. https://requestbin.com and watch the kubernetes logs for the queue worker and wait for the following error queueworker_crash.txt

Context

It started crashing some functions I thought failed for another reason.

Your Environment

Kubernetes

alexellis commented 4 years ago

Thanks for the fix. Please provide concrete instructions in the steps section. We shouldn't have any ambiguity here.

andeplane commented 4 years ago

Yep, will do.

alexellis commented 4 years ago

Fixed via #89 thank you for the PR. please give exact repeatable steps for future issues 👍