Spin app with Redis trigger stuck in crash loop with no useful logs

[calebschoepp]

I'm having trouble running a Spin app with the Redis trigger. The app is stuck in a crash loop. No Spin logs are being produced. Using the steps in #197 the last containerd-shim-spin log line I see is >>> running spin trigger.

My app source code is as follows:

//go:build ignore

package main

import (
    "fmt"

    "github.com/fermyon/spin/sdk/go/v2/redis"
)

func init() {
    // redis.Handle() must be called in the init() function.
    redis.Handle(func(payload []byte) error {
        fmt.Println("Payload::::")
        fmt.Println(string(payload))
        return nil
    })
}

// main functiion must be included for the compiler but is not executed.
func main() {}

spin_manifest_version = 2

[application]
name = "order-processor"
version = "0.1.0"
authors = ["Caleb Schoepp <caleb.schoepp@fermyon.com>"]
description = "Process orders off of redis queue"

[application.trigger.redis]
address = "redis://redis-master.default.svc.cluster.local:6379"

[[trigger.redis]]
channel = "orders"
component = "order-processor"

[component.order-processor]
source = "main.wasm"
allowed_outbound_hosts = []
[component.order-processor.build]
command = "tinygo build -target=wasi -gc=leaking -no-debug -o main.wasm main.go"

• I installed Redis into my cluster via helm install redis bitnami/redis.

• I'm running on a local k3d cluster.

  My hunch is that I'm configuring the Redis trigger address incorrectly, but I'm not unsure how to debug further because I'm not getting any useful logs. Is there a way to turn on more verbose tracing/logging?

[Mossaka]

Could you please provide containerd logs?

[calebschoepp]

Well this is embarrassing. I went back to grab containerd logs for you and found the solution. Turns out it there was a log line I had missed:

time="2024-01-19T19:32:57.498518805Z" level=info msg="error running start function: Redis trigger failed to connect to redis://localhost:6379"

This made me realize I hadn't been rerunning spin build && spin registry push when I was messing around with the Redis address. After re-building and pushing the app I ran into:

time="2024-01-19T19:33:25.090823054Z" level=info msg="error running start function: NOAUTH: Authentication required."

I resolved this by adding the password of the Redis server to the URL and then it worked as expected.

I guess that I just didn't notice the relevant log line the first time around. Related to #198 though, I wonder if there is any way that the containerd logs can be made more accessible. Execing into the node isn't very convenient or obvious.

@Mossaka it'd be great to hear if you have any thoughts on improving the findability of the containerd logs but otherwise I'm closing this issue.

[Mossaka]

Oh thanks!

Related to https://github.com/deislabs/containerd-wasm-shims/issues/198 though, I wonder if there is any way that the containerd logs can be made more accessible. Execing into the node isn't very convenient or obvious.

yes, stuff like this should go into the pod log.

[calebschoepp]

yes, stuff like this should go into the pod log.

Is there a clear path to getting these containerd logs into the pod logs?

Do you want me to write something up in #198 tracking this?

[Mossaka]

Do you want me to write something up in https://github.com/deislabs/containerd-wasm-shims/issues/198 tracking this?

That's okay. I can just re-open this issue to track.

Is there a clear path to getting these containerd logs into the pod logs?

The trigger hook provided by spin runtime may help? I am not entirely clear though.