nil pointer when listing workflow archive using HTTP without list options

[djanjic]

Pre-requisites

• [X] I have double-checked my configuration
• [X] I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• [X] I have searched existing issues and could not find a match for this bug
• [X] I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

Listing the workflow archive using HTTP without list options is failing.

Without list options:

~ curl -k "http://localhost:2746/api/v1/archived-workflows"
{"code":13,"message":"runtime error: invalid memory address or nil pointer dereference"}%

With list options:

 ~ curl -k "http://localhost:2746/api/v1/archived-workflows?listOptions.limit=5"
{"metadata":{},"items":null}%

The same thing is happening with Argo CLI

 ~ ARGO_HTTP1=true ARGO_SERVER=localhost:2746 ARGO_TOKEN='' ARGO_SECURE=false argo archive list
FATA[2024-10-24T10:56:52.314Z] rpc error: code = Internal desc = runtime error: invalid memory address or nil pointer dereference

gRPC is working well

 ~ ARGO_SERVER=localhost:2746 ARGO_TOKEN='' ARGO_SECURE=false argo archive list
No workflows found

Logs from Argo Server:

time="2024-10-24T10:53:44.493Z" level=debug msg=CanI name= namespace=argo resource=workflows verb=list
time="2024-10-24T10:53:44.496Z" level=debug msg=CanI name= namespace=argo resource=workflows status="{true false  }" verb=list
time="2024-10-24T10:53:44.496Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ListWorkflows grpc.service=workflow.WorkflowService grpc.start_time="2024-10-24T10:53:44+02:00" grpc.time_ms=3.616 span.kind=server system=grpc
time="2024-10-24T10:53:44.497Z" level=info duration=4.151667ms method=GET path=/api/v1/workflows/argo size=54 status=0
time="2024-10-24T10:53:44.520Z" level=debug msg="Piping events to channel"
time="2024-10-24T10:53:53.309Z" level=error msg="Recovered from panic: runtime error: invalid memory address or nil pointer dereference\ngoroutine 1144 [running]:\nruntime/debug.Stack()\n\t/opt/homebrew/opt/go/libexec/src/runtime/debug/stack.go:26 +0x64\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4.1()\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:25 +0x48\npanic({0x10399c460?, 0x1057b4a50?})\n\t/opt/homebrew/opt/go/libexec/src/runtime/panic.go:785 +0x124\ngithub.com/argoproj/argo-workflows/v3/server/workflowarchive.(*archivedWorkflowServer).ListArchivedWorkflows(0x14000a024b0, {0x103ec9bc0, 0x1400147d590}, 0x10312de00?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/workflowarchive/archived_workflow_server.go:44 +0x48\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler.func1({0x103ec9bc0?, 0x1400147d590?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:737 +0xd0\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.SetVersionHeaderUnaryServerInterceptor.func6({0x103ec9bc0, 0x1400147d590}, {0x103d4d4a0?, 0x1400147be50?}, 0x1400000fc50?, 0x14000864218?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:64 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d590?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.RatelimitUnaryServerInterceptor.func5({0x103ec9bc0, 0x1400147d590}, {0x103d4d4a0, 0x1400147be50}, 0x14000484d40, 0x14000484d60)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:113 +0x11c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d590?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/auth.(*gatekeeper).UnaryServerInterceptor.func1({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0, 0x1400147be50}, 0x14000a03820?, 0x14000484dc0)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/auth/gatekeeper.go:98 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/util/grpc.init.func1({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0?, 0x1400147be50?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:53 +0x34\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0?, 0x1400147be50?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:29 +0x70\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d440?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-middleware/logging/logrus.UnaryServerInterceptor.func1({0x103ec9bc0, 0x1400147d350}, {0x103d4d4a0, 0x1400147be50}, 0x14000484d40, 0x14000484e80)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/logging/logrus/server_interceptors.go:31 +0xbc\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d350?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-prometheus.init.(*ServerMetrics).UnaryServerInterceptor.func3({0x103ec9bc0, 0x1400147d350}, {0x103d4d4a0, 0x1400147be50}, 0x18?, 0x14000484ee0)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400147d350?}, {0x103d4d4a0?, 0x1400147be50?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7({0x103ec9bc0, 0x1400147d350}, {0x103d4d4a0, 0x1400147be50}, 0x78?, 0x103d00660?)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34 +0xb4\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler({0x103bd0d80, 0x14000a024b0}, {0x103ec9bc0, 0x1400147d350}, 0x14000d03480, 0x14000a027e0)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:739 +0x148\ngoogle.golang.org/grpc.(*Server).processUnaryRPC(0x14000329000, {0x103ec9bc0, 0x1400147d2c0}, {0x103ed99c0, 0x14000f44000}, 0x14000fa2fc0, 0x14000a02d50, 0x1057eaca0, 0x0)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1385 +0xb40\ngoogle.golang.org/grpc.(*Server).handleStream(0x14000329000, {0x103ed99c0, 0x14000f44000}, 0x14000fa2fc0)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1796 +0xc00\ngoogle.golang.org/grpc.(*Server).serveStreams.func2.1()\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1029 +0x84\ncreated by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 222\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1040 +0x13c\n"
time="2024-10-24T10:53:53.309Z" level=error msg="finished unary call with code Internal" error="rpc error: code = Internal desc = runtime error: invalid memory address or nil pointer dereference" grpc.code=Internal grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:53:53+02:00" grpc.time_ms=0.987 span.kind=server system=grpc
time="2024-10-24T10:53:53.309Z" level=info duration=1.856375ms method=GET path=/api/v1/archived-workflows size=88 status=500
time="2024-10-24T10:54:28.285Z" level=info msg="Alloc=17457 TotalAlloc=44513 Sys=31633 NumGC=9 Goroutines=190"
time="2024-10-24T10:55:45.388Z" level=error msg="Recovered from panic: runtime error: invalid memory address or nil pointer dereference\ngoroutine 1234 [running]:\nruntime/debug.Stack()\n\t/opt/homebrew/opt/go/libexec/src/runtime/debug/stack.go:26 +0x64\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4.1()\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:25 +0x48\npanic({0x10399c460?, 0x1057b4a50?})\n\t/opt/homebrew/opt/go/libexec/src/runtime/panic.go:785 +0x124\ngithub.com/argoproj/argo-workflows/v3/server/workflowarchive.(*archivedWorkflowServer).ListArchivedWorkflows(0x14000a024b0, {0x103ec9bc0, 0x14001685290}, 0x10312de00?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/workflowarchive/archived_workflow_server.go:44 +0x48\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler.func1({0x103ec9bc0?, 0x14001685290?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:737 +0xd0\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.SetVersionHeaderUnaryServerInterceptor.func6({0x103ec9bc0, 0x14001685290}, {0x103d4d4a0?, 0x140008645a0?}, 0x1400000fc50?, 0x14000864218?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:64 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685290?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.RatelimitUnaryServerInterceptor.func5({0x103ec9bc0, 0x14001685290}, {0x103d4d4a0, 0x140008645a0}, 0x1400052cd00, 0x1400052cd60)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:113 +0x11c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685290?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/auth.(*gatekeeper).UnaryServerInterceptor.func1({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0, 0x140008645a0}, 0x14000a03820?, 0x1400052cdc0)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/auth/gatekeeper.go:98 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/util/grpc.init.func1({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0?, 0x140008645a0?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:53 +0x34\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0?, 0x140008645a0?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:29 +0x70\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685140?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-middleware/logging/logrus.UnaryServerInterceptor.func1({0x103ec9bc0, 0x14001685050}, {0x103d4d4a0, 0x140008645a0}, 0x1400052cd00, 0x1400052cee0)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/logging/logrus/server_interceptors.go:31 +0xbc\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685050?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-prometheus.init.(*ServerMetrics).UnaryServerInterceptor.func3({0x103ec9bc0, 0x14001685050}, {0x103d4d4a0, 0x140008645a0}, 0x18?, 0x1400052cf40)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x14001685050?}, {0x103d4d4a0?, 0x140008645a0?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7({0x103ec9bc0, 0x14001685050}, {0x103d4d4a0, 0x140008645a0}, 0x78?, 0x103d00660?)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34 +0xb4\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler({0x103bd0d80, 0x14000a024b0}, {0x103ec9bc0, 0x14001685050}, 0x14000da2a00, 0x14000a027e0)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:739 +0x148\ngoogle.golang.org/grpc.(*Server).processUnaryRPC(0x14000329000, {0x103ec9bc0, 0x14001684fc0}, {0x103ed99c0, 0x14000f44000}, 0x1400175e000, 0x14000a02d50, 0x1057eaca0, 0x0)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1385 +0xb40\ngoogle.golang.org/grpc.(*Server).handleStream(0x14000329000, {0x103ed99c0, 0x14000f44000}, 0x1400175e000)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1796 +0xc00\ngoogle.golang.org/grpc.(*Server).serveStreams.func2.1()\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1029 +0x84\ncreated by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 222\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1040 +0x13c\n"
time="2024-10-24T10:55:45.388Z" level=error msg="finished unary call with code Internal" error="rpc error: code = Internal desc = runtime error: invalid memory address or nil pointer dereference" grpc.code=Internal grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:55:45+02:00" grpc.time_ms=0.596 span.kind=server system=grpc
time="2024-10-24T10:55:45.388Z" level=info duration=1.521084ms method=GET path=/api/v1/archived-workflows size=88 status=500
time="2024-10-24T10:56:48.391Z" level=debug msg=CanI name= namespace=argo resource=workflows verb=list
time="2024-10-24T10:56:48.393Z" level=debug msg=CanI name= namespace=argo resource=workflows status="{true false  }" verb=list
time="2024-10-24T10:56:48.393Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:56:48+02:00" grpc.time_ms=2.344 span.kind=server system=grpc
time="2024-10-24T10:56:48.393Z" level=info duration=3.081791ms method=GET path=/api/v1/archived-workflows size=28 status=0
time="2024-10-24T10:56:52.313Z" level=error msg="Recovered from panic: runtime error: invalid memory address or nil pointer dereference\ngoroutine 1247 [running]:\nruntime/debug.Stack()\n\t/opt/homebrew/opt/go/libexec/src/runtime/debug/stack.go:26 +0x64\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4.1()\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:25 +0x48\npanic({0x10399c460?, 0x1057b4a50?})\n\t/opt/homebrew/opt/go/libexec/src/runtime/panic.go:785 +0x124\ngithub.com/argoproj/argo-workflows/v3/server/workflowarchive.(*archivedWorkflowServer).ListArchivedWorkflows(0x14000a024b0, {0x103ec9bc0, 0x1400130ac00}, 0x10312de00?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/workflowarchive/archived_workflow_server.go:44 +0x48\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler.func1({0x103ec9bc0?, 0x1400130ac00?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:737 +0xd0\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.SetVersionHeaderUnaryServerInterceptor.func6({0x103ec9bc0, 0x1400130ac00}, {0x103d4d4a0?, 0x140000d5040?}, 0x1400000fc50?, 0x14000864218?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:64 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130ac00?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.RatelimitUnaryServerInterceptor.func5({0x103ec9bc0, 0x1400130ac00}, {0x103d4d4a0, 0x140000d5040}, 0x1400052d520, 0x1400052d540)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:113 +0x11c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130ac00?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/auth.(*gatekeeper).UnaryServerInterceptor.func1({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0, 0x140000d5040}, 0x14000a03820?, 0x1400052d580)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/server/auth/gatekeeper.go:98 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/util/grpc.init.func1({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0?, 0x140000d5040?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:53 +0x34\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.PanicLoggerUnaryServerInterceptor.func4({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0?, 0x140000d5040?}, 0x0?, 0x0?)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/util/grpc/interceptor.go:29 +0x70\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130aa20?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-middleware/logging/logrus.UnaryServerInterceptor.func1({0x103ec9bc0, 0x1400130a900}, {0x103d4d4a0, 0x140000d5040}, 0x1400052d520, 0x1400052d6c0)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/logging/logrus/server_interceptors.go:31 +0xbc\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130a900?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/grpc-ecosystem/go-grpc-prometheus.init.(*ServerMetrics).UnaryServerInterceptor.func3({0x103ec9bc0, 0x1400130a900}, {0x103d4d4a0, 0x140000d5040}, 0x18?, 0x1400052d6e0)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x74\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7.1.1({0x103ec9bc0?, 0x1400130a900?}, {0x103d4d4a0?, 0x140000d5040?})\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3c\ngithub.com/argoproj/argo-workflows/v3/server/apiserver.(*argoServer).newGRPCServer.ChainUnaryServer.func7({0x103ec9bc0, 0x1400130a900}, {0x103d4d4a0, 0x140000d5040}, 0x78?, 0x103d00660?)\n\t/Users/darkojanjic/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34 +0xb4\ngithub.com/argoproj/argo-workflows/v3/pkg/apiclient/workflowarchive._ArchivedWorkflowService_ListArchivedWorkflows_Handler({0x103bd0d80, 0x14000a024b0}, {0x103ec9bc0, 0x1400130a900}, 0x14000da3d80, 0x14000a027e0)\n\t/Users/darkojanjic/Workspace/pipekit/argo-workflows/pkg/apiclient/workflowarchive/workflow-archive.pb.go:739 +0x148\ngoogle.golang.org/grpc.(*Server).processUnaryRPC(0x14000329000, {0x103ec9bc0, 0x1400130a810}, {0x103ed99c0, 0x14000f44000}, 0x1400175eea0, 0x14000a02d50, 0x1057eaca0, 0x0)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1385 +0xb40\ngoogle.golang.org/grpc.(*Server).handleStream(0x14000329000, {0x103ed99c0, 0x14000f44000}, 0x1400175eea0)\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1796 +0xc00\ngoogle.golang.org/grpc.(*Server).serveStreams.func2.1()\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1029 +0x84\ncreated by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 222\n\t/Users/darkojanjic/go/pkg/mod/google.golang.org/grpc@v1.61.0/server.go:1040 +0x13c\n"
time="2024-10-24T10:56:52.313Z" level=error msg="finished unary call with code Internal" error="rpc error: code = Internal desc = runtime error: invalid memory address or nil pointer dereference" grpc.code=Internal grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:56:52+02:00" grpc.time_ms=0.473 span.kind=server system=grpc
time="2024-10-24T10:56:52.314Z" level=info duration="941.5µs" method=GET path=/api/v1/archived-workflows size=88 status=500
time="2024-10-24T10:58:06.332Z" level=debug msg=CanI name= namespace=argo resource=workflows verb=list
time="2024-10-24T10:58:06.334Z" level=debug msg=CanI name= namespace=argo resource=workflows status="{true false  }" verb=list
time="2024-10-24T10:58:06.334Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:58:06+02:00" grpc.time_ms=2.051 span.kind=server system=grpc
time="2024-10-24T10:58:48.286Z" level=debug msg=CanI name= namespace= resource=workflows verb=list
time="2024-10-24T10:58:48.289Z" level=debug msg=CanI name= namespace= resource=workflows status="{true false  }" verb=list
time="2024-10-24T10:58:48.289Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ListArchivedWorkflows grpc.service=workflowarchive.ArchivedWorkflowService grpc.start_time="2024-10-24T10:58:48+02:00" grpc.time_ms=2.913 span.kind=server system=grpc
time="2024-10-24T10:58:48.289Z" level=info duration=3.620375ms method=GET path=/api/v1/archived-workflows size=28 status=0
time="2024-10-24T10:59:28.280Z" level=info msg="Alloc=12328 TotalAlloc=51932 Sys=35729 NumGC=12 Goroutines=189"

Version(s)

latest, 2cc6b32df3a5527c3bb54d13ebdcf74c6318c3aa

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

-

Logs from the workflow controller

-

Logs from in your workflow's wait container

-

[agilgur5]

If my tracing is correct, it looks like this is a regression that dates back to 3.5.7's SQLite addition, these lines specifically: https://github.com/argoproj/argo-workflows/pull/12736#discussion_r1815439059.

So this will need a backport to 3.5.x