search

openziti / ziti

The parent project for OpenZiti. Here you will find the executables for a fully zero trust, application embedded, programmable network @OpenZiti
https://openziti.io
Apache License 2.0
2.69k stars 153 forks source link

Create / Get of External JWT Signer - getting errors in controller logs #778

Closed Russell-Allen closed 2 years ago

Russell-Allen commented 2 years ago

I issued a POST to /edge/management/v1/external-jwt-signers with body:

{
    "name": "My Fourth Auth",
    "enabled": false,
    "issuer": "MOP-4",
    "audience": "ziti-of-course",
    "jwksEndpoint": "https://netfoundry-sandbox.auth0.com/.well-known/jwks.json"
}

to a local Ziti Controller version 0.25.13, and received a 201 Created response. The Controller logged:

[1136.331]   ERROR fabric/controller/models.(*BaseEntityManager).ValidateNameOnCreate: entity of type *persistence.ExternalJwtSigner is named, but store doesn't have name index
[1136.968]   ERROR edge/controller/model.(*AuthModuleExtJwt).addSigner: {jwksEndpoint=[0xc0018bc790] id=[6.d5j.3Qqk] name=[My Fourth Auth] hasCertPem=[false] error=[could not resolve jwks endpoint: invalid content type, expected application/json]} could not resolve signer cert/jwks

I am pretty sure the entity exists in the DB, because I can delete it. However, if I try to read it (get or find), I receive a 500 server error with no response body. The Controller logs:

[1391.044]   ERROR fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1: panic caught by timeout next: runtime error: invalid memory address or nil pointer dereference
goroutine 6411 [running]:
github.com/openziti/foundation/util/debugz.generateStack(0x2000, 0xe0?)
    github.com/openziti/foundation@v0.17.28/util/debugz/stack.go:39 +0x4a
github.com/openziti/foundation/util/debugz.GenerateLocalStack(...)
    github.com/openziti/foundation@v0.17.28/util/debugz/stack.go:34
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1()
    github.com/openziti/fabric@v0.18.5/controller/api/timeouts.go:75 +0xfa
panic({0x20fab20, 0x37b43f0})
    runtime/panic.go:838 +0x207
github.com/openziti/edge/controller/model.(*ExternalJwtSigner).fillFrom(0xc0011d23c0, {0x3fe0000000000000?, 0xc001da242d?}, 0xa?, {0x281b100?, 0xc0013f75f0?})
    github.com/openziti/edge@v0.21.253/controller/model/external_jwt_signer_model.go:144 +0x125
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntityInTx(0xc001cde840, 0xc000966ac0?, {0xc001da242d, 0xa}, {0x28202f8, 0xc0011d23c0})
    github.com/openziti/edge@v0.21.253/controller/model/base_manager.go:310 +0x183
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity.func1(0x0?)
    github.com/openziti/edge@v0.21.253/controller/model/base_manager.go:296 +0x33
go.etcd.io/bbolt.(*DB).View(0x224bd60?, 0xc00192a7e0)
    go.etcd.io/bbolt@v1.3.6/db.go:772 +0x82
github.com/openziti/fabric/controller/db.(*Db).View(0x7f43e4072601?, 0xc002060d50?)
    github.com/openziti/fabric@v0.18.5/controller/db/db.go:64 +0x1c
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity(0xc001cde840, {0xc001da242d, 0xa}, {0x28202f8?, 0xc0011d23c0})
    github.com/openziti/edge@v0.21.253/controller/model/base_manager.go:295 +0xe6
github.com/openziti/edge/controller/model.(*baseEntityManager).BaseLoad(0xc001cde840, {0xc001da242d, 0xa})
    github.com/openziti/edge@v0.21.253/controller/model/base_manager.go:81 +0x5c
github.com/openziti/edge/controller/internal/routes.DetailWithHandler.func1(0xe12c07?, {0xc001da242d?, 0x218dea0?})
    github.com/openziti/edge@v0.21.253/controller/internal/routes/base_router.go:199 +0x42
github.com/openziti/edge/controller/internal/routes.Detail(0xc002398000, 0x27f0620?)
    github.com/openziti/edge@v0.21.253/controller/internal/routes/base_router.go:218 +0x1a4
github.com/openziti/edge/controller/internal/routes.DetailWithHandler(0x0?, 0xc0020612b0?, {0x2820348?, 0xc001cde840?}, 0x200b080?)
    github.com/openziti/edge@v0.21.253/controller/internal/routes/base_router.go:198 +0x5d
github.com/openziti/edge/controller/internal/routes.(*ExternalJwtSignerRouter).Detail(0xc002389014?, 0x280a020?, 0x262ded0?)
    github.com/openziti/edge@v0.21.253/controller/internal/routes/external_jwt_signer_router.go:83 +0x35
github.com/openziti/edge/controller/env.(*AppEnv).IsAllowed.func1({0x28165b0, 0xc0028c1ce0}, {0x280a020, 0x262ded0})
    github.com/openziti/edge@v0.21.253/controller/env/appenv.go:596 +0x38d
github.com/go-openapi/runtime/middleware.ResponderFunc.WriteResponse(0x20e0c20?, {0x28165b0?, 0xc0028c1ce0?}, {0x280a020?, 0x262ded0?})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:69 +0x3d
github.com/go-openapi/runtime/middleware.(*Context).Respond(0xc0020812f0, {0x28165b0?, 0xc0028c1ce0}, 0xc0030b0d00, {0xc0018eaf80?, 0x1, 0x1}, 0xc0030b0a00, {0x21169a0, 0xc0025a7b00})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:510 +0x59a
github.com/openziti/edge/rest_management_api_server/operations/external_jwt_signer.(*DetailExternalJWTSigner).ServeHTTP(0xc001ea4a38, {0x28165b0, 0xc0028c1ce0}, 0xc0030b0d00)
    github.com/openziti/edge@v0.21.253/rest_management_api_server/operations/external_jwt_signer/detail_external_jwt_signer.go:93 +0x2ee
github.com/go-openapi/runtime/middleware.NewOperationExecutor.func1({0x28165b0, 0xc0028c1ce0}, 0xc0030b0d00)
    github.com/go-openapi/runtime@v0.24.1/middleware/operation.go:28 +0x59
net/http.HandlerFunc.ServeHTTP(0xc002061b40?, {0x28165b0?, 0xc0028c1ce0?}, 0x0?)
    net/http/server.go:2084 +0x2f
github.com/go-openapi/runtime/middleware.NewRouter.func1({0x28165b0, 0xc0028c1ce0}, 0xc0030b0700)
    github.com/go-openapi/runtime@v0.24.1/middleware/router.go:78 +0x257
net/http.HandlerFunc.ServeHTTP(0xc002061c18?, {0x28165b0?, 0xc0028c1ce0?}, 0x7f43e4097b88?)
    net/http/server.go:2084 +0x2f
github.com/go-openapi/runtime/middleware.Redoc.func1({0x28165b0, 0xc0028c1ce0}, 0xa?)
    github.com/go-openapi/runtime@v0.24.1/middleware/redoc.go:72 +0x242
net/http.HandlerFunc.ServeHTTP(0xc00192a090?, {0x28165b0?, 0xc0028c1ce0?}, 0xc001f03ba0?)
    net/http/server.go:2084 +0x2f
github.com/go-openapi/runtime/middleware.Spec.func1({0x28165b0, 0xc0028c1ce0}, 0xc00192a090?)
    github.com/go-openapi/runtime@v0.24.1/middleware/spec.go:46 +0x18c
net/http.HandlerFunc.ServeHTTP(0xc002398000?, {0x28165b0?, 0xc0028c1ce0?}, 0xc0030b0700?)
    net/http/server.go:2084 +0x2f
github.com/openziti/edge/controller/server.ManagementApiHandler.newHandler.func1({0x28165b0, 0xc0028c1ce0}, 0xc0030b0700)
    github.com/openziti/edge@v0.21.253/controller/server/management-api.go:133 +0x210
net/http.HandlerFunc.ServeHTTP(0xc000d9fec0?, {0x28165b0?, 0xc0028c1ce0?}, 0xe19245?)
    net/http/server.go:2084 +0x2f
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc00274cbd0, {0x28165b0, 0xc0028c1ce0}, 0xc0030b0700)
    github.com/gorilla/handlers@v1.5.1/cors.go:54 +0x370
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1()
    github.com/openziti/fabric@v0.18.5/controller/api/timeouts.go:79 +0x7c
created by github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP
    github.com/openziti/fabric@v0.18.5/controller/api/timeouts.go:72 +0x430
andrewpmartinez commented 2 years ago

It looks like both of these issues are already fixed, just not released.

andrewpmartinez commented 2 years ago

A release is in the mix, hard getting PRs through atm.

Russell-Allen commented 2 years ago

Attempting to create an External Jwt Signer with version 0.26.7 fails, although it looks like it may have gotten further than the prior version's attempt.

Command sent:

{
    "networkId": "6eaf83ce-a2ba-11eb-8def-a85e45cd45e3",
    "issuer": "iss 1",
    "audience": "aud 1",
    "enabled": false,
    "name": "test 9",
    "jwksEndpoint": "https://netfoundry-sandbox.auth0.com/.well-known/jwks.json"
}

Error in Controller logs:

[ 482.599]   ERROR fabric/controller/models.(*BaseEntityManager).ValidateNameOnCreate: entity of type *persistence.ExternalJwtSigner is named, but store doesn't have name index
[ 482.606]   ERROR fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1: panic caught by timeout next: runtime error: invalid memory address or nil pointer dereference
goroutine 2383 [running]:
github.com/openziti/foundation/v2/debugz.generateStack(0x2000, 0xb0?)
    github.com/openziti/foundation/v2@v2.0.4/debugz/stack.go:38 +0x4a
github.com/openziti/foundation/v2/debugz.GenerateLocalStack(...)
    github.com/openziti/foundation/v2@v2.0.4/debugz/stack.go:33
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1()
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:91 +0xb4
panic({0x2178ea0, 0x3676440})
    runtime/panic.go:884 +0x212
github.com/openziti/edge/controller/model.(*ExternalJwtSigner).fillFrom(0xc0039dd680, {0x3fe0000000000000?, 0xc00311542d?}, 0x16?, {0x28b1760?, 0xc003297d40?})
    github.com/openziti/edge@v0.22.91/controller/model/external_jwt_signer_model.go:103 +0x125
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntityInTx(0xc00138e360, 0xc0002ced00?, {0xc00311542d, 0x16}, {0x28ba200, 0xc0039dd680})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:273 +0x183
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity.func1(0x770000000020?)
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:259 +0x33
go.etcd.io/bbolt.(*DB).View(0x30?, 0xc00214b980)
    go.etcd.io/bbolt@v1.3.6/db.go:772 +0x82
github.com/openziti/storage/boltz.(*DbImpl).View(0x1?, 0xc0005d0d68?)
    github.com/openziti/storage@v0.1.20/boltz/db.go:116 +0x96
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity(0xc00138e360, {0xc00311542d, 0x16}, {0x28ba200?, 0xc0039dd680})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:258 +0xe6
github.com/openziti/edge/controller/model.(*baseEntityManager).BaseLoad(0xc00138e360, {0xc00311542d, 0x16})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:81 +0x5c
github.com/openziti/edge/controller/internal/routes.DetailWithHandler.func1(0xc001022f10?, {0xc00311542d?, 0xc0005d0ec8?})
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:201 +0x42
github.com/openziti/edge/controller/internal/routes.Detail(0xc002fa7260, 0x2882c90?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:220 +0x1a4
github.com/openziti/edge/controller/internal/routes.DetailWithHandler(0xc002fa76c0?, 0xc0005d12d0?, {0x28ba260?, 0xc00138e360?}, 0x207e280?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:200 +0x5d
github.com/openziti/edge/controller/internal/routes.(*ExternalJwtSignerRouter).Detail(0x1bb4d30?, 0x289dbe0?, 0x26bd7c0?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/external_jwt_signer_router.go:83 +0x35
github.com/openziti/edge/controller/env.(*AppEnv).IsAllowed.func1({0x28ab4e0, 0xc002fd4ba0}, {0x289dbe0, 0x26bd7c0})
    github.com/openziti/edge@v0.22.91/controller/env/appenv.go:597 +0x38d
github.com/go-openapi/runtime/middleware.ResponderFunc.WriteResponse(0x215d4a0?, {0x28ab4e0?, 0xc002fd4ba0?}, {0x289dbe0?, 0x26bd7c0?})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:69 +0x3d
github.com/go-openapi/runtime/middleware.(*Context).Respond(0xc001cd73e0, {0x28ab4e0?, 0xc002fd4ba0}, 0xc002fdb000, {0xc001f965e0?, 0x1, 0x1}, 0xc002fdaf00, {0x2197aa0, 0xc002fd5080})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:510 +0x5b5
github.com/openziti/edge/rest_management_api_server/operations/external_jwt_signer.(*DetailExternalJWTSigner).ServeHTTP(0xc0018a8630, {0x28ab4e0, 0xc002fd4ba0}, 0xc002fdb000)
    github.com/openziti/edge@v0.22.91/rest_management_api_server/operations/external_jwt_signer/detail_external_jwt_signer.go:93 +0x2ee
github.com/go-openapi/runtime/middleware.NewOperationExecutor.func1({0x28ab4e0, 0xc002fd4ba0}, 0xc002fdb000)
    github.com/go-openapi/runtime@v0.24.1/middleware/operation.go:28 +0x59
net/http.HandlerFunc.ServeHTTP(0x50?, {0x28ab4e0?, 0xc002fd4ba0?}, 0x0?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.NewRouter.func1({0x28ab4e0, 0xc002fd4ba0}, 0xc002fdae00)
    github.com/go-openapi/runtime@v0.24.1/middleware/router.go:78 +0x257
net/http.HandlerFunc.ServeHTTP(0xc0005d1c18?, {0x28ab4e0?, 0xc002fd4ba0?}, 0xe48157?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.Redoc.func1({0x28ab4e0, 0xc002fd4ba0}, 0xa?)
    github.com/go-openapi/runtime@v0.24.1/middleware/redoc.go:72 +0x242
net/http.HandlerFunc.ServeHTTP(0xc00214ad20?, {0x28ab4e0?, 0xc002fd4ba0?}, 0xc001703b60?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.Spec.func1({0x28ab4e0, 0xc002fd4ba0}, 0xc00214ad20?)
    github.com/go-openapi/runtime@v0.24.1/middleware/spec.go:46 +0x18c
net/http.HandlerFunc.ServeHTTP(0xc002fa7260?, {0x28ab4e0?, 0xc002fd4ba0?}, 0xc002fdae00?)
    net/http/server.go:2109 +0x2f
github.com/openziti/edge/controller/server.ManagementApiHandler.newHandler.func1({0x28ab4e0, 0xc002fd4ba0}, 0xc002fdae00)
    github.com/openziti/edge@v0.22.91/controller/server/management-api.go:133 +0x20a
net/http.HandlerFunc.ServeHTTP(0xc00214ab70?, {0x28ab4e0?, 0xc002fd4ba0?}, 0x41?)
    net/http/server.go:2109 +0x2f
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc0036e6ea0, {0x28ab4e0, 0xc002fd4ba0}, 0xc002fdae00)
    github.com/gorilla/handlers@v1.5.1/cors.go:54 +0x370
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1()
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:95 +0x7c
created by github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:88 +0x2ea

[ 482.810]   ERROR edge/controller/model.(*AuthModuleExtJwt).addSigner: {name=[test 9] hasCertPem=[false] error=[could not resolve jwks endpoint: invalid content type, expected application/json] jwksEndpoint=[0xc001703030] id=[3zx0VKvJ4jiRp8ll28p3Lf]} could not resolve signer cert/jwks
Russell-Allen commented 1 year ago

I've tested with a different JWKS endpoint (AWS Cognito this time), and I am observing a different error.

Create Command to Ziti Controller (0.26.7):

CreateZitiExternalJwtSigner(name=test 10, enabled=true, issuer=iss 10, audience=aud 10, jwksEndpoint=https://cognito-idp.us-east-1.amazonaws.com/us-east-1_3uDA8bXTz/.well-known/jwks.json, claimsProperty=null, useExternalId=false, externalAuthUrl=null, tags={network-id=6eaf83ce-a2ba-11eb-8def-a85e45cd45e3, resource-id=63e9d76e-2042-41ff-b774-524927dff984})

The POST comes back with a 200... POST to https://127.0.0.1:443/edge/management/v1/external-jwt-signers responded in 4ms with status 201 CREATED.

A following attempt to GET the entity fails... GET to https://127.0.0.1:443/edge/management/v1/external-jwt-signers/1DqLEdIOBgFy8h4GtPU1OH responded in 3ms with status 500 INTERNAL_SERVER_ERROR.

Ziti Controller logs:

[489210.869]    INFO : http: TLS handshake error from 127.0.0.1:60204: remote error: tls: bad certificate
[556680.630]    INFO : http: TLS handshake error from 127.0.0.1:34354: remote error: tls: unknown certificate
[597960.606]    INFO : http: TLS handshake error from 127.0.0.1:52680: remote error: tls: unknown certificate
[760223.857]   ERROR fabric/controller/models.(*BaseEntityManager).ValidateNameOnCreate: entity of type *persistence.ExternalJwtSigner is named, but store doesn't have name index
[760223.864]   ERROR fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1: panic caught by timeout next: runtime error: invalid memory address or nil pointer dereference
goroutine 3485015 [running]:
github.com/openziti/foundation/v2/debugz.generateStack(0x2000, 0xb0?)
    github.com/openziti/foundation/v2@v2.0.4/debugz/stack.go:38 +0x4a
github.com/openziti/foundation/v2/debugz.GenerateLocalStack(...)
    github.com/openziti/foundation/v2@v2.0.4/debugz/stack.go:33
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1.1()
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:91 +0xb4
panic({0x2178ea0, 0x3676440})
    runtime/panic.go:884 +0x212
github.com/openziti/edge/controller/model.(*ExternalJwtSigner).fillFrom(0xc003920b40, {0x3fe0000000000000?, 0xc000ab5c4d?}, 0x16?, {0x28b1760?, 0xc003b16000?})
    github.com/openziti/edge@v0.22.91/controller/model/external_jwt_signer_model.go:103 +0x125
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntityInTx(0xc00138e360, 0xc0002ced00?, {0xc000ab5c4d, 0x16}, {0x28ba200, 0xc003920b40})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:273 +0x183
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity.func1(0x20?)
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:259 +0x33
go.etcd.io/bbolt.(*DB).View(0x30?, 0xc001cb5c80)
    go.etcd.io/bbolt@v1.3.6/db.go:772 +0x82
github.com/openziti/storage/boltz.(*DbImpl).View(0x1?, 0xc004394d68?)
    github.com/openziti/storage@v0.1.20/boltz/db.go:116 +0x96
github.com/openziti/edge/controller/model.(*baseEntityManager).readEntity(0xc00138e360, {0xc000ab5c4d, 0x16}, {0x28ba200?, 0xc003920b40})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:258 +0xe6
github.com/openziti/edge/controller/model.(*baseEntityManager).BaseLoad(0xc00138e360, {0xc000ab5c4d, 0x16})
    github.com/openziti/edge@v0.22.91/controller/model/base_manager.go:81 +0x5c
github.com/openziti/edge/controller/internal/routes.DetailWithHandler.func1(0xc002bceeb0?, {0xc000ab5c4d?, 0xc004394ec8?})
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:201 +0x42
github.com/openziti/edge/controller/internal/routes.Detail(0xc000ac7ce0, 0x2882c90?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:220 +0x1a4
github.com/openziti/edge/controller/internal/routes.DetailWithHandler(0xc00117e1c0?, 0xc0043952d0?, {0x28ba260?, 0xc00138e360?}, 0x207e280?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/base_router.go:200 +0x5d
github.com/openziti/edge/controller/internal/routes.(*ExternalJwtSignerRouter).Detail(0x1bb4d30?, 0x289dbe0?, 0x26bd7c0?)
    github.com/openziti/edge@v0.22.91/controller/internal/routes/external_jwt_signer_router.go:83 +0x35
github.com/openziti/edge/controller/env.(*AppEnv).IsAllowed.func1({0x28ab4e0, 0xc0035bf860}, {0x289dbe0, 0x26bd7c0})
    github.com/openziti/edge@v0.22.91/controller/env/appenv.go:597 +0x38d
github.com/go-openapi/runtime/middleware.ResponderFunc.WriteResponse(0x215d4a0?, {0x28ab4e0?, 0xc0035bf860?}, {0x289dbe0?, 0x26bd7c0?})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:69 +0x3d
github.com/go-openapi/runtime/middleware.(*Context).Respond(0xc001cd73e0, {0x28ab4e0?, 0xc0035bf860}, 0xc00368ce00, {0xc001f965e0?, 0x1, 0x1}, 0xc00368cd00, {0x2197aa0, 0xc0035bfe00})
    github.com/go-openapi/runtime@v0.24.1/middleware/context.go:510 +0x5b5
github.com/openziti/edge/rest_management_api_server/operations/external_jwt_signer.(*DetailExternalJWTSigner).ServeHTTP(0xc0018a8630, {0x28ab4e0, 0xc0035bf860}, 0xc00368ce00)
    github.com/openziti/edge@v0.22.91/rest_management_api_server/operations/external_jwt_signer/detail_external_jwt_signer.go:93 +0x2ee
github.com/go-openapi/runtime/middleware.NewOperationExecutor.func1({0x28ab4e0, 0xc0035bf860}, 0xc00368ce00)
    github.com/go-openapi/runtime@v0.24.1/middleware/operation.go:28 +0x59
net/http.HandlerFunc.ServeHTTP(0xc004395b40?, {0x28ab4e0?, 0xc0035bf860?}, 0x0?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.NewRouter.func1({0x28ab4e0, 0xc0035bf860}, 0xc00368cc00)
    github.com/go-openapi/runtime@v0.24.1/middleware/router.go:78 +0x257
net/http.HandlerFunc.ServeHTTP(0xc004395c18?, {0x28ab4e0?, 0xc0035bf860?}, 0xe48157?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.Redoc.func1({0x28ab4e0, 0xc0035bf860}, 0xa?)
    github.com/go-openapi/runtime@v0.24.1/middleware/redoc.go:72 +0x242
net/http.HandlerFunc.ServeHTTP(0xc001cb5230?, {0x28ab4e0?, 0xc0035bf860?}, 0xc001d396a0?)
    net/http/server.go:2109 +0x2f
github.com/go-openapi/runtime/middleware.Spec.func1({0x28ab4e0, 0xc0035bf860}, 0xc001cb5230?)
    github.com/go-openapi/runtime@v0.24.1/middleware/spec.go:46 +0x18c
net/http.HandlerFunc.ServeHTTP(0xc000ac7ce0?, {0x28ab4e0?, 0xc0035bf860?}, 0xc00368cc00?)
    net/http/server.go:2109 +0x2f
github.com/openziti/edge/controller/server.ManagementApiHandler.newHandler.func1({0x28ab4e0, 0xc0035bf860}, 0xc00368cc00)
    github.com/openziti/edge@v0.22.91/controller/server/management-api.go:133 +0x20a
net/http.HandlerFunc.ServeHTTP(0xc001cb5110?, {0x28ab4e0?, 0xc0035bf860?}, 0xe48da5?)
    net/http/server.go:2109 +0x2f
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc0036e6ea0, {0x28ab4e0, 0xc0035bf860}, 0xc00368cc00)
    github.com/gorilla/handlers@v1.5.1/cors.go:54 +0x370
github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP.func1()
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:95 +0x7c
created by github.com/openziti/fabric/controller/api.(*timeoutHandler).ServeHTTP
    github.com/openziti/fabric@v0.19.67/controller/api/timeouts.go:88 +0x2ea

[760223.960]   ERROR edge/controller/model.(*AuthModuleExtJwt).addSigner: {jwksEndpoint=[0xc001604a00] id=[1DqLEdIOBgFy8h4GtPU1OH] name=[test 10] hasCertPem=[false] error=[could not parse JWKS keys, x509 chain was empty]} could not resolve signer cert/jwks

Let me know if you want me to move this to a new issue.

andrewpmartinez commented 1 year ago

external_jwt_signer_model.go:103 looks to be the same issue from the other stack trace. The next release (v0.28.9) should have this fixed.