device-coap failing with HTTP 409 Conflict message in secure mode

[bnevis-i]

🐞 Bug Report

Affected Services [REQUIRED]

device-coap

Is this a regression?

Unknown

Description and Minimal Reproduction [REQUIRED]

device-coap container fails when started under Kubernetes. See https://github.com/edgexfoundry/edgex-examples/pull/187 for testing instructions.

This also fails under docker

make run ds-coap

🔥 Exception or Error

Script for waiting on security bootstrapping ready-to-run
Tue Apr 18 17:23:10 UTC 2023 Executing waitFor with /device-coap -cp=consul://edgex-core-consul:8500 --registry --confdir=/res waiting on tcp://edgex-security-bootstrapper:54329
level=INFO ts=2023-04-18T17:23:10.273568021Z app=security-bootstrapper source=config.go:611 msg="Loading configuration file from /edgex-init/res/configuration.yaml"
level=INFO ts=2023-04-18T17:23:10.274440235Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/WaitFor/Timeout' by environment variable: STAGEGATE_WAITFOR_TIMEOUT=60s"
level=INFO ts=2023-04-18T17:23:10.274471781Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Database/Host' by environment variable: STAGEGATE_DATABASE_HOST=edgex-redis"
level=INFO ts=2023-04-18T17:23:10.274482736Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Ready/ToRunPort' by environment variable: STAGEGATE_READY_TORUNPORT=54329"
level=INFO ts=2023-04-18T17:23:10.274488333Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'SecretStore/Host' by environment variable: SECRETSTORE_HOST=edgex-vault"
level=INFO ts=2023-04-18T17:23:10.274492899Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/SecretStoreSetup/Host' by environment variable: STAGEGATE_SECRETSTORESETUP_HOST=edgex-security-secretstore-setup"
level=INFO ts=2023-04-18T17:23:10.274501516Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/SecretStoreSetup/Tokens/ReadyPort' by environment variable: STAGEGATE_SECRETSTORESETUP_TOKENS_READYPORT=54322"
level=INFO ts=2023-04-18T17:23:10.274506089Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Registry/ReadyPort' by environment variable: STAGEGATE_REGISTRY_READYPORT=54324"
level=INFO ts=2023-04-18T17:23:10.274510449Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Database/ReadyPort' by environment variable: STAGEGATE_DATABASE_READYPORT=6379"
level=INFO ts=2023-04-18T17:23:10.274514767Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/BootStrapper/StartPort' by environment variable: STAGEGATE_BOOTSTRAPPER_STARTPORT=54321"
level=INFO ts=2023-04-18T17:23:10.274520071Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/BootStrapper/Host' by environment variable: STAGEGATE_BOOTSTRAPPER_HOST=edgex-security-bootstrapper"
level=INFO ts=2023-04-18T17:23:10.274524153Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Registry/Port' by environment variable: STAGEGATE_REGISTRY_PORT=8500"
level=INFO ts=2023-04-18T17:23:10.274528639Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Database/Port' by environment variable: STAGEGATE_DATABASE_PORT=6379"
level=INFO ts=2023-04-18T17:23:10.274534709Z app=security-bootstrapper source=variables.go:457 msg="Variables override of 'StageGate/Registry/Host' by environment variable: STAGEGATE_REGISTRY_HOST=edgex-core-consul"
level=INFO ts=2023-04-18T17:23:10.274541161Z app=security-bootstrapper source=config.go:222 msg="Private configuration loaded from file with 13 overrides applied"
level=INFO ts=2023-04-18T17:23:10.274797494Z app=security-bootstrapper source=command.go:119 msg="Security bootstrapper running waitFor"
level=INFO ts=2023-04-18T17:23:10.274860101Z app=security-bootstrapper source=command.go:144 msg="Waiting for: [tcp://edgex-security-bootstrapper:54329] with timeout: [1m0s]"
level=INFO ts=2023-04-18T17:23:10.277108742Z app=security-bootstrapper source=command.go:214 msg="Connected to tcp://edgex-security-bootstrapper:54329"
Tue Apr 18 17:23:10 UTC 2023 Starting /device-coap -cp=consul://edgex-core-consul:8500 --registry --confdir=/res ...
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="iot_threadpool_alloc (threads: 8 max_jobs: 0 default_priority: -1 affinity: -1)"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="iot_scheduler_alloc (priority: -1 affinity: -1)"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Override config SecretStore/Host = edgex-vault"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Override config Service/Host = edgex-device-coap"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="vault: scheduled token refresh in 1589 seconds"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Found registry service at consul://edgex-core-consul:8500"
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Unable to get configuration from registry."
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Will load from file."
level=INFO ts=2023-04-18T17:23:10Z app=device-coap msg="Uploading configuration to registry."
level=ERROR ts=2023-04-18T17:23:10Z app=device-coap msg="Unable to upload config: HTTP 409 Conflict"
Error: 17: HTTP 409 Conflict

🌍 Your Environment

Deployment Environment: kubernetes or docker

EdgeX Version [REQUIRED]: minnesota dev

Anything else relevant?

[bnevis-i]

Still occurs in EdgeX 3.0

[lindseysimple]

@FelixTing Please investigate further.

[FelixTing]

I cannot reproduce it. The device-coap runs normally in secure mode. @bnevis-i Could you please try again when you have time?

[bnevis-i]

I cannot reproduce it. The device-coap runs normally in secure mode. @bnevis-i Could you please try again when you have time?

No longer able to reproduce the failure (using released versions of the containers).