Fail to start service with error "registry is not available"

[cherrycl]

🐞 Bug Report

Affected Services [REQUIRED]

The issue is located in: device-onvif-camera ### Is this a regression? Yes ### Description and Minimal Reproduction [**REQUIRED**] 1. Deploy EdgeX with [TAF compose file](https://github.com/edgexfoundry/edgex-compose/blob/7890ef9232c37fd7451a8a36992d226649d52797/taf/docker-compose-taf-no-secty.yml#L27). The compose file only can be deploy by TAF script. 2. Show logs for device-onvif-camera. ## 🔥 Exception or Error device-onvif-camera is always in restarting

level=INFO ts=2023-05-04T07:55:23.811896269Z app=device-onvif-camera source=registry.go:61 msg="Using Registry access token of length 0"
level=INFO ts=2023-05-04T07:55:23.811940882Z app=device-onvif-camera source=registry.go:85 msg="Using Registry (consul) from http://localhost:8500"
level=WARN ts=2023-05-04T07:55:23.812600995Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:24.813971487Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:25.814994975Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:26.816678048Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:27.818844789Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:28.821306308Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:29.822375884Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-04T07:55:30.823913223Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"

## 🌍 Your Environment **Deployment Environment:** Ubuntu 2204 / x86 **EdgeX Version [**REQUIRED**]:** Main **Anything else relevant?**

[lenny-goodell]

For some reason it is using localhost rather than the docker network name.

Using Registry (consul) from http://localhost:8500/

[lenny-goodell]

The env override is there properly: https://github.com/edgexfoundry/edgex-compose/blob/7890ef9232c37fd7451a8a36992d226649d52797/taf/docker-compose-taf-no-secty.yml#L27

Please upload complete logs of core-common-config so we can see if overrides are being applied. You can also check value in Consul.

[lenny-goodell]

I am not able to recreate this issue using referenced compose file. I did have to modify the compose to eliminate PROFILE_VOLUME_PLACE_HOLDER, but that only impacts Device Virtual.

level=INFO ts=2023-05-04T15:34:19.769427673Z app=device-onvif-camera source=registry.go:85 msg="Using Registry (consul) from http://edgex-core-consul:8500"

[cherrycl]

@lenny-intel Would you please use the attached compose file that I generated by compose builder to try again? Thank you. docker-compose.yml.zip

The full logs is as below.

cherry@cherry-VirtualBox:~/edgexfoundry/edgex-compose/compose-builder$ docker logs edgex-device-onvif-camera 
level=INFO ts=2023-05-05T00:46:55.578772751Z app=device-onvif-camera source=config.go:392 msg="Loaded service configuration from /res/configuration.toml"
level=INFO ts=2023-05-05T00:46:55.58179513Z app=device-onvif-camera source=variables.go:377 msg="Variables override of 'Service.Host' by environment variable: SERVICE_HOST=edgex-device-onvif-camera"
level=INFO ts=2023-05-05T00:46:55.582219591Z app=device-onvif-camera source=config.go:161 msg="Using Configuration Provider access token of length 0"
level=INFO ts=2023-05-05T00:46:55.582358593Z app=device-onvif-camera source=config.go:367 msg="Using Configuration provider (consul) from: http://edgex-core-consul:8500 with base path of edgex/devices/2.0/device-onvif-camera"
level=INFO ts=2023-05-05T00:47:01.918504219Z app=device-onvif-camera source=config.go:556 msg="Configuration has been pushed to into Configuration Provider (1 envVars overrides applied)"
level=INFO ts=2023-05-05T00:47:01.91854325Z app=device-onvif-camera source=registry.go:61 msg="Using Registry access token of length 0"
level=INFO ts=2023-05-05T00:47:01.918550905Z app=device-onvif-camera source=registry.go:85 msg="Using Registry (consul) from http://localhost:8500"
level=WARN ts=2023-05-05T00:47:01.91898738Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:02.920372481Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:03.921400055Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:04.922881345Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:05.923747921Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:06.925096101Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:07.92662907Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:08.927877868Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:09.929364138Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:10.930908755Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:11.932851838Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:12.934993646Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:13.936390037Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:14.937632006Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:15.938921656Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:16.940809639Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:17.942551883Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:18.944180512Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:19.945998945Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:20.947884241Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"
level=WARN ts=2023-05-05T00:47:21.949619232Z app=device-onvif-camera source=registry.go:118 msg="registry is not available"

[lenny-goodell]

I have recreated the issue, after pulling the latest image for device onvif camera. So it is a recent change to the service or the SDK.

[lenny-goodell]

Some how we are getting an old image. It is not doing Common config and loading from TOML.

msg="Loaded service configuration from /res/configuration.toml"

[lenny-goodell]

I just merged a dependa-bot PR which will trigger a new image to be built. Please try again once this pipeline has completed. https://jenkins.edgexfoundry.org/blue/organizations/jenkins/edgexfoundry%2Fdevice-onvif-camera/detail/main/133/pipeline

[lenny-goodell]

The pipeline failed. I left a note for DevOps to rerun it. I also changed the image tag from latest to main which, should be the same image. But it pulled a new one and that image works fine.

[cherrycl]

Test the new image and work fine. @lenny-intel Thank you.