richardwhiuk
commented
6 years ago @blsaws
Yep - it'd be useful if you could provide your build logs. On our internal k8s cluster, we get the following when we run the homestead container:
2018-01-29 18:34:08,053 CRIT Supervisor running as root (no user in config file)
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/socket-factory.supervisord.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/clearwater-group.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/homestead.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/snmpd.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/clearwater-infrastructure.conf" during parsing
2018-01-29 18:34:08,053 WARN Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2018-01-29 18:34:08,071 INFO RPC interface 'supervisor' initialized
2018-01-29 18:34:08,071 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2018-01-29 18:34:08,071 INFO supervisord started with pid 1
2018-01-29 18:34:09,074 INFO spawned: 'snmpd' with pid 9
2018-01-29 18:34:09,076 INFO spawned: 'socket-factory-sig' with pid 10
2018-01-29 18:34:09,078 INFO spawned: 'clearwater-infrastructure' with pid 12
2018-01-29 18:34:09,079 INFO spawned: 'socket-factory-mgmt' with pid 13
2018-01-29 18:34:09,080 INFO success: snmpd entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2018-01-29 18:34:09,080 INFO success: clearwater-infrastructure entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2018-01-29 18:34:09,111 INFO exited: snmpd (exit status 0; expected)
2018-01-29 18:34:09,111 CRIT reaped unknown pid 42)
2018-01-29 18:34:10,153 INFO success: socket-factory-sig entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-01-29 18:34:10,153 INFO success: socket-factory-mgmt entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-01-29 18:34:11,158 CRIT reaped unknown pid 69)
2018-01-29 18:34:11,159 CRIT reaped unknown pid 68)
2018-01-29 18:34:11,403 CRIT reaped unknown pid 152)
2018-01-29 18:34:11,403 CRIT reaped unknown pid 151)
2018-01-29 18:34:11,417 CRIT reaped unknown pid 177)
2018-01-29 18:34:12,585 CRIT reaped unknown pid 213)
2018-01-29 18:34:12,698 INFO spawned: 'nginx' with pid 216
2018-01-29 18:34:12,699 INFO spawned: 'homestead' with pid 217
2018-01-29 18:34:12,768 INFO exited: socket-factory-mgmt (exit status 2; expected)
2018-01-29 18:34:12,805 INFO spawned: 'socket-factory-mgmt' with pid 842
2018-01-29 18:34:13,808 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-01-29 18:34:13,809 INFO success: homestead entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-01-29 18:34:13,809 INFO success: socket-factory-mgmt entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-01-29 18:34:14,822 INFO exited: clearwater-infrastructure (exit status 0; expected)Note, it might be useful to do a clean build of the Docker images, without the cache to ensure you've got a consistent set of packages. We normally build with the following command:
for i in base astaire cassandra chronos bono ellis homer homestead homestead-prov ralf sprout; do docker build --no-cache -t clearwater/$i $i; done
blsaws
mdavis-xyz
AlessioDiama
Hi, I wanted to first float these issues on the clearwater list but am getting no response to list subscription requests. So reaching out here for help. I'm working on deploying clearwater-docker under Kubernetes for use in the OPNFV Models and VES projects. I'm having issues with homestead and homestead-prov crashing, and never completely coming up. It appears to be related to the snmp daemon, for which some binaries are missing. See below for the logs (output of "kubectl logs --namespace default homestead-1026420941-q3lk4 homestead", similar output for homestead-prov). I've looked into the Dockerfiles and AFAICT the issue must be somewhere in the package build process for homestead, vs the packages used for the other containers (for which this snmp issue does not occur). I have tried this with the latest (master) and stable repos.
I can provide build output for the container image as needed. Nothing unusual there, beyond a couple of errors that do not fail the build, and which are also present for the containers that do work.
From comparing the bono and homestead containers live, it's clear that there are files in /usr/share/clearwater/bin/ named clearwater-socket-factory* (a set of files) that are not present in the homestead container, as built. I believe side-effects of this error are that the homestead container never fully initializes, and the necessary DNS name for homestead not registered (as are all the other container names, even homestead-prov curiously), and thus sprout rejects registration requests as it can't find the homestead server.
Here is the log.