weaveworks / launcher

Weave Cloud Launcher
Apache License 2.0
10 stars 13 forks source link

bootstrap: Re-enabled cri-endpoint flag" #265

Closed dlespiau closed 5 years ago

dlespiau commented 5 years ago

This reverts commit f2995f068e9e7e9d6c1d4b69b3d27a9075fd768a.

stefanprodan commented 5 years ago

I've tested this on GKE with containerd and even if it detects the CRI endpoint Weave Cloud doesn't display any containers.

Install logs:

Detected container runtime endpoint: /run/containerd/containerd.sock. To override the container runtime endpoint set the '--cri-endpoint=<ENDPOINT>' flag.

Scope logs:

time="2018-12-03T04:38:07Z" level=info msg="publishing to: https://nzof3epodeu1cdxpubpqhrdbz53fqjoh@frontend.dev.weave.works"
<probe> INFO: 2018/12/03 04:38:07.654337 Basic authentication disabled
<probe> INFO: 2018/12/03 04:38:07.663822 command line args: --mode=probe --probe-only=true --probe.cri=true --probe.cri.endpoint=/run/containerd/containerd.sock --probe.kubernetes=true https://<elided>@frontend.dev.weave.works
<probe> INFO: 2018/12/03 04:38:07.663890 probe starting, version 1.10.1, ID 6ec5cadccd519cbf
<probe> WARN: 2018/12/03 04:38:07.674151 Cannot resolve 'scope.weave.local.': dial tcp 172.17.0.1:53: connect: connection refused
<probe> INFO: 2018/12/03 04:38:07.863799 Control connection to frontend.dev.weave.works starting
<probe> INFO: 2018/12/03 04:38:12.246804 kubernetes: targeting api server https://10.0.0.1:443
<probe> INFO: 2018/12/03 04:38:12.359288 volumesnapshotdatas are not supported by this Kubernetes version
<probe> INFO: 2018/12/03 04:38:12.360461 volumesnapshots are not supported by this Kubernetes version
<probe> INFO: 2018/12/03 04:38:12.374061 Publish loop for frontend.dev.weave.works starting
<probe> ERRO: 2018/12/03 04:38:16.563060 DNSSnooper: error decoding packet: qname '0x40' - RFC 2673 unsupported yet (data=54 index=12)
<probe> ERRO: 2018/12/03 04:40:20.646057 DNSSnooper: error decoding packet: No decoder for layer type VXLAN

Scope doesn't show any containers:

screen shot 2018-12-03 at 11 47 26

The home page shows the containers count (probably it comes from Cortex) but no workloads:

screen shot 2018-12-03 at 11 49 47

@dlespiau @bboreham as far as I can tell there is no difference between Scope with the CRI flag or without it, no container runtime data is being collected on GKE with containerd.

dlespiau commented 5 years ago

Good news: the installer option works! bad news, scope still needs work, but that was the expectation :)

Although to be frank, we should see the list of containers at least. I wonder if on GKE that path to the CRI socket is the right one.

stefanprodan commented 5 years ago

I've downloaded the crictl inside Scope container and it works:

/home/weave/containerd/usr/local/bin # ./crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps
CONTAINER ID        IMAGE               CREATED             STATE               NAME                       ATTEMPT             POD ID
f64c87347e78b       b804c18419af8       5 hours ago         Running             podinfod                   1                   a62a40fe915fb
1e2ff5e4288d4       7ad5d345bd8a3       5 hours ago         Running             jaeger                     1                   0c1222a0fd199
e3cf4deed22bc       9b59e0d336345       5 hours ago         Running             servicegraph               2                   641b3fa40b41c
1e4761d6b70d7       4718acfac1149       5 hours ago         Running             prometheus-to-sd           0                   558452aa7bbff
5af36a3c3ef3c       4718acfac1149       5 hours ago         Running             prometheus-to-sd           0                   088a24c5c2d89
58bd69b4c634d       4b2e93f0133d3       5 hours ago         Running             sidecar                    0                   088a24c5c2d89
c3b72baf7876f       4b2e93f0133d3       5 hours ago         Running             sidecar                    0                   558452aa7bbff

Also there are no errors in Scope logs regarding the containerd socket.. it must be something else.

dlespiau commented 5 years ago

Thanks for confirming that's the right path. We need to fix scope :)