debugd: return systemd logs if restarting a unit fails

edgelesssys / constellation

Constellation is the first Confidential Kubernetes. Constellation shields entire Kubernetes clusters from the (cloud) infrastructure using confidential computing.

GNU Affero General Public License v3.0

903 stars 47 forks source link

Context

We have observed multiple cases of debugd failing when the bootstrapper unit should be restarted. This is tracked in https://github.com/edgelesssys/issues/issues/609 Since this happens rather early in the deployment process, we don't have any logs pushed to OpenSearch at this point

Proposed change(s)

Read and return systemd logs of the unit that failed to be restarted on failure
Wait until logcollection containers are running before continuing
- This doesn't really solve the issue of no logs being pushed to OpenSearch since logs are not pushed on creation, but aggregated every 30 seconds and then pushed to OpenSearch, but I think its a good practice regardless

Name	Link
Latest commit	626830743120bbf5715a9bbced6f91a0f7442574
Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/66685f0a06e7660008dac6b2

Name

Link

Latest commit

626830743120bbf5715a9bbced6f91a0f7442574

Latest deploy log

https://app.netlify.com/sites/constellation-docs/deploys/66685f0a06e7660008dac6b2

Package	Old	New	Trend
debugd/internal/debugd/deploy	28.00%	28.00%	:construction:
debugd/internal/debugd/logcollector	6.10%	6.10%	:construction:

Package

Old

New

Trend

debugd/internal/debugd/deploy

28.00%

:construction:

debugd/internal/debugd/logcollector

6.10%

:construction:

edgelesssys / constellation