(fix) change the location of the termination-log

tphoney commented 2 weeks ago

when running the container, it would fail creating the termination-log

2024/09/04 15:46:03 maxprocs: Leaving GOMAXPROCS=16: CPU quota undefined
time="2024-09-04T15:46:03Z" level=info msg="otlptracehttp client configured itself: &{traces {localhost:4318 false <nil> map[] 0 10000000000 /v1/traces <nil> <nil>} {{localhost:4318 false <nil> map[] 0 10000000000 /v1/traces <nil> <nil>} {true 5000000000 30000000000 60000000000} 0  [] <nil>} 0x3b0c4a0 0xc00073fa70 0xc0001821c0 {{{} 0} {0 0}}}"
time="2024-09-04T15:46:04Z" level=info msg="Got config" auto-config=false aws-access-strategy=defaults aws-external-id= aws-profile= aws-regions="[]" aws-target-role-arn= health-check-port=8080 max-parallel=2000 nats-jwt= nats-name-prefix= nats-nkey-seed= nats-servers="[nats://localhost:4222 nats://nats:4222]"
Failed to fire hook: open /dev/termination-log: permission denied
time="2024-09-04T15:46:04Z" level=fatal msg="Could not create AWS configs" error="no regions specified"

We change the default user in the dockerfile, which has no access to /dev. we can set an arbitary path for kubernetes to pick up our log

https://github.com/overmindtech/aws-source/issues/596

tphoney commented 2 weeks ago

obviously /exit, is a horrible name for a folder, but we need to decide

if we want keep writing to the default place, we need to change the default user in the dockerfile
we change the user we run pods as in kubernetes, so it has permissions, NB OpenShift Kubernetes uses a range of unique user IDs for each project, which can appear to be random user IDs, so this may not work in longer term
we follow this approach and specify a folder for the termination logs, and check a different place with kubernetes https://kubernetes.io/docs/tasks/debug/debug-application/determine-reason-pod-failure/#:~:text=Kubernetes%20retrieves%20termination%20messages%20from,to%20use%20a%20different%20file

tphoney commented 1 week ago

I like the idea of just writing it to a location we control, but I do agree that /exit isn't a good name. Do these containers have any kind of a default log directory we could use like /log or /var/log?

I'll check permissions and see what makes sense.

overmindtech / aws-source

(fix) change the location of the termination-log #609