interTwin-eu / interlink-slurm-plugin

MIT License
1 stars 3 forks source link

kubectl logs -f immediately returns instead of waits for container death #35

Open antoinetran opened 2 weeks ago

antoinetran commented 2 weeks ago

Short Description of the issue

When doing kubectl logs [pod] -f, it returns the logs and dies immediately, even if the main container is still running. It should not return and continues to show logs until it dies.

Environment

Steps to reproduce

Logs, stacktrace, or other symptoms

output

Summary of proposed changes

antoinetran commented 2 weeks ago

I don't know if there is a more elegant solution than doing a regular sleep of 2s and check if the container died, in follow mode. Or with inotify , we can avoid sleep.

antoinetran commented 2 weeks ago

It took me a bit of time to understand how the logs are returned to virtual kubelet. I created an issue at virtual-kubelet because something needs to be done there first.

dciangot commented 2 weeks ago

@antoinetran as anticipated via chat, I think it’s matter of implementing a stream response in LogRetrieval function in execute.go file in interlink

can you cross check and see if that makes sense to you?

antoinetran commented 2 weeks ago

@antoinetran as anticipated via chat, I think it’s matter of implementing a stream response in LogRetrieval function in execute.go file in interlink

can you cross check and see if that makes sense to you?

I am trying a few things, it will take some time, I am a bit slow in GO :)

antoinetran commented 1 week ago

I tested a bit the GO http stream, and I think it is a good solution. The HTTP server can send the HTTP status code , and write the body continuously. The client can get the HTTP status, and read the HTTP body, it hangs until the server stops. Then the client gets EOF at the read.