search

containers / podman-desktop-extension-ai-lab

Work with LLMs on a local environment using containers
https://podman-desktop.io/extensions/ai-lab
Apache License 2.0
182 stars 39 forks source link

Stop using pulling update for Inference server health #982

Open axel7083 opened 6 months ago

axel7083 commented 6 months ago

We are currently using a pulling update mechanism to get the health check.

https://github.com/containers/podman-desktop-extension-ai-lab/blob/529bc5bef181032081fb5a616c0de7afabd27c4e/packages/backend/src/managers/inference/inferenceManager.ts#L280

We should move away from this behavior as it makes our extension use more resources when we already have access to this information through the podman desktop api events:

image

We already have some code able to capture container events.

https://github.com/containers/podman-desktop-extension-ai-lab/blob/529bc5bef181032081fb5a616c0de7afabd27c4e/packages/backend/src/managers/inference/inferenceManager.ts#L290-L293

The migration would be relatively easy and simplify the code for InferenceManager.

axel7083 commented 5 months ago

In the new Podman 5.1.0 version, they introduced a new healthcheck_events allowing to remove the health events.

A new field, healthcheck_events, has been added to containers.conf under the [engine] section to allow users to disable the generation of health_status events to avoid spamming logs on systems with many healthchecks.^1

When we will start using those events, we would have to check that the config does not have this, otherwise healthcheck could not be verified.