K8s Resource Management

kevinintel commented 3 months ago

eero-t commented 3 weeks ago

K8s Resource Management

@kevinintel Is this referring to DRA or something else?

skaajas commented 3 weeks ago

Current activities regarding K8s resource management include:

Applying of container runtime Node Resource Interface (NRI) (https://github.com/containerd/nri) and its reference policies for efficient management of CPU resources. This is done for both SPR-only OPEA and also Gaudi accelerate case. The objective is to improve node throughput and also use CPU resources as affectively as possible to leave room for adjacent services. The first PR (https://github.com/opea-project/GenAIEval/pull/73) was merged to OPEA v0.9 release and this will further enhanced in the forthcoming releases.
Applying new Gaudi Dynamic Resource Allocation (DRA) driver (https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/README.md) for OPEA workloads. This has now been validate to work with Gaudi accelerated OPEA ChatQnA service.

Is there something else that should be considered?

skaajas commented 3 weeks ago

I think we should consolidate or define the exact scope for each of these issues.

opea-project / GenAIEval