search

aws-samples / comfyui-on-eks

ComfyUI on AWS
MIT No Attribution
116 stars 21 forks source link

LW and GPU Node usage #8

Closed Gursehaj closed 4 months ago

Gursehaj commented 4 months ago

I am not able to understand what is the use of the two LW nodes we are creating and according to manifest files, we are only scaling these LW nodes instead of the GPU node. Am I understanding it wrong or is something else is happening that I am missing?

Shellmode commented 4 months ago

Rather than specifying a nodegroup for GPU instances, we establish a nodegroup with CPU instances dedicated to lightweight applications, which facilitates the operation of Addon pods. The management and scaling of GPU instances are exclusively handled by Karpenter.

Gursehaj commented 4 months ago

What are the addon pods? I can't see pods within AWS console.

Shellmode commented 4 months ago

Use command kubectl get pods -A to see all pods in all namespaces

Gursehaj commented 4 months ago

What direction should I take if I want to use multiple gpus for multiple requests? Is it possible to scale no of gpus within an ec2 instance on demand?

edwinwu2014 commented 4 months ago

gpu 不会自动扩容？？

Gursehaj commented 4 months ago

What I have seen is the requests get queued to single comfyui instance.

edwinwu2014 commented 4 months ago

What I have seen is the requests get queued to single comfyui instance.

I have also encountered this problem, feeling that this does not automatically expand the GPU Node

Shellmode commented 4 months ago

Please refer to https://github.com/aws-samples/comfyui-on-eks/issues/6

Gursehaj commented 4 months ago

I'll try adding a HPA using KEDA to autoscale.

edwinwu2014 commented 4 months ago

I'll try adding a HPA using KEDA to autoscale.

Great! Can you share the modified project?