search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
MIT No Attribution
151 stars 21 forks source link

KeyError: "Column(s) ['GPUMemoryUtilization', 'GPUUtilization'] do not exist" for non-GPU instances #111

Closed jimburtoft closed 1 week ago

jimburtoft commented 4 weeks ago

I tried to benchmark two inf2 instances against teach other.

I got the error KeyError: "Column(s) ['GPUMemoryUtilization', 'GPUUtilization'] do not exist"

error excerpt and config file attached.

However, it does work for configs/llama2/7b/config-llama2-7b-inf2-g5.yml

I think this is because the g5 instance does have those values, so the GPU columns exist in the dataframe. error log excerpt.txt Inf2-TGIvLMI.txt

jimburtoft commented 4 weeks ago

I confirmed it. If I add the configuration for a g5 onto the end of my config file, everything works.

aarora79 commented 4 weeks ago

ACK. Will fix this.

aarora79 commented 1 week ago

this is fixed as of v1.0.47.

harmeet-singh-discovery commented 1 week ago

Any ETA on this?

harmeet-singh-discovery commented 1 week ago

@aarora79 I am getting the error:

KeyError: "Column(s) ['GPUMemoryUtilization', 'GPUUtilization'] do not exist"

I am using ml.inf2.48xlarge machines to run benchmark.