Closed KDercksen closed 2 months ago
Thanks a lot for pointing this out! Should work now
Looks like prepare_hf_model_weights
is no longer available in the vllm submodule as well (see: weight_utils.py).
Im getting the following error after pulling the latest changes.
2.686 Traceback (most recent call last):
2.686 File "/download_model.py", line 6, in <module>
2.686 from vllm.model_executor.model_loader.weight_utils import prepare_hf_model_weights, Disabledtqdm
2.686 ImportError: cannot import name 'prepare_hf_model_weights' from 'vllm.model_executor.model_loader.weight_utils' (/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/weight_utils.py)
Thanks @nuckcrews, will fix this shortly, sorry for the oversight!
@alpayariyak any updates on this issue? I'm still seeing the same error.
Pushing the fix today, apologies for the delay!
@alpayariyak Has the fix been pushed? I'm still getting this error
Hi there,
The current version of the download_model.py script does not work due to the empty
TENSORIZE_MODEL
env check on line 50.Once that is fixed, the
weight_utils
file in the vllm-base image does not exist - it seems there is some version mismatch going on with the vllm submodule and the new 1.0.0preview image.Could you take a look?