Locally loda model VLM2Vec-Full

[VincentVanNF]

Hello,due to the network environment requirements of the development machine, it cannot access huggingface.co to download and load the model. I have downloaded the VLM2Vec-full model from huggingface.co and saved it on the development machine. I hope to load the model locally.

Here is my code:

Phi_3_V_full = '/mnt/.../TIGER-Lab/VLM2Vec-Full'
model_args = ModelArguments(
    model_name=Phi_3_V_full,
    # checkpoint_path=Phi_3_V_full,
    pooling='last',
    normalize=True
)
model = MMEBModel.load(model_args)

However, it seems that this approach still attempts to download the model from huggingface. Could you please provide a demo for loading the model locally? Thank you very much!

Could not locate the configuration_phi3_v.py inside microsoft/Phi-3.5-vision-instruct.
Traceback (most recent call last):
  File ".../lib/python3.9/site-packages/transformers/utils/hub.py", line 402, in cached_file
    resolved_file = hf_hub_download(
  File ".../lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File ".../lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File ".../lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1240, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File ".../lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1347, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File ".../lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1848, in _raise_on_head_call_error
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: Cannot find the requested files in the disk cache and outgoing traffic has been disabled. To enable hf.co look-ups and downloads online, set 'local_files_only' to False.

[XMHZZ2018]

Sure, I will provide it in the next few days.

[VincentVanNF]

Sure, I will provide it in the next few days.

Any update?

[XMHZZ2018]

@VincentVanNF Sorry to my delay. Could you just change "--model_name" parameter in the bash script to your local directory path to enable running locally? I tested from my side and it works.

In our code, the processor loading could also download from HF, I guess it might be the reason. processor = AutoProcessor.from_pretrained( model_args.model_name, trust_remote_code=True, num_crops=model_args.num_crops, )

Please let me know whether it works!

[VincentVanNF]

@VincentVanNF Sorry to my delay. Could you just change "--model_name" parameter in the bash script to your local directory path to enable running locally? I tested from my side and it works.

In our code, the processor loading could also download from HF, I guess it might be the reason. processor = AutoProcessor.from_pretrained( model_args.model_name, trust_remote_code=True, num_crops=model_args.num_crops, )

Please let me know whether it works!

Simply modifying this is not sufficient because the corresponding modeling_phi3_v.py and configuration_phi3_v.py files are missing in MMEB.fullmodel.bs2048. I ran the model loading code in an environment with internet access and found these two files in the ~/.cache directory. These two files need to be placed into the checkpoints dir, and the auto_map value in config.json needs to be modified to point to the local file paths instead of the remote repository.

[XMHZZ2018]

Thank you for letting me know! I hadn’t realized these files were missing on HF. I’ll upload them now.