cant find ”load_model_and_preprocess“

[easonchan2023]

from lavis.models import load_model_and_preprocess

i dont know where the load_model_and_preprocess is，thanks！

[xiujiesong]

Hi, maybe you can just add this before you import it.

import sys
sys.path.append("/your/path/to/LAVIS")

[macqueen09]

load_model_and_preprocess are in /your/path/to/LAVIS/init.py

My question is how to use this funcion offline, which is without network. I couldn;t connect huggingface. but i can connect https://hf-mirror.com like that . how could i setting HF_ENDPOINT or use offline file ?

[YiboZhao624]

load_model_and_preprocess are in /your/path/to/LAVIS/init.py

My question is how to use this funcion offline, which is without network. I couldn;t connect huggingface. but i can connect https://hf-mirror.com like that . how could i setting HF_ENDPOINT or use offline file ?

I have tried it in my Jupyter Notebook supported by Auto DL platform, but it doesn't work and confuse me as well. If you have got any possible solution, please reply me and I will appreciate it a lot.

[YiboZhao624]

I come and comment again. I dug in the source code and found the reason for the error caused by the function load_model_and_preprocess.
In this function, it used model_cls = registry.get_model_class(name) to get the model class I added the code print(model_cls) to find what it is. In my file, I used the Blip_feature_extractor, then I got the <class 'lavis.models.blip_models.blip_feature_extractor.BlipFeatureExtractor'>, which used the method from_pretrained and caused the error. Thus, it is located in the file BlipFeatureExtractor.py, if you are using the vscode, try ctrl and left mouse to visit the package files. You will find it has no method named "from_pretrained", then traced back to its father class BlipBase, no such method as will, but got "load_from_pretrained" method, which is more flexible and can load from path or URL.
It should use this method instead of the stilted "from_pretrained" from my perspective, and I am curious about the reason why the developers used it. I would appreciate it if any collaborator could reply. @dxli94
Keep tracing back, in the grandfather class called base model, I got this function but I can't understand it for I am exhausted now. I just paste the function below. If anyone can help with it, I would appreciate it a lot.

@classmethod
    def from_pretrained(cls, model_type):
        """
        Build a pretrained model from default configuration file, specified by model_type.

        Args:
            - model_type (str): model type, specifying architecture and checkpoints.

        Returns:
            - model (nn.Module): pretrained or finetuned model, depending on the configuration.
        """
        model_cfg = OmegaConf.load(cls.default_config_path(model_type)).model
        model = cls.from_config(model_cfg)

        return model

The above code is from lavis/models/base_model.py.

[macqueen09]

不知道是否符合你的解决办法，我已经搞成了。 我看一共两个地方需要下载模型，bert的特征提取器，是huggingface里下载的。 其余blip模型是google drive下载的，yaml里可以找到链接。

后者在翻墙电脑下载好，2G一个，挪到服务器里，然后yaml里改成路径就行 前者 通过命令行里 export HF_ENDPOINT=https://hf-mirror.com

可以变为国内镜像自动下载，或者找到加载bert代码那行，load_from_pretrain 地方改成路径即可

[YiboZhao624]

感谢您的帮助！谢谢！     ------------------ Original ------------------ From: "Macqueen Ma"; Date: 2024年4月10日(星期三) 晚上6:00 To: "salesforce/LAVIS"; Cc: @.***>; "Comment"; Subject: Re: [salesforce/LAVIS] cant find ”load_model_and_preprocess“ (Issue #619)

 

不知道是否符合你的解决办法，我已经搞成了。 我看一共两个地方需要下载模型，bert的特征提取器，是huggingface里下载的。 其余blip模型是google drive下载的，yaml里可以找到链接。

后者在翻墙电脑下载好，2G一个，挪到服务器里，然后yaml里改成路径就行 前者 通过命令行里 export HF_ENDPOINT=https://hf-mirror.com

可以变为国内镜像自动下载，或者找到加载bert代码那行，load_from_pretrain 地方改成路径即可

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>