ms_wrapper.py have wrong, `device_map` had weights offloaded to the disk

[toniedeng]

class SunsimiaoTextGeneration(TorchModel): def init(self, model_dir=None, *args, *kwargs): super().init(model_dir, args, **kwargs) self.logger = get_logger()

loading tokenizer

    self.tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
    self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True)
    self.model = self.model.eval()

Message=The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format. Source=C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py StackTrace: File "C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py", line 42, in init self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True) File "C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py", line 20, in init (Current frame) model = SunsimiaoTextGeneration(model) if isinstance(model, str) else model File "C:\Users\Administrator\source\repos\Sunsimiao\scripts\inference_ms.py", line 4, in pipe = pipeline(task=Tasks.text_generation,

[jingnant]

可能是由于显存不足，尝试在model给offload_folder参数 self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True,offload_folder="offload_folder")