Closed sidharthiimc closed 1 year ago
Uhm ok, could you provide the error message and the configuration/os of your new and old machines?
I can only do cross-testing between Windows and Linux. Maybe i missed something.
Could you also try the rustformers/llm package driectly on your new machine and try to load the model?
Both are Linux servers - with Ubuntu 18.
New machine config -
Error -
I tried building your model instead of pip. It turns out that I need to install rust and cargo. After doing that and building this worked.
The core gets dumped on new machine. When tried loading model for inference at this step: -
model = AutoModel.from_pretrained("MPT_7B_IE_TAG-q8_0-ggjt.bin", model_type=KnownModels.Mpt, session_config=session_config, verbose=True)
But this works on machine on which model was created.