Moving quantized mode with bin and meta file to new machine doesn't works

[sidharthiimc]

The core gets dumped on new machine. When tried loading model for inference at this step: -

model = AutoModel.from_pretrained("MPT_7B_IE_TAG-q8_0-ggjt.bin", model_type=KnownModels.Mpt, session_config=session_config, verbose=True)

But this works on machine on which model was created.

[LLukas22]

Uhm ok, could you provide the error message and the configuration/os of your new and old machines?

I can only do cross-testing between Windows and Linux. Maybe i missed something.

Could you also try the rustformers/llm package driectly on your new machine and try to load the model?

[sidharthiimc]

Both are Linux servers - with Ubuntu 18.

New machine config -

Error -

[sidharthiimc]

I tried building your model instead of pip. It turns out that I need to install rust and cargo. After doing that and building this worked.