Error "llama_model_load: error loading model: illegal split file: <number>, model must be loaded with the first split"

felladrin commented 5 hours ago

While setting up v2.0, I've noticed it's not able to load this model:

hfRepoId: "Felladrin/gguf-Q8_0-SmolLM2-135M-Instruct",
hfFilePath: "model.shard-00001-of-00005.gguf",

[Actually, the same error is happening with all models that I split (which were all working on v1).]

It downloads the models correctly, but if triggers the following error when loading the model:

llama_model_load: error loading model: illegal split file: <number>, model must be loaded with the first split

Any clues?

Device info

OS: MacOS Browser: Tested on Brave, Chromium, and Safari

Access https://github.ngxson.com/wllama/examples/main/dist/
Click the "+ Add GGUF" button
Fill the modal with Felladrin/gguf-Q8_0-SmolLM2-135M-Instruct and model.shard-00001-of-00005.gguf, and save.
Download the model
Try to load the model, and it will inform the error.

felladrin commented 5 hours ago

Update:

I've just noticed it's actually happening with any split GGUF model, including the ones from the Demo.

I've tried loading the qwen2-1_5b-instruct-q4_k_m-(shards).gguf and got the same error.

It downloads fine:

But fails when trying to load it:

felladrin commented 5 hours ago

Update: I've confirmed the issue in these 3 browsers (Brave, Chromium, and Safari), on MacOS, in the Demo app.