Support loading shard GGUF models

[Isotr0py]

Feature request

• Support loading from sharded GGUF model files (For example, legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF).

Motivation

• For some large models, the GGUF format model will be split into multiple shards during converting.
• However, the GGUF integration in transformers only supports single file GGUF model currently.

Your contribution

• This kind of GGUF model has been split into multiple shards, formatted as <ShardNum>-of-<ShardTotal>, just like sharded safetensors.
• The header for config and tokenizer handing are kept in the first shard file (For example, Meta-Llama-3.1-70B-Instruct.Q5_K-00001-of-00003.gguf).
• For more details about sharded GGUF files, please refer to gguf.md.

[LysandreJik]

Ah very cool! Thanks foir sharing @Isotr0py, it would be nice to indeed have support for this sharding.

If you'd like, we're very open to PRs :)

cc @SunMarc

[julien-c]

maybe cc @phymbert for visibility too

[julien-c]

and linking the related https://github.com/ggerganov/llama.cpp/issues/9023