Wrong os.path

[semoal]

Hello! Kudos for your nice work on the open-source community of LLM's. I'm learning a lot for your findings.

I've tried to run the latest commit on a GCP VM with 32GB of ram.

And I've found this error:

(env) (base) sergiomoreno@production:~/mpt-30B-inference$ python inference.py
/home/sergiomoreno/mpt-30B-inference/models/mpt-30b-chat.ggmlv0.q4_1.bin
Fetching 1 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 7752.87it/s]
Traceback (most recent call last):
  File "/home/sergiomoreno/mpt-30B-inference/inference.py", line 51, in <module>
    llm = AutoModelForCausalLM.from_pretrained(
  File "/home/sergiomoreno/mpt-30B-inference/env/lib/python3.10/site-packages/ctransformers/hub.py", line 157, in from_pretrained
    return LLM(
  File "/home/sergiomoreno/mpt-30B-inference/env/lib/python3.10/site-packages/ctransformers/llm.py", line 203, in __init__
    if not Path(model_path).is_file():
  File "/home/sergiomoreno/miniconda3/lib/python3.10/pathlib.py", line 960, in __new__
    self = cls._from_parts(args)
  File "/home/sergiomoreno/miniconda3/lib/python3.10/pathlib.py", line 594, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/home/sergiomoreno/miniconda3/lib/python3.10/pathlib.py", line 578, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

After looking the files structure, I've saw that folder didn't exist, and had to replace it to this:

    llm = AutoModelForCausalLM.from_pretrained(
        os.path.abspath("models/models--TheBloke--mpt-30B-chat-GGML/snapshots/60df632f84e8b99fa7aeadf314467152be55adf4/mpt-30b-chat.ggmlv0.q4_1.bin"),
        model_type="mpt",

Is it something expected or can we improve it somehow?

[abacaj]

I see issue is with the cache_dir usage fixing it to point to local files. Should be fixed I tested it locally with a new download of the model

[semoal]

Confirmed that with that change works like a charm