Unable to use CTransformers

[smd29]

I am using the same app.py, but there seems to be some issue with this line llm = CTransformers( model_type='llama', model='models/llama-2-7b.ggmlv3.q8_0.bin', config={'max_new_tokens': 256, 'temperature': 0.01} )

this is giving following error: Repository Not Found for url: https://huggingface.co/api/models/models/llama-2-7b-chat.ggmlv3.q8_0.bin/revision/main. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated.

I have downloaded the model locally, how can I use it? @krishnaik06

[traromal]

create a folder named model and put your downloaded model inside the models folder you can find the models in this Huggingface repo TheBloke/Llama-2-7B-Chat-GGML https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main you can download the model you want

[smd29]

@traromal thanks for helping out, but still getting the same error for this model

RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-65a570c1-7fef2aaf068a5fb14eeae764;f3f9d476-22b1-452c-9010-b073a9e0bba2) Repository Not Found for url: https://huggingface.co/api/models/models/llama-2-7b-chat.ggmlv3.q8_0.bin/revision/main. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated.

[traromal]

If you are trying to use the llama model locally you don't need to add the repo id Can you share the screenshot of the code?

[smd29]

I am using the model locally. @traromal

[traromal]

You sure u installed CTransformers and langchain inside the env?

[smd29]

Name: langchain Version: 0.1.1

Name: ctransformers Version: 0.2.27

these are the versions of ctransformers and langchain inside my venv

[traromal]

it looks like some problem with langchain package from the screenshot provided did you try it by creating a new workspace and env?

[smd29]

yes, I created a new workspace and environment.

[traromal]

Try changing the python interpreter

[smd29]

tried, but no luck.