zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://privategpt.dev
Apache License 2.0
53.79k stars 7.22k forks source link

Need help with defining constants for #237

Closed GeeksikhSecurity closed 1 year ago

GeeksikhSecurity commented 1 year ago

Is your feature request related to a problem? Please describe. Running the ingest.py gives errors below: PT\constants.py", line 11, in CHROMA_SETTINGS = Settings( File "pydantic\env_settings.py", line 39, in pydantic.env_settings.BaseSettings.init File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings persist_directory none is not an allowed value (type=type_error.none.not_allowed)

Describe the solution you'd like Provide some guidance on the new setup of persist db from scratch.

Describe alternatives you've considered ChatGPT Response:

The error message you provided indicates a validation error in the Settings class, specifically for the persist_directory attribute. It seems that the value none is not allowed for that attribute, as indicated by the error message none is not an allowed value (type=type_error.none.not_allowed).

To resolve this error, you need to assign a valid value to the persist_directory attribute in the Settings class. Here are a few steps you can take to address the issue:

Check the documentation or source code for the Settings class: Look for information about the allowed values for the persist_directory attribute. Make sure you are assigning a valid value.

Verify your input: Check the part of your code where you create an instance of the Settings class and set the persist_directory attribute. Ensure that you are not assigning the value none explicitly. If you intended to assign a None value, make sure it is capitalized correctly (None).

Review any changes in dependencies or libraries: If you recently updated any libraries or dependencies in your project, it's possible that the validation rules for the Settings class have changed. Double-check the documentation or release notes for any updates that could affect the usage of the Settings class.

Debug the code: If you're still unable to identify the issue, you can try adding some print statements or using a debugger to inspect the values and trace the code execution path. This can help you pinpoint the exact location where the persist_directory attribute is being set with an invalid value.

Once you identify the problem and correct it, the error should be resolved.

Codeium Resonse: It seems that the persist_directory setting in your CHROMA_SETTINGS object is set to None, which is not allowed. You need to set it to a valid directory path.

Here's an example of how you can set the persist_directory setting to a valid directory path:

📋 Copy code from pydantic import BaseSettings

class Settings(BaseSettings): persist_directory: str = "/path/to/persist_directory"

CHROMA_SETTINGS = Settings() Replace /path/to/persist_directory with a valid directory path on your system.

Thanks in advance for this awesome resource to enhance privacy of LLM stores.

maozdemir commented 1 year ago

I don't know why but the README.md got environment variables part removed.

Rename example.env file to .env.

GeeksikhSecurity commented 1 year ago

Tried and did a fetch to get the latest changes but no joy.

 privateGPT  git fetch  privateGPT  git reset --hard HEAD HEAD is now at fdb4574 Merge pull request #211 from mdeweerd/extra_loaders  privateGPT  git merge '@{u}' Updating fdb4574..355b4be Fast-forward README.md | 17 ++++++++--------- example.env | 2 +- ingest.py | 15 ++++++++------- privateGPT.py | 8 ++++---- requirements.txt | 2 +- 5 files changed, 22 insertions(+), 22 deletions(-)

+++++++++++++++++++

 privateGPT  rm .\example.env  privateGPT  type .env PERSIST_DIRECTORY=db LLAMA_EMBEDDINGS_MODEL=models/ggml-model-q4_0.bin MODEL_TYPE=GPT4All MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin MODEL_N_CTX=1000  privateGPT  python.exe .\ingest.py Loading documents from source_documents Loaded 4 documents from source_documents Split into 327 chunks of text (max. 500 characters each) Traceback (most recent call last): File "\privateGPT\ingest.py", line 97, in main() File "\privateGPT\ingest.py", line 88, in main embeddings = HuggingFaceEmbeddings(model_name=embeddings_model_name) File "\Python310\site-packages\langchain\embeddings\huggingface.py", line 44, in init super().init(**kwargs) File "pydantic\main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for HuggingFaceEmbeddings model_name none is not an allowed value (type=type_error.none.not_allowed)

PulpCattel commented 1 year ago

Your .env should be something like:

PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000

See https://github.com/imartinez/privateGPT/blob/main/example.env

GeeksikhSecurity commented 1 year ago

Thank you! It worked.

GeeksikhSecurity commented 1 year ago

The correct settings which worked for .env PERSIST_DIRECTORY=db MODEL_TYPE=GPT4All MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2 MODEL_N_CTX=1000

GeeksikhSecurity commented 1 year ago

 privateGPT  python.exe .\ingest.py Loading documents from source_documents Loaded 4 documents from source_documents Split into 327 chunks of text (max. 500 characters each) Downloading: 100%|████████████████████████████████████████████████████████████████| 1.18k/1.18k [00:00<00:00, 1.11MB/s] Downloading: 100%|█████████████████████████████████████████████████████████████████████| 190/190 [00:00<00:00, 207kB/s] Downloading: 100%|████████████████████████████████████████████████████████████████| 10.6k/10.6k [00:00<00:00, 10.5MB/s] Downloading: 100%|█████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 374kB/s] Downloading: 100%|████████████████████████████████████████████████████████████████████| 116/116 [00:00<00:00, 59.0kB/s] Downloading: 100%|████████████████████████████████████████████████████████████████| 39.3k/39.3k [00:00<00:00, 6.36MB/s] Downloading: 100%|████████████████████████████████████████████████████████████████| 90.9M/90.9M [00:01<00:00, 63.0MB/s] Downloading: 100%|██████████████████████████████████████████████████████████████████| 53.0/53.0 [00:00<00:00, 26.5kB/s] Downloading: 100%|█████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 113kB/s] Downloading: 100%|██████████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 17.6MB/s] Downloading: 100%|█████████████████████████████████████████████████████████████████████| 350/350 [00:00<00:00, 351kB/s] Downloading: 100%|████████████████████████████████████████████████████████████████| 13.2k/13.2k [00:00<00:00, 6.60MB/s] Downloading: 100%|██████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 10.1MB/s] Downloading: 100%|█████████████████████████████████████████████████████████████████████| 349/349 [00:00<00:00, 349kB/s] Using embedded DuckDB with persistence: data will be stored in: db

kyuBin422 commented 1 year ago

I don't know why but the README.md got environment variables part removed.

Rename example.env file to .env.

Appreciate your solution. Done