Always getting the message : I'm sorry, an internal error has occurred.

[pradeepdev-1995]

I am using the Nemo guardrails for off-topic rail functionality here is the config.yml file content

instructions:
  - type: general
    content: |
      You are an AI assistant that supports FDC copilot queries
rails:
  dialog:
    user_messages:
      embeddings_only: True

off-topic.co file contents(not original)

define user ask off topic
    "Generate a sql for dbt model which gives me position-wise count of employees in each department."
    "Generate a dbt model which gives me product details which has similar recommendation_score for each customer."
    "generate code for data visualization"

define bot explain cant off topic
    "True"

define flow
    user ask off topic
    bot explain cant off topic

define user express greeting
    "hello"
    "hi"
    "what's up?"
define bot express greeting
    "False"

define flow greeting
    user express greeting
    bot express greeting

and app.py file

from langchain_openai import AzureChatOpenAI
model = AzureChatOpenAI(credentials)
from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("./config")
rails = LLMRails(config,llm=model,verbose=True)
response = rails.generate(messages=[{
            "role": "user",
            "content": user_query
        }])
print(response)

But the response is always

I'm sorry, an internal error has occurred

And here are the logs

Event UtteranceUserActionFinished {'final_transcript': 'my query'}
Event StartInternalSystemAction {'uid': '9a708e69-10af-468f-a01f-e752971dc080', 'event_created_at': '2024-05-20T16:35:20.464816+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'bb4dc3aa-11b3-4460-8202-21691766a1de', 'is_system_action': True}
Executing action create_event
Event UserMessage {'uid': '5e2c191e-7b6b-4a98-bfb7-3965c74ae1e8', 'event_created_at': '2024-05-20T16:35:20.464988+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'How to deploy a model in refract'}
Event StartInternalSystemAction {'uid': '7936b051-e426-4e9b-a275-1e692411ab78', 'event_created_at': '2024-05-20T16:35:20.465250+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': 'f5eabcf2-1631-4c1b-99cf-9250ced6f60c', 'is_system_action': True}
Executing action generate_user_intent
Phase 1 Generating user intent
2024-05-20 22:05:20.542 | ERROR    | fastembed.common.model_management:download_model:236 - Could not download model from HuggingFace: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/qdrant/all-MiniLM-L6-v2-onnx/revision/main (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)')))"), '(Request ID: e682d2b8-a897-4c7f-bea3-663cc0914a84)')Falling back to other sources.
Event hide_prev_turn {}
--- Total processing took 0.10 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies

@drazvan

[drazvan]

Hi @pradeepdev-1995 ! This could be a network issue.

This line:

2024-05-20 22:05:20.542 | ERROR    | fastembed.common.model_management:download_model:236 - Could not download model from HuggingFace: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/qdrant/all-MiniLM-L6-v2-onnx/revision/main (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)')))"), '(Request ID: e682d2b8-a897-4c7f-bea3-663cc0914a84)')Falling back to other sources.

shows that the embeddings model can't be downloaded from HuggingFace. Can you double check you can download that model? Perhaps with a small python script that runs on the same server.

[pradeepdev-1995]

@drazvan Okay, I can download that model in a local directory path separately. Now, how can I specify the embedding model locally stored path in the rail configuration?

[drazvan]

Setting the FASTEMBED_CACHE_PATH environment variable should do it. Let me know if it works.

[pradeepdev-1995]

@drazvan the offtopic rail working well in my local machine. i can see more logs and intermediate results

But the same code, after deploying to a kubernet cluster it always show as I'm sorry, an internal error has occurred without any other error message the intermediate steps also lesser than the local what might be the reason? my local and kubernet cluster, both use python 3.9 The stranger thing is we cant see any other error to debug this

[drazvan]

A few more questions to help debug this:

• Are you using the provided Dockerfile to build the container?
• Do you run the nemoguardrails server or some other Python code that uses the LLMRails instance?
• Does running the container locally work?

[pradeepdev-1995]

@drazvan 1) Are you using the provided Dockerfile to build the container? - yes the entire application is containerizing using docker file for the deployment. but I am not using any nemo-guardrail specific docker file/image tag anything

2) Do you run the nemoguardrails server or some other Python code that uses the LLMRails instance? - I am not using nemoguardrails server. using the LLMRails python client as below

from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("./config")
rails = LLMRails(config,llm=model,verbose=True)

also now I can see like this in the logs

Error while execution generate user intent: Permission denied: '.cache' nemo guardrails

[drazvan]

This is likely coming from here: https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/nemoguardrails/embeddings/embedding_providers/fastembed.py#L54

Can you create a .cache folder in the current working directory for your app and make sure it's writable? Also, you can try putting the downloaded model in that folder.

[BrownTen]

I'm just do like hello_world, but also get the same problem which the response is always that"I'm sorry, an internal error has occurred."

Then I set verbose=True and then print the log like u show ,but i didn't have the error info.

i want to know how can i locate the internal error or how can i print the detail error info??

i have try and confim this:

1. the folder .cache can be created
2. the mode all-MiniLM-L6-v2 i has already downloaded under the .cache/embedding

[BrownTen]

I'm just do like hello_world, but also get the same problem which the response is always that"I'm sorry, an internal error has occurred."

Then I set verbose=True and then print the log like u show ,but i didn't have the error info.

i want to know how can i locate the internal error or how can i print the detail error info??

i have try and confim this:

1. the folder .cache can be created
2. the mode all-MiniLM-L6-v2 i has already downloaded under the .cache/embedding

oh , i finally find the mistake. it's my fault , in the config.yml file, i miss the s for the models