search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
4.02k stars 367 forks source link

NeMo-Guardrails Container Setup: Production Jailbreak Functionality Not Working as Expected #690

Closed mohilmakwana3107 closed 1 month ago

mohilmakwana3107 commented 1 month ago

I am following the instructions to integrate NeMo-Guardrails into my production environment. I cloned the repository and outsourced setting up the environment in a Docker container according to the file jailbreak_cpu Dockerfile. The container build finished without any issues.

rails:
  config: 
    jailbreak_detection:
      server_endpoint: "http://0.0.0.0:1337/heuristics"
      length_per_plexity_threshold: 89.79
      prefix_suffix_perplexity_threshold: 1845.65

Although when I try to make some use of the service in production, it returns this kind of error:

image

Despite the successful build, I am unable to achieve the expected functionality as described in the documentation. The error I receive when running the service is:

image

I also tried curl from my pc : image

I would appreciate any guidance on how to solve this problem or even understand if I'm missing any configurations for this setup.

Thank you!

mohilmakwana3107 commented 1 month ago

Ok so I found the issue. The issue was with host port mis-configurations. Issue is solved with given command : 

docker run --name <container_name> -p 1337:1337 <image_name>:latest