search

NVIDIA / NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Other
3.74k stars 331 forks source link

llm chain not working AttributeError: 'NoneType' object has no attribute 'agenerate_prompt' #298

Open Komal-99 opened 5 months ago

Komal-99 commented 5 months ago

I have Implemented Neoguardrails in Retrieval QA chain with llama2 model but it is giving error

My code:

MODEL_TYPE=GPT4All
MODEL_PATH=r'C:\Users\komal\Desktop\mages\chatbot\llama-2-7b-chat.Q3_K_M.gguf'
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4
llm = LlamaCpp(model_path=MODEL_PATH, n_ctx=MODEL_N_CTX, n_batch=MODEL_N_BATCH, verbose=False)

qa = RetrievalQA.from_chain_type(
    llm=llm ,chain_type="stuff", retriever=retriever, return_source_documents=True,
    chain_type_kwargs={
        "prompt": PromptTemplate(
            template=template,
            input_variables=["context", "query"],
        ),
    },
)
prompt =PromptTemplate(
            template=template,
            input_variables=["context", 'query'],
        )
config = RailsConfig.from_path("./config")
rails = LLMRails(config)
guardrails = RunnableRails(config)
chain_with_guardrails = prompt| guardrails | qa
chain_with_guardrails = RunnableRails(config, runnable=qa, input_key='query')
a=chain_with_guardrails.invoke({'query': "what is fat grafting"})
print(a)

rails.co as follows 

define flow
  user ...
  $output = execute qa($user_message)
  bot $output

config.yml


instructions:
  - type: general
    content: |
      Below is a conversation between a user and a health assistant bot called Helpful Holly.
      The bot is designed to answer questions related to breast reconstruction surgery.
      If the bot does not know the answer to a question, it truthfully says it does not know.

sample_conversation: |
  user "Hi there. What can you do for me?"
    express greeting and ask for assistance
  bot express greeting and confirm and offer assistance
    "Hi there! Thank you for asking! I'm here to assist you with any questions or concerns you may have regarding your recent breast surgery. To provide the most accurate advice possible, could you please clarify what you mean by 'what can be done for me?' Are there specific concerns or symptoms you are experiencing that you would like guidance on? As a health assistant, I'm happy to help in any way I can, while also suggesting consulting a healthcare professional if necessary. Please feel free to share more details or ask your question directly."
  user "What are the types of breast reconstruction?"
    ask a question about breast reconstruction
  bot respond to the question 
    "There are several types of breast reconstruction options available, including autologous tissue reconstruction (using your body's tissue), implant-based reconstruction, and a combination of both. Each option has its advantages and considerations. It's essential to consult with your healthcare provider to determine the most suitable choice for your unique situation."

Error Occuring 

INFO:nemoguardrails.actions.action_dispatcher:Action dispatcher initialized
INFO:nemoguardrails.actions.action_dispatcher:Initializing action dispatcher
INFO:nemoguardrails.actions.action_dispatcher:Adding create_event to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding wolfram_alpha_request to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding retrieve_relevant_chunks to actions
INFO:nemoguardrails.actions.action_dispatcher:Added summarize_document to actions
INFO:nemoguardrails.actions.action_dispatcher:Added apify to actions
INFO:nemoguardrails.actions.action_dispatcher:Added bing_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added google_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added google_serper to actions
INFO:nemoguardrails.actions.action_dispatcher:Added openweather_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added searx_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added serp_api_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added wikipedia_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added wolframalpha_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added zapier_nla_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added apify to actions
INFO:nemoguardrails.actions.action_dispatcher:Added bing_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added google_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added google_serper to actions
INFO:nemoguardrails.actions.action_dispatcher:Added openweather_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added searx_search to actions
INFO:nemoguardrails.actions.action_dispatcher:Added serp_api_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added wikipedia_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added wolframalpha_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Added zapier_nla_query to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding call_activefence_api to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding alignscore_check_facts to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding alignscore_request to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding self_check_facts to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding llama_guard_check_input to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding llama_guard_check_output to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding self_check_facts to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding self_check_input to actions
INFO:nemoguardrails.actions.action_dispatcher:Adding self_check_output to actions
INFO:nemoguardrails.actions.action_dispatcher:Registered Actions: {'create_event': <function create_event at 0x00000152C33BF920>, 'wolfram alpha request': <function wolfram_alpha_request at 0x00000152C33BFB00>, 'retrieve_relevant_chunks': <function retrieve_relevant_chunks at 0x00000152C33BEAC0>, 'summarize_document': <class 'summarize_document.py.SummarizeDocument'>, 'apify': <class 'nemoguardrails.actions.langchain.safetools.ApifyWrapperSafe'>, 'bing_search': <class 'nemoguardrails.actions.langchain.safetools.BingSearchAPIWrapperSafe'>, 'google_search': <class 'nemoguardrails.actions.langchain.safetools.GoogleSearchAPIWrapperSafe'>, 'google_serper': <class 'nemoguardrails.actions.langchain.safetools.GoogleSerperAPIWrapperSafe'>, 'openweather_query': <class 'nemoguardrails.actions.langchain.safetools.OpenWeatherMapAPIWrapperSafe'>, 'searx_search': <class 'nemoguardrails.actions.langchain.safetools.SearxSearchWrapperSafe'>, 'serp_api_query': <class 'nemoguardrails.actions.langchain.safetools.SerpAPIWrapperSafe'>, 'wikipedia_query': <class 'nemoguardrails.actions.langchain.safetools.WikipediaAPIWrapperSafe'>, 'wolframalpha_query': <class 'nemoguardrails.actions.langchain.safetools.WolframAlphaAPIWrapperSafe'>, 'zapier_nla_query': <class 'nemoguardrails.actions.langchain.safetools.ZapierNLAWrapperSafe'>, 'call activefence api': <function call_activefence_api at 0x00000153C1B58EA0>, 'alignscore_check_facts': <function alignscore_check_facts at 0x00000153C1B59080>, 'alignscore request': <function alignscore_request at 0x00000152C33ED440>, 'self_check_facts': <function self_check_facts at 0x00000153C1B593A0>, 'llama_guard_check_input': <function llama_guard_check_input at 0x00000153C1B591C0>, 'llama_guard_check_output': <function llama_guard_check_output at 0x00000153C1B59260>, 'self_check_input': <function self_check_input at 0x00000153C1B59440>, 'self_check_output': <function self_check_output at 0x00000153C1B59620>}
INFO:nemoguardrails.actions.action_dispatcher:Action dispatcher initialized
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'UtteranceUserActionFinished', 'final_transcript': 'what is fat grafting'}
INFO:nemoguardrails.flows.runtime:Event :: UtteranceUserActionFinished {'final_transcript': 'what is fat grafting'}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': 'f359c6bb-8e3c-46b9-8541-386c9058c2e6', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': '59acbc52-e7ee-4dc5-b1bd-0a8a6521630c', 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Event :: StartInternalSystemAction {'uid': 'f359c6bb-8e3c-46b9-8541-386c9058c2e6', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': '59acbc52-e7ee-4dc5-b1bd-0a8a6521630c', 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Executing action :: create_event
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: create_event
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'UserMessage', 'uid': '2cecc3d1-2f48-4d57-ba40-a860bb0516f0', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'what is fat grafting'}
INFO:nemoguardrails.flows.runtime:Event :: UserMessage {'uid': '2cecc3d1-2f48-4d57-ba40-a860bb0516f0', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'what is fat grafting'}
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': '0b150819-2a1d-4765-8a96-93c8c39ad8ee', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '786944f7-86d3-41a5-9bd0-8435fbb43bd1', 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Event :: StartInternalSystemAction {'uid': '0b150819-2a1d-4765-8a96-93c8c39ad8ee', 'event_created_at': '2024-02-02T16:45:15.138920+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '786944f7-86d3-41a5-9bd0-8435fbb43bd1', 'is_system_action': True}
INFO:nemoguardrails.flows.runtime:Executing action :: generate_user_intent
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: generate_user_intent
INFO:nemoguardrails.actions.llm.generation:Phase 1: Generating user intent
WARNING:nemoguardrails.llm.params:Parameter temperature does not exist for NoneType
ERROR:nemoguardrails.actions.action_dispatcher:Error 'NoneType' object has no attribute 'agenerate_prompt' while execution generate_user_intent
Traceback (most recent call last):
  File "C:\Users\komal\Desktop\gcp\venv\Lib\site-packages\nemoguardrails\actions\action_dispatcher.py", line 178, in execute_action
    result = await result
             ^^^^^^^^^^^^
  File "C:\Users\komal\Desktop\gcp\venv\Lib\site-packages\nemoguardrails\actions\llm\generation.py", line 311, in generate_user_intent
    result = await llm_call(llm, prompt)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\komal\Desktop\gcp\venv\Lib\site-packages\nemoguardrails\actions\llm\utils.py", line 53, in llm_call
    result = await llm.agenerate_prompt(
                   ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'agenerate_prompt'
INFO:nemoguardrails.flows.runtime:Processing event: {'type': 'hide_prev_turn'}
INFO:nemoguardrails.flows.runtime:Event :: hide_prev_turn {}
INFO:nemoguardrails.rails.llm.llmrails:--- :: Total processing took 0.02 seconds.
INFO:nemoguardrails.rails.llm.llmrails:--- :: Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens
{'output': None}

Please Help How I can use llama2 model correctly 🙏🙏

rohitgarud commented 4 months ago

HI @Komal-99 , did you find a solution to your query.. I am facing same issue

drazvan commented 4 months ago

@rohitgarud @Komal-99 , the error message is indeed misleading. This exception is thrown when there is no LLM configured in the config. To fix the original example, there are two ways:

  1. Define the LLM in config.yml or pass it directly to the LLMRails instance, since it's already initialized before the qa chain.
  2. Add the following to config.yml 
    rails:
dialog:
user_messages:
  embeddings_only: True

The second option is recommended since the flow doesn't care about the actual user intent, so we don't need to generate one. Also, for both of them to work correctly, make sure there is at least one user message defined:

define user ask something
  "something"

I'll mark this as a bug to provide a more explicit error message. It should actually be fixed by https://github.com/NVIDIA/NeMo-Guardrails/pull/223 once merged. But also to trigger a warning if flows are being defined and no user message is defined.