trebedea
commented
6 months ago HI @ahsan3219 ,
There is an VertexAI Langchain wrapper that you can use. Is there a reason for not using it?
I don't fully understand the code you shared above. In this part are you implementing the full Langchain wrapper over VertexAI? That seems complex and duplicating the work of the wrapper already exiting in Langchain.
class CustomLLM(BaseLanguageModel):
"""A custom LLM."""
register_llm_provider("custom_llm", CustomLLM)Here you are also registering a custom LLM provider for NeMo Guardrails called "custom_llm" and in the config.yml file you call it "genai" (which does not exist).
If you opt to use the existing VertexAI wrapper from Langchain, all you need to do is have this in your config.yml:
engine: vertexai
model: gemini-pro
parameters:
model_name: gemini-proIn the next release, you will be able to use it without adding both model and model_name in the config.
drazvan
ahsan3219
parth-verma7
Pouyanpi
I try this code :
!rm -r config
!pip install -q -U google-generativeai import pathlib import textwrap import google.generativeai as genai from IPython.display import display from IPython.display import Markdown def tomarkdown(text): text = text.replace('•', ' *') return Markdown(textwrap.indent(text, '> ', predicate=lambda : True)) configure genai.configure(api_key="AIzaSyByffophpM1vZBeJdcV_mL7t6A2htetK3M") GenerativeModel model = genai.GenerativeModel('gemini-pro') model %%time response = model.generate_content("What is the meaning of life?") response If you're running this inside a notebook, patch the AsyncIO loop. import nest_asyncio nest_asyncio.apply() Step 1: create a new guardrails configuration Every guardrails configuration must be stored in a folder. The standard folder structure is as follows:
. ├── config │ ├── actions.py │ ├── config.py │ ├── config.yml │ ├── rails.co │ ├── ... See the Configuration Guide for information about the contents of these files.
Create a folder, such as config, for your configuration: !mkdir config A subdirectory or file config already exists. Create a config.yml file with the following content: %%writefile config/config.yml models:
Step 2: load the guardrails configuration To load a guardrails configuration from a path, you must create a RailsConfig instance using the from_path method in your Python code:
from nemoguardrails import RailsConfig config = RailsConfig.from_path("./config") 7 Step 3: use the guardrails configuration Use this empty configuration by creating an LLMRails instance and using the generate_async method in your Python code:
from nemoguardrails import LLMRails rails = LLMRails(config) response = rails.generate(messages=[{ "role": "user", "content": "Hello!" }]) print(response)
Exception Traceback (most recent call last) Cell In[22], line 3 1 from nemoguardrails import LLMRails ----> 3 rails = LLMRails(config) 5 response = rails.generate(messages=[{ 6 "role": "user", 7 "content": "Hello!" 8 }]) 9 print(response)
File D:\all programs install\anaconda\Lib\site-packages\nemoguardrails\rails\llm\llmrails.py:204, in LLMRails.init(self, config, llm, verbose) 201 self._validate_config() 203 # Next, we initialize the LLM engines (main engine and action engines if specified). --> 204 self._init_llms() 206 # Next, we initialize the LLM Generate actions and register them. 207 llm_generation_actions_class = ( 208 LLMGenerationActions 209 if config.colang_version == "1.0" 210 else LLMGenerationActionsV2dotx 211 )
File D:\all programs install\anaconda\Lib\site-packages\nemoguardrails\rails\llm\llmrails.py:321, in LLMRails._init_llms(self) 318 if llm_config.engine == "openai": 319 msg += " Please install langchain-openai using
pip install langchain-openai." --> 321 raise Exception(msg) 323 provider_cls = get_llm_provider(llm_config) 324 # We need to compute the kwargs for initializing the LLMException: Unknown LLM engine: genai.