search

langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
88.23k stars 13.83k forks source link

SQLDatabaseChain. It's "azure-openai-chat LLM cannot be loaded #10961

Open prasad-databricks opened 9 months ago

prasad-databricks commented 9 months ago

System Info

Hi @harrisoUnable to load the model logged using mlflow

import mlflow
logged_model = 'runs:/runid/sql_database_chain'
# Load model as a PyFuncModel.
loaded_model = mlflow.pyfunc.load_model(logged_model)

mlflow -> 2.7.2.dev0 langchain ->0.0.297 langchain-experimental -> 0.0.20

Exception:

`ValueError Traceback (most recent call last) File :5 2 logged_model = 'runs:/8998b4fd57c743fe8e0dae9a19ca5155/sql_database_chain' 4 # Load model as a PyFuncModel. ----> 5 loaded_model = mlflow.pyfunc.load_model(logged_model)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/mlflow/pyfunc/init.py:637, in load_model(model_uri, suppress_warnings, dst_path) 635 data_path = os.path.join(local_path, conf[DATA]) if (DATA in conf) else local_path 636 try: --> 637 model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path) 638 except ModuleNotFoundError as e: 639 if conf[MAIN] == _DATABRICKS_FS_LOADER_MODULE:

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/mlflow/langchain/init.py:778, in _load_pyfunc(path) 773 """ 774 Load PyFunc implementation for LangChain. Called by pyfunc.load_model. 775 :param path: Local filesystem path to the MLflow Model with the langchain flavor. 776 """ 777 wrapper_cls = _TestLangChainWrapper if _MLFLOW_TESTING.get() else _LangChainModelWrapper --> 778 return wrapper_cls(_load_model_from_local_fs(path))

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/mlflow/langchain/init.py:807, in _load_model_from_local_fs(local_model_path) 804 model_type = flavor_conf.get(_MODEL_TYPE_KEY) 805 loader_arg = flavor_conf.get(_LOADER_ARG_KEY) --> 807 return _load_model( 808 lc_model_path, 809 model_type, 810 loader_arg, 811 agent_model_path, 812 tools_model_path, 813 agent_primitive_path, 814 loader_fn_path, 815 persist_dir, 816 )

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/mlflow/langchain/init.py:660, in _load_model(path, model_type, loader_arg, agent_path, tools_path, agent_primitive_path, loader_fn_path, persist_dir) 658 model = _RetrieverChain.load(path, kwargs).retriever 659 else: --> 660 model = load_chain(path, kwargs) 661 elif agent_path is None and tools_path is None: 662 model = load_chain(path)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:595, in load_chain(path, kwargs) 593 return hub_result 594 else: --> 595 return _load_chain_from_file(path, kwargs)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:622, in _load_chain_from_file(file, kwargs) 619 config["memory"] = kwargs.pop("memory") 621 # Load the chain from the config now. --> 622 return load_chain_from_config(config, kwargs)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:585, in load_chain_from_config(config, kwargs) 582 raise ValueError(f"Loading {config_type} chain not supported") 584 chain_loader = type_to_loader_dict[config_type] --> 585 return chain_loader(config, kwargs)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:369, in _load_sql_database_chain(config, kwargs) 367 if "llm_chain" in config: 368 llm_chain_config = config.pop("llm_chain") --> 369 chain = load_chain_from_config(llm_chain_config) 370 return SQLDatabaseChain(llm_chain=chain, database=database, config) 371 if "llm" in config:

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:585, in load_chain_from_config(config, kwargs) 582 raise ValueError(f"Loading {config_type} chain not supported") 584 chain_loader = type_to_loader_dict[config_type] --> 585 return chain_loader(config, kwargs)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/chains/loading.py:41, in _load_llm_chain(config, **kwargs) 39 if "llm" in config: 40 llm_config = config.pop("llm") ---> 41 llm = load_llm_from_config(llm_config) 42 elif "llm_path" in config: 43 llm = load_llm(config.pop("llm_path"))

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-0d48d952-9c91-4092-be60-44e0bb77c25a/lib/python3.10/site-packages/langchain/llms/loading.py:19, in load_llm_from_config(config) 16 config_type = config.pop("_type") 18 if config_type not in type_to_cls_dict: ---> 19 raise ValueError(f"Loading {config_type} LLM not supported") 21 llm_cls = type_to_cls_dict[config_type] 22 return llm_cls(**config)

ValueError: Loading azure-openai-chat LLM not supported `

Who can help?

na

Information

Related Components

Reproduction

na

Expected behavior

it should able to load the model

dosubot[bot] commented 9 months ago

🤖

Hello,

The error message you're encountering suggests that the LangChain version you're using (0.0.297) does not support the azure-openai-chat language model.

To resolve this issue, you can use the AzureOpenAI class provided in the LangChain framework. This class is a wrapper for the Azure OpenAI chat completion API and can be used to interact with the azure-openai-chat model. Here is an example of how you can use the AzureOpenAI class:

from langchain.llms import AzureOpenAI

azure_chat = AzureOpenAI(
    deployment_name="35-turbo-dev",
    openai_api_version="2023-05-15",
)

Please replace "35-turbo-dev" with your actual deployment name and "2023-05-15" with your actual API version.

You need to have the openai Python package installed and the following environment variables set or passed in the constructor in lower case:

The above code is just an example and the actual implementation may vary depending on the specific requirements of the azure-openai-chat model.

In your context, it seems like you are trying to load the model using the MlflowLogger class in LangChain. If you are trying to load the model logged using mlflow, you might be facing the issue because the MlflowLogger class in LangChain uses the mlflow.langchain.log_model(chain, "langchain-model") method to log the model. This method might not be compatible with the azure-openai-chat model. You might need to modify the MlflowLogger class to make it compatible with the azure-openai-chat model.

Additionally, the LangChain framework uses the openai Python package for interacting with the OpenAI API. If you are trying to load the azure-openai-chat model, you might need to ensure that the openai Python package is installed and the necessary environment variables are set. You might also need to ensure that the openai Python package is compatible with the azure-openai-chat model.

As for the changes in LangChain version 0.0.300, I couldn't find specific information about changes that could potentially resolve the "Loading azure-openai-chat LLM not supported" error. It would be helpful if you could provide more information about the changes you made or the specific features you used in version 0.0.300.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

kyutcho commented 8 months ago

@prasad-databricks Were you able to find a workaround for this problem? I am experiencing the same with RetrievalQA but still not working :(

danilopeixoto commented 8 months ago

I cannot load chains with llms from langchain.chat_models module exported by MLflow. Is it possible to load llms from chat_models module?

I'm wondering if libs/langchain/langchain/llms/loading.py should also take chat_models into account.

8164

1715

kyutcho commented 8 months ago

@danilopeixoto please refer to answer in #2627

danilopeixoto commented 8 months ago

Thanks @kyutcho!

907Resident commented 8 months ago

I cannot load chains with llms from langchain.chat_models module exported by MLflow. Is it possible to load llms from chat_models module?

I'm wondering if libs/langchain/langchain/llms/loading.py should also take chat_models into account.

8164 #1715

I am having pretty much the same issue as well. I cannot load chains that contain the AzureChatOpenAI model as my LLM. I have looked at response #2627 as @kyutcho mentioned and I could not get my code to work.

I am pasting the modifications to the langchain/llms/__init__.py that I inferred from the issue referenced above. It helped a little because I believe I was able to partially load the model; however, I am attempting to use mlflow.evaluate() with my loaded chain and I am getting an error: ValueErrror: Loading chat prompt not supported. I am assuming that this has to do with the fact that AzureChatOpenAI is not a base llm class for langchain but rather chat_models.

I am using langchain==0.0.321 and mlflow==2.8.0

Click to toggle contents of `__init__.py` ``` """ **LLM** classes provide access to the large language model (**LLM**) APIs and services. **Class hierarchy:** .. code-block:: BaseLanguageModel --> BaseLLM --> LLM --> # Examples: AI21, HuggingFaceHub, OpenAI **Main helpers:** .. code-block:: LLMResult, PromptValue, CallbackManagerForLLMRun, AsyncCallbackManagerForLLMRun, CallbackManager, AsyncCallbackManager, AIMessage, BaseMessage """ # noqa: E501 from typing import Any, Callable, Dict, Type from langchain.llms.base import BaseLLM def _import_ai21() -> Any: from langchain.llms.ai21 import AI21 return AI21 def _import_aleph_alpha() -> Any: from langchain.llms.aleph_alpha import AlephAlpha return AlephAlpha def _import_amazon_api_gateway() -> Any: from langchain.llms.amazon_api_gateway import AmazonAPIGateway return AmazonAPIGateway def _import_anthropic() -> Any: from langchain.llms.anthropic import Anthropic return Anthropic def _import_anyscale() -> Any: from langchain.llms.anyscale import Anyscale return Anyscale def _import_arcee() -> Any: from langchain.llms.arcee import Arcee return Arcee def _import_aviary() -> Any: from langchain.llms.aviary import Aviary return Aviary def _import_azureml_endpoint() -> Any: from langchain.llms.azureml_endpoint import AzureMLOnlineEndpoint return AzureMLOnlineEndpoint def _import_baidu_qianfan_endpoint() -> Any: from langchain.llms.baidu_qianfan_endpoint import QianfanLLMEndpoint return QianfanLLMEndpoint def _import_bananadev() -> Any: from langchain.llms.bananadev import Banana return Banana def _import_baseten() -> Any: from langchain.llms.baseten import Baseten return Baseten def _import_beam() -> Any: from langchain.llms.beam import Beam return Beam def _import_bedrock() -> Any: from langchain.llms.bedrock import Bedrock return Bedrock def _import_bittensor() -> Any: from langchain.llms.bittensor import NIBittensorLLM return NIBittensorLLM def _import_cerebriumai() -> Any: from langchain.llms.cerebriumai import CerebriumAI return CerebriumAI def _import_chatglm() -> Any: from langchain.llms.chatglm import ChatGLM return ChatGLM def _import_clarifai() -> Any: from langchain.llms.clarifai import Clarifai return Clarifai def _import_cohere() -> Any: from langchain.llms.cohere import Cohere return Cohere def _import_ctransformers() -> Any: from langchain.llms.ctransformers import CTransformers return CTransformers def _import_ctranslate2() -> Any: from langchain.llms.ctranslate2 import CTranslate2 return CTranslate2 def _import_databricks() -> Any: from langchain.llms.databricks import Databricks return Databricks def _import_deepinfra() -> Any: from langchain.llms.deepinfra import DeepInfra return DeepInfra def _import_deepsparse() -> Any: from langchain.llms.deepsparse import DeepSparse return DeepSparse def _import_edenai() -> Any: from langchain.llms.edenai import EdenAI return EdenAI def _import_fake() -> Any: from langchain.llms.fake import FakeListLLM return FakeListLLM def _import_fireworks() -> Any: from langchain.llms.fireworks import Fireworks return Fireworks def _import_forefrontai() -> Any: from langchain.llms.forefrontai import ForefrontAI return ForefrontAI def _import_google_palm() -> Any: from langchain.llms.google_palm import GooglePalm return GooglePalm def _import_gooseai() -> Any: from langchain.llms.gooseai import GooseAI return GooseAI def _import_gpt4all() -> Any: from langchain.llms.gpt4all import GPT4All return GPT4All def _import_gradient_ai() -> Any: from langchain.llms.gradient_ai import GradientLLM return GradientLLM def _import_huggingface_endpoint() -> Any: from langchain.llms.huggingface_endpoint import HuggingFaceEndpoint return HuggingFaceEndpoint def _import_huggingface_hub() -> Any: from langchain.llms.huggingface_hub import HuggingFaceHub return HuggingFaceHub def _import_huggingface_pipeline() -> Any: from langchain.llms.huggingface_pipeline import HuggingFacePipeline return HuggingFacePipeline def _import_huggingface_text_gen_inference() -> Any: from langchain.llms.huggingface_text_gen_inference import ( HuggingFaceTextGenInference, ) return HuggingFaceTextGenInference def _import_human() -> Any: from langchain.llms.human import HumanInputLLM return HumanInputLLM def _import_javelin_ai_gateway() -> Any: from langchain.llms.javelin_ai_gateway import JavelinAIGateway return JavelinAIGateway def _import_koboldai() -> Any: from langchain.llms.koboldai import KoboldApiLLM return KoboldApiLLM def _import_llamacpp() -> Any: from langchain.llms.llamacpp import LlamaCpp return LlamaCpp def _import_manifest() -> Any: from langchain.llms.manifest import ManifestWrapper return ManifestWrapper def _import_minimax() -> Any: from langchain.llms.minimax import Minimax return Minimax def _import_mlflow_ai_gateway() -> Any: from langchain.llms.mlflow_ai_gateway import MlflowAIGateway return MlflowAIGateway def _import_modal() -> Any: from langchain.llms.modal import Modal return Modal def _import_mosaicml() -> Any: from langchain.llms.mosaicml import MosaicML return MosaicML def _import_nlpcloud() -> Any: from langchain.llms.nlpcloud import NLPCloud return NLPCloud def _import_octoai_endpoint() -> Any: from langchain.llms.octoai_endpoint import OctoAIEndpoint return OctoAIEndpoint def _import_ollama() -> Any: from langchain.llms.ollama import Ollama return Ollama def _import_opaqueprompts() -> Any: from langchain.llms.opaqueprompts import OpaquePrompts return OpaquePrompts def _import_azure_openai() -> Any: from langchain.llms.openai import AzureOpenAI return AzureOpenAI def _import_azure_openai_chat() -> Any: from langchain.chat_models import AzureChatOpenAI return AzureChatOpenAI def _import_openai() -> Any: from langchain.llms.openai import OpenAI return OpenAI def _import_openai_chat() -> Any: from langchain.llms.openai import OpenAIChat return OpenAIChat def _import_openllm() -> Any: from langchain.llms.openllm import OpenLLM return OpenLLM def _import_openlm() -> Any: from langchain.llms.openlm import OpenLM return OpenLM def _import_pai_eas_endpoint() -> Any: from langchain.llms.pai_eas_endpoint import PaiEasEndpoint return PaiEasEndpoint def _import_petals() -> Any: from langchain.llms.petals import Petals return Petals def _import_pipelineai() -> Any: from langchain.llms.pipelineai import PipelineAI return PipelineAI def _import_predibase() -> Any: from langchain.llms.predibase import Predibase return Predibase def _import_predictionguard() -> Any: from langchain.llms.predictionguard import PredictionGuard return PredictionGuard def _import_promptlayer() -> Any: from langchain.llms.promptlayer_openai import PromptLayerOpenAI return PromptLayerOpenAI def _import_promptlayer_chat() -> Any: from langchain.llms.promptlayer_openai import PromptLayerOpenAIChat return PromptLayerOpenAIChat def _import_replicate() -> Any: from langchain.llms.replicate import Replicate return Replicate def _import_rwkv() -> Any: from langchain.llms.rwkv import RWKV return RWKV def _import_sagemaker_endpoint() -> Any: from langchain.llms.sagemaker_endpoint import SagemakerEndpoint return SagemakerEndpoint def _import_self_hosted() -> Any: from langchain.llms.self_hosted import SelfHostedPipeline return SelfHostedPipeline def _import_self_hosted_hugging_face() -> Any: from langchain.llms.self_hosted_hugging_face import SelfHostedHuggingFaceLLM return SelfHostedHuggingFaceLLM def _import_stochasticai() -> Any: from langchain.llms.stochasticai import StochasticAI return StochasticAI def _import_symblai_nebula() -> Any: from langchain.llms.symblai_nebula import Nebula return Nebula def _import_textgen() -> Any: from langchain.llms.textgen import TextGen return TextGen def _import_titan_takeoff() -> Any: from langchain.llms.titan_takeoff import TitanTakeoff return TitanTakeoff def _import_together() -> Any: from langchain.llms.together import Together return Together def _import_tongyi() -> Any: from langchain.llms.tongyi import Tongyi return Tongyi def _import_vertex() -> Any: from langchain.llms.vertexai import VertexAI return VertexAI def _import_vertex_model_garden() -> Any: from langchain.llms.vertexai import VertexAIModelGarden return VertexAIModelGarden def _import_vllm() -> Any: from langchain.llms.vllm import VLLM return VLLM def _import_vllm_openai() -> Any: from langchain.llms.vllm import VLLMOpenAI return VLLMOpenAI def _import_writer() -> Any: from langchain.llms.writer import Writer return Writer def _import_xinference() -> Any: from langchain.llms.xinference import Xinference return Xinference def _import_yandex_gpt() -> Any: from langchain.llms.yandex import YandexGPT return YandexGPT def __getattr__(name: str) -> Any: if name == "AI21": return _import_ai21() elif name == "AlephAlpha": return _import_aleph_alpha() elif name == "AmazonAPIGateway": return _import_amazon_api_gateway() elif name == "Anthropic": return _import_anthropic() elif name == "Anyscale": return _import_anyscale() elif name == "Arcee": return _import_arcee() elif name == "Aviary": return _import_aviary() elif name == "AzureMLOnlineEndpoint": return _import_azureml_endpoint() elif name == "QianfanLLMEndpoint": return _import_baidu_qianfan_endpoint() elif name == "Banana": return _import_bananadev() elif name == "Baseten": return _import_baseten() elif name == "Beam": return _import_beam() elif name == "Bedrock": return _import_bedrock() elif name == "NIBittensorLLM": return _import_bittensor() elif name == "CerebriumAI": return _import_cerebriumai() elif name == "ChatGLM": return _import_chatglm() elif name == "Clarifai": return _import_clarifai() elif name == "Cohere": return _import_cohere() elif name == "CTransformers": return _import_ctransformers() elif name == "CTranslate2": return _import_ctranslate2() elif name == "Databricks": return _import_databricks() elif name == "DeepInfra": return _import_deepinfra() elif name == "DeepSparse": return _import_deepsparse() elif name == "EdenAI": return _import_edenai() elif name == "FakeListLLM": return _import_fake() elif name == "Fireworks": return _import_fireworks() elif name == "ForefrontAI": return _import_forefrontai() elif name == "GooglePalm": return _import_google_palm() elif name == "GooseAI": return _import_gooseai() elif name == "GPT4All": return _import_gpt4all() elif name == "GradientLLM": return _import_gradient_ai() elif name == "HuggingFaceEndpoint": return _import_huggingface_endpoint() elif name == "HuggingFaceHub": return _import_huggingface_hub() elif name == "HuggingFacePipeline": return _import_huggingface_pipeline() elif name == "HuggingFaceTextGenInference": return _import_huggingface_text_gen_inference() elif name == "HumanInputLLM": return _import_human() elif name == "JavelinAIGateway": return _import_javelin_ai_gateway() elif name == "KoboldApiLLM": return _import_koboldai() elif name == "LlamaCpp": return _import_llamacpp() elif name == "ManifestWrapper": return _import_manifest() elif name == "Minimax": return _import_minimax() elif name == "MlflowAIGateway": return _import_mlflow_ai_gateway() elif name == "Modal": return _import_modal() elif name == "MosaicML": return _import_mosaicml() elif name == "NLPCloud": return _import_nlpcloud() elif name == "OctoAIEndpoint": return _import_octoai_endpoint() elif name == "Ollama": return _import_ollama() elif name == "OpaquePrompts": return _import_opaqueprompts() elif name == "AzureOpenAI": return _import_azure_openai() elif name == "AzureChatOpenAI": return _import_azure_openai_chat() elif name == "OpenAI": return _import_openai() elif name == "OpenAIChat": return _import_openai_chat() elif name == "OpenLLM": return _import_openllm() elif name == "OpenLM": return _import_openlm() elif name == "PaiEasEndpoint": return _import_pai_eas_endpoint() elif name == "Petals": return _import_petals() elif name == "PipelineAI": return _import_pipelineai() elif name == "Predibase": return _import_predibase() elif name == "PredictionGuard": return _import_predictionguard() elif name == "PromptLayerOpenAI": return _import_promptlayer() elif name == "PromptLayerOpenAIChat": return _import_promptlayer_chat() elif name == "Replicate": return _import_replicate() elif name == "RWKV": return _import_rwkv() elif name == "SagemakerEndpoint": return _import_sagemaker_endpoint() elif name == "SelfHostedPipeline": return _import_self_hosted() elif name == "SelfHostedHuggingFaceLLM": return _import_self_hosted_hugging_face() elif name == "StochasticAI": return _import_stochasticai() elif name == "Nebula": return _import_symblai_nebula() elif name == "TextGen": return _import_textgen() elif name == "TitanTakeoff": return _import_titan_takeoff() elif name == "Together": return _import_together() elif name == "Tongyi": return _import_tongyi() elif name == "VertexAI": return _import_vertex() elif name == "VertexAIModelGarden": return _import_vertex_model_garden() elif name == "VLLM": return _import_vllm() elif name == "VLLMOpenAI": return _import_vllm_openai() elif name == "Writer": return _import_writer() elif name == "Xinference": return _import_xinference() elif name == "YandexGPT": return _import_yandex_gpt() elif name == "type_to_cls_dict": # for backwards compatibility type_to_cls_dict: Dict[str, Type[BaseLLM]] = { k: v() for k, v in get_type_to_cls_dict().items() } return type_to_cls_dict else: raise AttributeError(f"Could not find: {name}") __all__ = [ "AI21", "AlephAlpha", "AmazonAPIGateway", "Anthropic", "Anyscale", "Arcee", "Aviary", "AzureMLOnlineEndpoint", "AzureOpenAI", "AzureChatOpenAI", "Banana", "Baseten", "Beam", "Bedrock", "CTransformers", "CTranslate2", "CerebriumAI", "ChatGLM", "Clarifai", "Cohere", "Databricks", "DeepInfra", "DeepSparse", "EdenAI", "FakeListLLM", "Fireworks", "ForefrontAI", "GPT4All", "GooglePalm", "GooseAI", "GradientLLM", "HuggingFaceEndpoint", "HuggingFaceHub", "HuggingFacePipeline", "HuggingFaceTextGenInference", "HumanInputLLM", "KoboldApiLLM", "LlamaCpp", "TextGen", "ManifestWrapper", "Minimax", "MlflowAIGateway", "Modal", "MosaicML", "Nebula", "NIBittensorLLM", "NLPCloud", "Ollama", "OpenAI", "OpenAIChat", "OpenLLM", "OpenLM", "PaiEasEndpoint", "Petals", "PipelineAI", "Predibase", "PredictionGuard", "PromptLayerOpenAI", "PromptLayerOpenAIChat", "OpaquePrompts", "RWKV", "Replicate", "SagemakerEndpoint", "SelfHostedHuggingFaceLLM", "SelfHostedPipeline", "StochasticAI", "TitanTakeoff", "Tongyi", "VertexAI", "VertexAIModelGarden", "VLLM", "VLLMOpenAI", "Writer", "OctoAIEndpoint", "Xinference", "JavelinAIGateway", "QianfanLLMEndpoint", "YandexGPT", ] def get_type_to_cls_dict() -> Dict[str, Callable[[], Type[BaseLLM]]]: return { "ai21": _import_ai21, "aleph_alpha": _import_aleph_alpha, "amazon_api_gateway": _import_amazon_api_gateway, "amazon_bedrock": _import_bedrock, "anthropic": _import_anthropic, "anyscale": _import_anyscale, "arcee": _import_arcee, "aviary": _import_aviary, "azure": _import_azure_openai, "azureml_endpoint": _import_azureml_endpoint, "azure-openai-chat":_import_azure_openai_chat, "bananadev": _import_bananadev, "baseten": _import_baseten, "beam": _import_beam, "cerebriumai": _import_cerebriumai, "chat_glm": _import_chatglm, "clarifai": _import_clarifai, "cohere": _import_cohere, "ctransformers": _import_ctransformers, "ctranslate2": _import_ctranslate2, "databricks": _import_databricks, "deepinfra": _import_deepinfra, "deepsparse": _import_deepsparse, "edenai": _import_edenai, "fake-list": _import_fake, "forefrontai": _import_forefrontai, "google_palm": _import_google_palm, "gooseai": _import_gooseai, "gradient": _import_gradient_ai, "gpt4all": _import_gpt4all, "huggingface_endpoint": _import_huggingface_endpoint, "huggingface_hub": _import_huggingface_hub, "huggingface_pipeline": _import_huggingface_pipeline, "huggingface_textgen_inference": _import_huggingface_text_gen_inference, "human-input": _import_human, "koboldai": _import_koboldai, "llamacpp": _import_llamacpp, "textgen": _import_textgen, "minimax": _import_minimax, "mlflow-ai-gateway": _import_mlflow_ai_gateway, "modal": _import_modal, "mosaic": _import_mosaicml, "nebula": _import_symblai_nebula, "nibittensor": _import_bittensor, "nlpcloud": _import_nlpcloud, "ollama": _import_ollama, "openai": _import_openai, "openlm": _import_openlm, "pai_eas_endpoint": _import_pai_eas_endpoint, "petals": _import_petals, "pipelineai": _import_pipelineai, "predibase": _import_predibase, "opaqueprompts": _import_opaqueprompts, "replicate": _import_replicate, "rwkv": _import_rwkv, "sagemaker_endpoint": _import_sagemaker_endpoint, "self_hosted": _import_self_hosted, "self_hosted_hugging_face": _import_self_hosted_hugging_face, "stochasticai": _import_stochasticai, "together": _import_together, "tongyi": _import_tongyi, "titan_takeoff": _import_titan_takeoff, "vertexai": _import_vertex, "vertexai_model_garden": _import_vertex_model_garden, "openllm": _import_openllm, "openllm_client": _import_openllm, "vllm": _import_vllm, "vllm_openai": _import_vllm_openai, "writer": _import_writer, "xinference": _import_xinference, "javelin-ai-gateway": _import_javelin_ai_gateway, "qianfan_endpoint": _import_baidu_qianfan_endpoint, "yandex_gpt": _import_yandex_gpt, } ```
danilopeixoto commented 7 months ago

I'm currently using this patch:

import mlflow

import langchain.agents
from langchain.chains import LLMChain
from langchain.chat_models import ChatMLflowAIGateway
from langchain.llms import huggingface_hub
from langchain.prompts import PromptTemplate

gateway = ChatMLflowAIGateway(
    route='gpt-3.5-turbo',
    params={
        'temperature': 0.25
    }
)

llm_chain = LLMChain(
    llm=gateway,
    prompt=PromptTemplate(
        input_variables=['adjective'],
        template='Tell me a {adjective} joke'
    )
)

# Run model
result = llm_chain.run(adjective='funny')
print(result)

# Log model
with mlflow.start_run():
    model_info = mlflow.langchain.log_model(llm_chain, 'model')

# Load model
from langchain.llms import loading
loading.get_type_to_cls_dict = lambda: {
    ChatMLflowAIGateway._llm_type.fget(None): lambda: ChatMLflowAIGateway
}

model = mlflow.pyfunc.load_model(model_info.model_uri)
print(model.predict([{'adjective': 'funny'}]))
kyutcho commented 7 months ago 
from lanchain.llms import type_to_cls_dict

type_to_cls_dict["openai-chat"] = OpenAIChat
# similarly for other chat models like AzureOpenAIChat etc.

@907Resident This didn't work for you?

907Resident commented 7 months ago 
from lanchain.llms import type_to_cls_dict

type_to_cls_dict["openai-chat"] = OpenAIChat
# similarly for other chat models like AzureOpenAIChat etc.

@907Resident This didn't work for you?

@kyutcho, no unfortunately it did not work for me. It was the first thing I tried. I added:

from langchain.llms import type_to_cls_dict

type_to_cls_dict["azure-openai-chat"] = AzureChatOpenAI

and I continued to get the same ValueError that indicated "azure-openai-chat" cannot be loaded.

I am going to try @danilopeixoto's work around next.

renata-gotler commented 7 months ago

I'm getting the same error.

I had to use AzureChatOpenAI which is not inside llm module from langchain, since gpt-4 doesn't work with AzureOpenAI module, then this error showed up. None of the workaround mentioned above worked, since gpt-4 is not supported for provider Azure OpenAI using the mlflow gateway https://mlflow.org/docs/latest/llms/gateway/index.html#providers.

Blackkadder commented 7 months ago

I'm facing the same issue

mabreuortega commented 6 months ago

same thing here

michal-adach commented 6 months ago

facing the same issue

mmrgiken commented 4 months ago

same "ValueError: Loading azure-openai-chat LLM not supported"

opalchonlapat commented 3 months ago

I found the same issue when using mlflow with AzureChatOpenAI from langchain MlflowException: Unsupported type azure-openai-chat for loading

green-sh commented 2 months ago

You can work around it by monkey patching before loading the model:

example:

def _import_azure_openai_chat():
    from langchain_openai.chat_models.azure import AzureChatOpenAI

    return AzureChatOpenAI

flavors = lambda: {"azure-openai-chat": _import_azure_openai_chat, ...} # add other flavors you want to use 
langchain.llms.get_type_to_cls_dict = flavors
langchain.llms.llms_get_type_to_cls_dict = flavors
langchain_community.llms.get_type_to_cls_dict = flavors

note that this is just a code snipped, you probably don't want to completly replace all functions. Instead you want to add the other flavors you want to use. If this doesn't work you can try to debug what else is missing.

green-sh commented 2 months ago

looks like this is worked on in the mlflow repo 11644. There also exists another issue related to this issue by the same author #18420

opalchonlapat commented 2 months ago

looks like this is worked on in the mlflow repo 11644. There also exists another issue related to this issue by the same author #18420

When will they release new version to support this feature?

gwd666 commented 2 months ago

still same issue ...