[BUG] facing wront llm mapping to OpenAI insted of AzureAI in VisionTool()

[anslin-raj]

Description

I have encountered an error while trying to use the VisionTool(), while I'm trying to use AzureAI with the vision tool, but it wrongly mapped to the OpenAI API,

can we use AzureOpenAI in the vision tool?

error:

I encountered an error while trying to use the tool. This was the error: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable. Tool Vision Tool accepts these inputs: Vision Tool(image_path_url: 'string') - This tool uses OpenAI's Vision API to describe the contents of an image.

vision_tool = VisionTool( config= { "llm": { "provider": "azure_openai", "config": { "api_key": os.getenv('AZURE_OPENAI_API_KEY'), "model": os.getenv('AZURE_MODEL'), "deployment_name": os.getenv('AZURE_MODEL') } } } )

Steps to Reproduce

use the above code with the AzureAI credits.

Expected behavior

Run with AzureAI

Screenshots/Code snippets

from crewai import Agent, Crew, Task from langchain_openai import AzureChatOpenAI from openai import AzureOpenAI, OpenAI from pdf2image import convert_from_path from pydantic import BaseModel from typing import List from textwrap import dedent from crewai_tools import VisionTool import os

vision_tool = VisionTool( config= { "llm": { "provider": "azure_openai", "config": { "api_key": os.getenv('AZURE_OPENAI_API_KEY'), "model": os.getenv('AZURE_MODEL'), "deployment_name": os.getenv('AZURE_MODEL') } } } )

def Document_content_analyzer_agent(path_of_page): agent = Agent( role="Document Content Analyst", goal=dedent(""" ... """),

backstory=( """... ... ... ... """ ), allow_delegation=False, tools=[vision_tool], llm=azure_llm,

llm=LLM_GPT4o,

verbose=True, memory=False)

task = Task( description=dedent(f""" page_path = {path_of_page} ... ... ... ... """), expected_output=""" "figures,charts,flowchart,plots and tables informations only." """, agent=agent, human_input=False)

crew_results_editor = Crew( agents=[agent], tasks=[task], verbose=True ) output_editor = crew_results_editor.kickoff() return output_editor.raw

Operating System

Ubuntu 20.04

Python Version

3.10

crewAI Version

0.41.1

crewAI Tools Version

0.13.2

Virtual Environment

Venv

Evidence

I encountered an error while trying to use the tool. This was the error: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable. Tool Vision Tool accepts these inputs: Vision Tool(image_path_url: 'string') - This tool uses OpenAI's Vision API to describe the contents of an image.

vision_tool = VisionTool( config= { "llm": { "provider": "azure_openai", "config": { "api_key": os.getenv('AZURE_OPENAI_API_KEY'), "model": os.getenv('AZURE_MODEL'), "deployment_name": os.getenv('AZURE_MODEL') } } } )

Possible Solution

Currently No

Additional context

NA

[bhancockio]

Hey @anslin-raj !

Could you please provide the full error?