joaomdmoura / crewAI

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
https://crewai.com
MIT License
16.73k stars 2.26k forks source link

Claude-3 model usage_metrics #485

Open jonny7737 opened 2 months ago

jonny7737 commented 2 months ago

I am building a crew trask/agent by task/agent (one at a time). This process works but is a little chatty. Chatty makes me curious about usage so I asked - response from crew.usage_metrics: {'total_tokens': 0, 'prompt_tokens': 0, 'completion_tokens': 0, 'successful_requests': 0}

How do I get usage stats when using a Claude-3 model?

xssMociN commented 1 month ago

I'm having the same issue. I looked at the code and I think I found where the issue lies.

In agent.py, there is this piece of code.

@model_validator(mode="after")
    def set_agent_executor(self) -> "Agent":
        """set agent executor is set."""
        if hasattr(self.llm, "model_name"):
            self.llm.callbacks = [
                TokenCalcHandler(self.llm.model_name, self._token_process)
            ]
        if not self.agent_executor:
            self.set_cache_handler(self.cache_handler)
        return self

In my case, if hasattr(self.llm, "model_name"): would return false since the Claude 3 model doesn't contain the model_name attribute.

I am using the langchain_aws ChatBedrock class to instantiate my LLM.

crewai version: 0.28.8 langchain-aws: 0.1.3