Corrected and refactored main.py for 'markdown-validator' example

[alphaveneno]

just a single line for .env file:

.env

OPENAI_API_KEY=insert your openai key here

Note: best-practice is not to put any type of quotation marks around the value of a key, tuck it in close to the equals sign

main.py

import sys, os
from crewai import Agent, Task
from dotenv import load_dotenv
# from langchain_community.chat_models import ChatOllama
from langchain_openai import ChatOpenAI
from MarkdownTools import markdown_validation_tool

load_dotenv()

api_key = os.environ['OPENAI_API_KEY']

"""
option for open-source models:

1. must install and setup ollama: https://ollama.com/
2. see codewithbrandon tutorial on how to install ollama
and how to reconfigure open-source models for crewAi

https://www.youtube.com/watch?v=0ai-L50VCYU&t=1101s

tested: llama3, qwen2 and mistral
of these three only mistral is working for me right now,
however this varies day-to-day

open-source seems to work much better if you make
this change in MarkdownTools.py:

#from langchain.tools import tool
from crewai_tools import tool

"crewai-llama3:latest" is my reconfigured version of llama3
"""

"""
Although ChatOllama works well with Langchain
it currently  (July, 2024) lacks the necessary features
to work well with CrewAI. I present the code below only to
show how it should look some day
"""
# llm = ChatOllama(
#     model="crewai-llama3:latest"
# )

"""
Open-source LLMs are accessible in CrewAI via
the configuration of ChatOpenAI shown below.
http://localhost:11434/v1 accesses the ollama-server
which in turn serves the open-source LLMs on your own ollama list
api_key='NA' keeps ChatOpenAI from throwing an error
because it did not get an api key
"""
# llm = ChatOpenAI(
#                        base_url="http://localhost:11434/v1",
#                        api_key='NA',
#                        temperature=0.1,
#                       model="crewai-llama3:latest",
#                        top_p=0.3)

##########################

"""
Unlike Langchain, to work with OpenAI in crewAI,
ChatOpenAI must access the remote OpenAI server.
It does so with this line of code:
openai_api_base="https://api.openai.com/v1",
o/w LLMs are instantiated with ChatOpenAI
the same way as in Langchain.
Interesting, the first invocation of openai_api_base will
last the whole session, however you should still leave
the code in
"""
llm = ChatOpenAI(
                openai_api_base="https://api.openai.com/v1",
                api_key=api_key,
                temperature=0.1,
                # model="gpt-3.5-turbo",
                model="gpt-4o-mini",
                top_p=0.3)

def process_markdown_document(filename):
    """
    Processes a markdown document by reviewing its syntax validation 
    results and providing feedback on necessary changes.

    Args:
        filename (str): The path to the markdown file to be processed.

    Returns:
        str: The list of recommended changes to make to the document.

    """

    # Define general agent
    general_agent  = Agent(role='Requirements Manager',
                    goal="""Provide a detailed list of the markdown 
                            linting results. Give a summary with actionable 
                            tasks to address the validation results. Write your 
                            response as if you were handing it to a developer 
                            to fix the issues.
                            DO NOT provide examples of how to fix the issues or
                            recommend other tools to use.""",
                    backstory="""You are an expert business analyst 
                    and software QA specialist. You provide high quality, 
                    thorough, insightful and actionable feedback via 
                    detailed list of changes and actionable tasks.""",
                    allow_delegation=False, 
                    verbose=True,
                    tools=[markdown_validation_tool],
                    llm=llm)

    # Define Tasks Using Crew Tools
    syntax_review_task = Task(
        description=f"""
        Use the markdown_validation_tool to review 
        the file(s) at this path: {filename}

        Be sure to pass only the file path to the markdown_validation_tool.
        Use the following format to call the markdown_validation_tool:
        Do I need to use a tool? Yes
        Action: markdown_validation_tool
        Action Input: {filename}

        Get the validation results from the tool 
        and then summarize it into a list of changes
        the developer should make to the document.
        DO NOT recommend ways to update the document.
        DO NOT change any of the content of the document or
        add content to it. It is critical to your task to
        only respond with a list of changes.

        If you already know the answer or if you do not need 
        to use a tool, return it as your Final Answer.""",
        expected_output="""A detailed list of changes the developer should make
        to the document based on the markdown validation results.
        The list should be actionable and specific,
        without recommending ways to update the document or changing its content.""",
        agent=general_agent)

# depending on your dependencies, 
# one of these two lines of code will work, 
# and the other will not (see comment below)
#   updated_markdown = syntax_review_task.execute()
    updated_markdown = syntax_review_task.execute_sync()

    return updated_markdown

# Note: application is installed here with poetry
# any python package manager(anaconda, mamba, pip, pdm, etc.) can be used

# from Linux bash terminal:
# poetry shell
# poetry run python main.py README.md
if __name__ == "__main__":

    if len(sys.argv) > 1:
        filename = sys.argv[1]
        processed_document = process_markdown_document(filename)
        print(processed_document)

[alphaveneno]

Credit: @TheCyberTech

[alphaveneno]

Dependencies and function calls

This comment is relevant to at least installations made with poetry

I made two installations (remember to use poetry install --no-init not poetry install), one with the original set of dependencies and one with dependencies up-to-date as of July 22/2024.

1. original dependencies (with slight modification):

[tool.poetry.dependencies] python = ">=3.10.0,<3.12" crewai = ">=0.11.0" python-dotenv = "1.0.0" markdown = "3.4.3" pymarkdownlnt = "0.9.15" mdurl = "^0.1.2" langchain-community = "^0.2.9" crewai-tools = "^0.4.26" openai = "^1.36.1"

This works with:

updated_markdown = syntax_review_task.execute_sync()

2. updated dependencies:

[tool.poetry.dependencies] python = "<=3.13,>=3.10" crewai = "^0.36.1" python-dotenv = "^1.0.1" markdown = "^3.6" pymarkdownlnt = "^0.9.21" mdurl = "^0.1.2" langchain-community = "^0.2.7" crewai-tools = "^0.4.26"

this works with:

updated_markdown = syntax_review_task.execute()

[pravincoder]

1. original dependencies (with slight modification):

[tool.poetry.dependencies] python = ">=3.10.0,<3.12" crewai = ">=0.11.0" python-dotenv = "1.0.0" markdown = "3.4.3" pymarkdownlnt = "0.9.15" mdurl = "^0.1.2" langchain-community = "^0.2.9" crewai-tools = "^0.4.26" openai = "^1.36.1"

This works with:

updated_markdown = syntax_review_task.execute_sync()

2. updated dependencies:

[tool.poetry.dependencies] python = "<=3.13,>=3.10" crewai = "^0.36.1" python-dotenv = "^1.0.1" markdown = "^3.6" pymarkdownlnt = "^0.9.21" mdurl = "^0.1.2" langchain-community = "^0.2.7" crewai-tools = "^0.4.26"

this works with:

updated_markdown = syntax_review_task.execute()

@alphaveneno

My dependences in project.toml are :-

[tool.poetry.dependencies] python = ">=3.10.0,<3.12" crewai = "^0.11.0" python-dotenv = "1.0.0" markdown = "3.4.3" pymarkdownlnt = "0.9.15"

for me the original code work's :- updated_markdown = syntax_review_task.execute()
Just wanted you to know😵

Also i came here to ask , As you post is recent did you encounter any Tool usage error like below :- Output :-

Here is my response:

Thought: Do I need to use a tool? Yes
Action: markdown_validation_tool
Action Input: README.md 

It seems we encountered an unexpected error while trying to use the tool. This was the error: Failed to parse ToolCalling from completion {"$schema": "http://-schema.org/draft-07/schema#", "type": "object", "properties": {"tool_name": {"type": "string"}, "arguments": {"type": "object", "required": ["file_path"], "additionalProperties": false, "properties": {"file_path": {"type": "string"}}}}, "required": ["tool_name", "arguments"]}. Got: 2 validation errors for ToolCalling
tool_name
  field required (type=value_error.missing)
arguments
  field required (type=value_error.missing)

'str' object has no attribute 'tool_name' 

[alphaveneno]

1. original dependencies (with slight modification):

[tool.poetry.dependencies] python = ">=3.10.0,<3.12" crewai = ">=0.11.0" python-dotenv = "1.0.0" markdown = "3.4.3" pymarkdownlnt = "0.9.15" mdurl = "^0.1.2" langchain-community = "^0.2.9" crewai-tools = "^0.4.26" openai = "^1.36.1" This works with: updated_markdown = syntax_review_task.execute_sync()

2. updated dependencies:

[tool.poetry.dependencies] python = "<=3.13,>=3.10" crewai = "^0.36.1" python-dotenv = "^1.0.1" markdown = "^3.6" pymarkdownlnt = "^0.9.21" mdurl = "^0.1.2" langchain-community = "^0.2.7" crewai-tools = "^0.4.26" this works with: updated_markdown = syntax_review_task.execute()

@alphaveneno

My dependences in project.toml are :-

[tool.poetry.dependencies] python = ">=3.10.0,<3.12" crewai = "^0.11.0" python-dotenv = "1.0.0" markdown = "3.4.3" pymarkdownlnt = "0.9.15"

for me the original code work's :- updated_markdown = syntax_review_task.execute() Just wanted you to know😵

Also i came here to ask , As you post is recent did you encounter any Tool usage error like below :- Output :-

Here is my response:

Thought: Do I need to use a tool? Yes
Action: markdown_validation_tool
Action Input: README.md 

It seems we encountered an unexpected error while trying to use the tool. This was the error: Failed to parse ToolCalling from completion {"$schema": "http://-schema.org/draft-07/schema#", "type": "object", "properties": {"tool_name": {"type": "string"}, "arguments": {"type": "object", "required": ["file_path"], "additionalProperties": false, "properties": {"file_path": {"type": "string"}}}}, "required": ["tool_name", "arguments"]}. Got: 2 validation errors for ToolCalling
tool_name
  field required (type=value_error.missing)
arguments
  field required (type=value_error.missing)

'str' object has no attribute 'tool_name' 

No, I cannot say I had any problem with the tools.

Glad to hear the code worked for you. I have a suspicion the problem is the result of some package conflicts in my virtual environment.