search

jgravelle / pocketgroq

PocketGroq is a powerful Python library that simplifies integration with the Groq API, offering advanced features for natural language processing, web scraping, and autonomous agent capabilities. Key Features Seamless integration with Groq API for text generation and completion Chain of Thought (CoT) reasoning for complex problem-solving and more.
https://j.gravelle
156 stars 48 forks source link

How to solve URL and probably other hallucinations? #9

Open PieBru opened 1 month ago

PieBru commented 1 month ago

Hi J, just tryed AutoGroqAgent Chat, it doesn't solve URL hallucinations.

This is my query:

Search the medical domain for authoritative Systematic Reviews or Meta-Analysis of Randomized Controlled Trials, that prove the efficacy of "Saw Palmetto" (Serenoa repens) in improving BPH.

Exclude: 
  1) commercial or biased sites;
  2) publications dated before 2019.
  2) publications with PubMed impact Factor less than 4.

Rank the remaining publications by impact factor and list only the top 5 publications.
Give the public links to the listed publications only if the URL points to the correct corresponding publication.

Any hint? Thank you from Italy. Piero

Edit: forgot to mention I checked "Double check".

jgravelle commented 1 month ago

Be sure to set the temperature to 0 and try different models. If you pull down the full repo, you can also bump up the retries from 5 to 50.

Buona fortuna, amico mio...!

PieBru commented 1 month ago

Sorry to bother you, but I'm not fluent in Python. After modifying test.py:

def test_autonomous_agent():
    print("\nTesting Autonomous Agent...")
    agent = AutonomousAgent(groq, max_sources=50)  # Limit to 3 sources for faster testing

    request = """
Search the medical domain for authoritative Systematic Reviews or Meta-Analysis of Randomized Controlled Trials, that prove the efficacy of "Saw Palmetto" (Serenoa repens) in improving BPH.

Exclude: 
  1) commercial or biased sites;
  2) publications dated before 2019.
  2) publications with PubMed impact Factor less than 4.

Rank the remaining publications by impact factor and list only the top 5 publications.
Give the public links to the listed publications only if the URL points to the correct corresponding publication.
"""
    response = agent.process_request(request) 
...

... choice 25 doesn't look at 50 sources and ends as before:

Response is satisfactory: False
Assertion Error: The autonomous agent should provide a satisfactory response

Btw, my profile picture is from 25 years ago, so please help ;) Piero

jgravelle commented 1 month ago

For this we're going to need PocketGroq's chain-of-thought mechanism to call our new autonomous agent rather than going the more generic '.generate' route. That's where we're headed, but we're not there yet...