search

jgravelle / pocketgroq

121 stars 35 forks source link

How to solve URL and probably other hallucinations? #9

Open PieBru opened 2 hours ago

PieBru commented 2 hours ago

Hi J, just tryed AutoGroqAgent Chat, it doesn't solve URL hallucinations.

This is my query:

Search the medical domain for authoritative Systematic Reviews or Meta-Analysis of Randomized Controlled Trials, that prove the efficacy of "Saw Palmetto" (Serenoa repens) in improving BPH.

Exclude: 
  1) commercial or biased sites;
  2) publications dated before 2019.
  2) publications with PubMed impact Factor less than 4.

Rank the remaining publications by impact factor and list only the top 5 publications.
Give the public links to the listed publications only if the URL points to the correct corresponding publication.

Any hint? Thank you from Italy. Piero

Edit: forgot to mention I checked "Double check".

jgravelle commented 2 hours ago

Be sure to set the temperature to 0 and try different models. If you pull down the full repo, you can also bump up the retries from 5 to 50.

Buona fortuna, amico mio...!

PieBru commented 1 hour ago

Sorry to bother you, but I'm not fluent in Python. After modifying test.py:

def test_autonomous_agent():
    print("\nTesting Autonomous Agent...")
    agent = AutonomousAgent(groq, max_sources=50)  # Limit to 3 sources for faster testing

    request = """
Search the medical domain for authoritative Systematic Reviews or Meta-Analysis of Randomized Controlled Trials, that prove the efficacy of "Saw Palmetto" (Serenoa repens) in improving BPH.

Exclude: 
  1) commercial or biased sites;
  2) publications dated before 2019.
  2) publications with PubMed impact Factor less than 4.

Rank the remaining publications by impact factor and list only the top 5 publications.
Give the public links to the listed publications only if the URL points to the correct corresponding publication.
"""
    response = agent.process_request(request) 
...

... choice 25 doesn't look at 50 sources and ends as before:

Response is satisfactory: False
Assertion Error: The autonomous agent should provide a satisfactory response

Btw, my profile picture is from 25 years ago, so please help ;) Piero