search

gnosis / prediction-market-agent

GNU Lesser General Public License v3.0
22 stars 4 forks source link

Look for improvements in think-thoroughly agent's predictions in metaculus #371

Open evangriffiths opened 1 month ago

evangriffiths commented 1 month ago

See predictions here: https://www.metaculus.com/accounts/profile/192049/, and look for common themes with why the predictions are incorrect.

Relates to <TODO link 'storing agent histories in langfuse/postgres' issue>, as we don't have the full research report (I don't think?)

One reason might be this TODO here https://github.com/gnosis/prediction-market-agent/blob/main/prediction_market_agent/agents/metaculus_agent/deploy.py#L62

You can login to the agent here https://share.1password.com/s#Px7f3Wu9yMMGqdx4f6-Z0DMfI0tQqaaO8bOK6ftPBLc but I don't think there's much more to see. You can look at other bots 'reasoning' by looking at the comments on the markets here https://www.metaculus.com/project/aibq3/, but actual predictions are hidden until the markets are resolved.

kongzii commented 4 weeks ago

This is a good use-case to test out Langfuse once it's merged https://github.com/gnosis/prediction-market-agent/pull/398

gabrielfior commented 3 weeks ago

Will look into this once I'm done with my open PRs