search

gnosis / prediction-market-agent-tooling

Tools to benchmark, deploy and monitor prediction market agents.
GNU Lesser General Public License v3.0
18 stars 4 forks source link

Add cheap way of deciding whether to re-run predictions for long-running markets #487

Open evangriffiths opened 1 month ago

evangriffiths commented 1 month ago

Follow-on from https://github.com/gnosis/prediction-market-agent-tooling/issues/485

Current behaviour, defined in DeployableTraderAgent.verify_market, is for agent to skip the market if it has bet on it in the last 24 hours.

If agents place bets on long running markets, this will be too frequent. But it could involve some combination of:

gabrielfior commented 1 month ago

using a news API, and seeing if something relevant has happened since the last bet (must be v. cheap to be worth using instead of a full prediction)

This is the hardest but also the best option for me. One option that comes to mind is e.g. monitoring "twitter trends" from the last week and seeing if it has high correlation with any market on Omen and order markets by that. This is kind of an alternative to the newsfeed API.

kongzii commented 1 month ago

We could use our market-insights API for this. It uses Tavily to get news. It's cached for 3 days and we expect that it will be used a lot, so the "price per agent" is cheap.

evangriffiths commented 1 month ago

I've just had a look at this. It's a nice idea, but the the output is not really suitable for the use case. e.g. for the market with title "Will there be more recorded forest fires in the US and Canada in 2024 than 2023?"

the summary you get is

'Forest fires have been increasingly severe, with 2023 marking the worst year on record globally. In Canada, the 2023 wildfire season was unprecedented, burning six times more forest than the average of the previous 20 years. Experts attribute this to climate change, though they note that years like 2023 might be rare. As 2024 unfolds, early signs suggest another intense fire season. Consider these factors and conduct thorough research before making any predictions in the market.'

What we want is different in 2 ways:

But I forgot that Tavily also have a news API feature (see docs), where you can do e.g. search(topic="news", days=14). I'll give that a go!

Also good point on the caching. The queries will be identical for all agents, so it makes sense to use it. Even just using existing TavilyStorage should save a lot.