search

gnosis / prediction-market-agent

GNU Lesser General Public License v3.0
23 stars 4 forks source link

Adapt prompt to make sure it also calls other functions (e.g. sell) #146

Open gabrielfior opened 4 months ago

gabrielfior commented 4 months ago

We should improve the prompt for the general agent to make sure of the following:

-> It should also call sell function for positions it thinks are undervalued - according to Martin, even an explicit command like “Reasoning(“Check all you positions and reason wether you should hold or sell it.“)” didn't work -> It should be capable to create bets of different sizes, not only $0.10 -> Predictions for events seem to neglect time part of the question:

(from Martin's thread) Unrelated to microchain: for the predictions exact time still seems to play too little role: “>> Reasoning(“The predicted probability of the event (‘Will there be another deadly pandemic by 8 May 2024?’) is 0.3” - basically it seems to give you a % for the general likelihood for something but it does not consider the exact date. If today is 7th of May the likelihood for a global deadly pandamic TOMORROW (8th of May) should essentially be 0 and not 30%.

kongzii commented 4 months ago

-> Predictions for events seem to neglect time part of the question:

Isn't that problem because it's using PredictProbabilityForQuestionLocal? We can change it to ThinkThoroughly's predictions.

evangriffiths commented 4 months ago

I think could be made into 3 separate GH issues:

  1. It should also call sell function for positions it thinks are undervalued -- a duplicate of https://github.com/gnosis/prediction-market-agent-tooling/issues/240, so this is resolved.

  2. It should be capable to create bets of different sizes, not only $0.10 -- it can create a bet of size >0, <= 0.1 atm. We can remove the 0.1xDai cap, but test that the prompt is sufficient to stop the agent betting all its funds in one go.

  3. Predictions for events seem to neglect time part of the question: -- @kongzii agreed, it's just using a not very good mech atm.

gabrielfior commented 3 months ago

  1. It should also call sell function for positions it thinks are undervalued -- a duplicate of Fix OmenAgentMarket.sell_tokens prediction-market-agent-tooling#240, so this is resolved.

I might have misunderstood, but I believe https://github.com/valory-xyz/mech/pull/237/files# i is not fixing this - I tried running poetry run python prediction_market_agent/agents/microchain_agent/microchain_agent.py and the agent doesn't call sellTokens at all. Note that the referenced branch had been merged when I tested this.

-> One thing we could try is to run the microchain_agent with a wallet that already has a few open positions, thus "forcing" it to recognize the positions and selling a few outcome tokens. -> In my mind, the problem is that the agent doesn't know when/whether it should sell tokens, i.e. it can't reason about the market_condition and the current value of the outcome_tokens. If this logic were to be added, I think the agent would call sell_tokens more often.

kongzii commented 3 months ago

In my mind, the problem is that the agent doesn't know when/whether it should sell tokens

Great opportunity for a new LearnAboutPredictionMarketWhenToSellBets function! 😄

evangriffiths commented 3 months ago

I might have misunderstood, but I believe https://github.com/gnosis/prediction-market-agent-tooling/issues/240 is not fixing this

Sorry, to be clearer, https://github.com/gnosis/prediction-market-agent-tooling/issues/240 meant that it became functionally possible for the agent to call sell on all of its positions returned from GetPositions - without changing how often it actually tried to do it.

-> In my mind, the problem is that the agent doesn't know when/whether it should sell tokens, i.e. it can't reason about the market_condition and the current value of the outcome_tokens. If this logic were to be added, I think the agent would call sell_tokens more often.

I think it has sufficient info to make this decision atm. It could do something like: "I have a position YES in market X, GetMarketProbability(X) is greater than PredictProbabilityForQuestionLocal(X), therefore I think my position is overvalued, so I should sell.

Agreed that it is missing info on how much it spend to get those shares, so doesn't know if it's making a profit or loss. So that could be useful too, but not essential.

evangriffiths commented 2 months ago

Closing crteria: See the agent call SellTokens by itself https://deployed-general-agent-viewer.ai.gnosisdev.com/