[MALFUNCTION] Service stopped working after update

[mkuzmentsov]

I attempted update of the service as described in README.md , after wich it stopped trading. Log output:

[ec2-user@ip-172-31-25-255 trader]$ poetry run autonomy analyse logs --from-dir trader_service/abci_build/persistent_data/logs/ --agent aea_0 --reset-db
--- aea_0 ---
[2024-06-18 13:45:45.926000][WARNING] The same event 'Event.FETCH_ERROR' has been found in several apps:
<class 'packages.valory.skills.decision_maker_abci.rounds.DecisionMakerAbciApp'>
<class 'packages.valory.skills.market_manager_abci.rounds.MarketManagerAbciApp'>
It will be interpreted as the same event. If this is not the intended behaviour, please rename it to enforce its uniqueness.

[2024-06-18 13:45:45.937000][WARNING] The same event 'Event.FETCH_ERROR' has been found in several apps:
<class 'packages.valory.skills.decision_maker_abci.rounds.DecisionMakerAbciApp'>
<class 'packages.valory.skills.market_manager_abci.rounds.MarketManagerAbciApp'>
It will be interpreted as the same event. If this is not the intended behaviour, please rename it to enforce its uniqueness.

[2024-06-18 13:45:45.993000][WARNING] The kwargs={'mech_request_price': None, 'mech_chain_id': 'ethereum', 'mech_wrapped_native_token_address': '0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d', 'mech_to_subscription_params': [['base_url', 'https://marketplace-api.gnosis.nevermined.app/api/v1/metadata/assets/ddo'], ['did', 'did:nv:0ea01d5de3b34e3792db825f2a5f5595c393c68b19fd5efdacd00fcc63a53483'], ['escrow_payment_condition_address', '0x9dDC4F1Ea5b94C138A23b60EC48c0d01d172629a'], ['lock_payment_condition_address', '0xDE85A368Ee6f374d236500d176814365370778dA'], ['transfer_nft_condition_address', '0xbBa4A25262745a55f020D0a3E9a82c25bb6F4979'], ['token_address', '0x80A9b55F8604acC26dF2Ac6e07F9dC5B0eAa05Ce'], ['order_address', '0xc7751eff5396a846e7bc83ac31d3cb7d37cb49e4'], ['nft_amount', '100'], ['payment_token', '0x0000000000000000000000000000000000000000'], ['order_address', '0xc7751eff5396a846e7bc83ac31d3cb7d37cb49e4'], ['price', '1000000000000000000']], 'file_hash_to_strategies_json': [['bafybeihufqu2ra7vud4h6g2nwahx7mvdido7ff6prwnib2tdlc4np7dw24', ['bet_amount_per_threshold']], ['bafybeihcly4lt4pqiqhj3kndxitts5yzby77525v4elii23k6skcnbedu4', ['kelly_criterion_no_conf']]], 'strategies_kwargs': [['bet_kelly_fraction', 1.5], ['floor_balance', 500000000000000000], ['bet_amount_per_threshold', {'0.1': 0, '0.0': 0, '0.3': 0, '0.2': 0, '0.5': 0, '0.4': 0, '0.7': 80000000000000000, '0.6': 60000000000000000, '0.9': 1000000000000000000, '0.8': 160000000000000000, '1.0': 1000000000000000000}]]} passed to params have not been set!

[2024-06-18 13:45:46.186000][INFO] HTTP Server has connected to port: 8000.

[2024-06-18 13:45:46.508000][INFO] Connecting to libp2p node acn.staging.autonolas.tech:9005. Attempt 1

[2024-06-18 13:45:49.916000][INFO] Successfully connected to libp2p node acn.staging.autonolas.tech:9005

[2024-06-18 13:45:49.920000][INFO] Entered in the 'registration_startup_round' round for period 0

[2024-06-18 13:45:49.922000][INFO] Start processing messages...

[2024-06-18 13:45:49.923000][INFO] Entered in the 'registration_startup_behaviour' behaviour

[2024-06-18 13:45:49.924000][INFO] Synchronizing with Tendermint...

[2024-06-18 13:45:52.953000][INFO] local height == remote == 2; Synchronization complete.

[2024-06-18 13:45:52.954000][INFO] Sharing Tendermint config on start-up?: False

[2024-06-18 13:45:53.180000][WARNING] Slashing has not been enabled!

[2024-06-18 13:45:53.181000][INFO] 'registration_startup_round' round is done with event: Event.DONE

[2024-06-18 13:45:53.181000][INFO] Entered in the 'check_benchmarking_mode_round' round for period 0

[2024-06-18 13:45:53.184000][INFO] Entered in the 'check_benchmarking_mode_behaviour' behaviour

[2024-06-18 13:45:54.197000][INFO] 'check_benchmarking_mode_round' round is done with event: Event.BENCHMARKING_DISABLED

[2024-06-18 13:45:54.198000][INFO] Entered in the 'update_bets_round' round for period 0

[2024-06-18 13:45:54.201000][INFO] Entered in the 'update_bets_behaviour' behaviour

[2024-06-18 13:45:54.201000][WARNING] No stored bets file was detected in /root/agent/bets.json. Assuming bets are empty.

[2024-06-18 13:45:54.272000][ERROR] Could not parse the response body!

[2024-06-18 13:45:54.273000][ERROR] Response: ""

[2024-06-18 13:45:54.273000][ERROR] Could not get questions from omen

[2024-06-18 13:45:56.361000][ERROR] Could not parse the response body!

[2024-06-18 13:45:56.362000][ERROR] Response: ""

[2024-06-18 13:45:56.362000][ERROR] Could not get questions from omen

[2024-06-18 13:46:00.445000][ERROR] Could not parse the response body!

[2024-06-18 13:46:00.445000][ERROR] Response: ""

[2024-06-18 13:46:00.446000][ERROR] Could not get questions from omen

[2024-06-18 13:46:08.518000][ERROR] Could not parse the response body!

[2024-06-18 13:46:08.519000][ERROR] Response: ""

[2024-06-18 13:46:08.519000][ERROR] Could not get questions from omen

[2024-06-18 13:46:24.613000][ERROR] Could not parse the response body!

[2024-06-18 13:46:24.613000][ERROR] Response: ""

[2024-06-18 13:46:24.614000][ERROR] Could not get questions from omen

[2024-06-18 13:46:56.718000][ERROR] Could not parse the response body!

[2024-06-18 13:46:56.719000][ERROR] Response: ""

[2024-06-18 13:46:56.719000][ERROR] Could not get questions from omen

--- End ---
[ec2-user@ip-172-31-25-255 trader]$

[mkuzmentsov]

Also, attaching the output of trades trades-7.txt command

[Adamantios]

@mkuzmentsov does this happen when using an API key for the graph?

[mkuzmentsov]

@Adamantios yes, when I upgraded the service using instructions, it asked me to provide one, so I provided the key. I see it's set in .trader_runner/.env

[mkuzmentsov]

@Adamantios you gave me a clue, I created account at https://thegraph.com/ and created an API Key, BUT I didn't notice it used different address, which is NOT agent's address. I recreated the API key by logging in with my agent's address and it's back in business now. Thank you!

BTW, any readings on tips how to improve performance of trader service?

[Adamantios]

@mkuzmentsov I'm glad that it works now. The address that you are using to obtain the API key should not matter. I suspect that it was an issue with the subgraph's indexing. We are looking into solutions because currently the indexers of the graph network are not incentivized to index the omen subgraph and as a result, it is not always operational.

If you ever need to check the status of the subgraph quickly, you can navigate here.

BTW, any readings on tips how to improve performance of trader service?

The performance of the trader service depends mostly on the utilized tool (it selects the answer of the bet) and the strategy (it selects the amount of the bet). As a developer, one can improve the performance by implementing betters tools and strategies. As an operator, one can improve the trader by running it, because it learns how to select the best tools by assigning weights to them, based on the outcomes of the past bets.