Example requests for dojo

[wwklsq]

Hi Team dojo,

I am a beginner with dojo package and struggled a little bit with creating an example script using dojo to provide liquidity and inspect the fees earned in a backtesting manner. So far, I have set up the environment and successfully executed run.py both in notebook as well as in script. Here are the problems I faced today:

• Documentation link is broken on tutorial page under code refererence, I believe the link is outdated, this link might be the official docs for dojo.

• Failed to use USDC/ETH 0.05% pool (stated here to be supported) to create trade simulation, got key Error from error message.

  

• The minimal viable strategy I wanted to create(cloned and modified from PassiveLP) is to use pool e.g. USDC/WETH 0.3% to mint a position and we should be able to check the uncollected fees in nft position at the end of simulation. Following code snippet explains my idea also the puzzling parts in comment after inspection of docs.

  class Simple_lp_deploy(BasePolicy):
  """Provide liquidity passively to a pool in the sepcified price bounds."""
  
  def __init__(self, agent: BaseAgent, lower_price: float, upper_price: float) -> None:
      """Initialize the policy.
  
      :param agent: The agent which is using this policy.
      :param lower_price: A custom lower price.
      :param upper_price: A custom upper price.
      """
      super().__init__(agent=agent)
      self.lower_price_bound = Decimal(lower_price)
      self.upper_price_bound = Decimal(upper_price)
      self.has_invested = False
  
  def fit(self):
      pass
  
  def inital_quote(self, obs: UniV3Obs) -> List[UniV3Action]:
      pool_idx = 0
      pool_address = obs.pools[pool_idx]
      token0, token1 = obs.pool_tokens(obs.pools[pool_idx])
      wallet_portfolio = self.agent.erc20_portfolio()
  
      address0, address1 = obs.pool_token_addresses(obs.pools[pool_idx])
      decimals0 = money.get_decimals(self.agent.backend, address0)
      decimals1 = money.get_decimals(self.agent.backend, address1)
  
      lower_price_range = self.lower_price_bound
      upper_price_range = self.upper_price_bound
      tick_spacing = obs.tick_spacing(pool_address)
  
      lower_tick = uniswapV3.price_to_tick(
          lower_price_range, tick_spacing, [decimals0, decimals1]
      )
      upper_tick = uniswapV3.price_to_tick(
          upper_price_range, tick_spacing, [decimals0, decimals1]
      )
      provide_action = UniV3Action(
          agent=self.agent,
          type="quote",
          pool=pool_address,
          quantities=[wallet_portfolio[token0], wallet_portfolio[token1]],
          tick_range=(lower_tick, upper_tick),
      )
      self.has_invested = True
      return [provide_action]
  
  def predict(self, obs: UniV3Obs) -> List[UniV3Action]:
      if not self.has_invested:
          return self.inital_quote(obs)
  
      # https://readthedocs.compasslabs.ai/dojo.observations.html#dojo.observations.UniV3Obs.lp_quantities
      # https://readthedocs.compasslabs.ai/dojo.observations.html#dojo.observations.UniV3Obs.nft_positions
      # I want to print the uncollected fees in each step for our minted position, where can I get the
      # toked_id for our position?
      # print(obs.nft_positions(token_id=))
  
      # alternatively, I can also call https://readthedocs.compasslabs.ai/dojo.observations.html#dojo.observations.UniV3Obs.pool_positions
      # but what is the owner address in this case? Should it be a real owner address? Or can I
      # get the synthetic owner address from self.agent?
      # print(obs.pool_positions(pool=pool_address, owner=))
      return []

• Last but not least, in which unit is rewards defined? I assume, it´s in USDC for run.py case. Do we have a lookup table of definition for other cases? Btw, When I ran the run.py in notebook and plotted the reward results, there´s a big plummet in one block, which looks not close to the result from homepage. Although the time span was different, it also didn´t match any pattern from chart below.

Thanks for the great work in general, I hope I can get supports for questions above and more examples from compass labs side regarding this project.

Best regards, Wwk

[0xprofessooor]

Hey @wwklsq, thanks for giving dojo a try!!

To answer a few of your questions:

• In the next release (hopefully on Monday), the error you got with the pool address won't happen anymore because we've replaced them with the actual uniswap pool names, so you'll be able to directly set pools = ["USDC/WETH-0.05"], in the meantime, the error is because the address is in lower case, try setting pools = ["0x88e6A0c2dDD26FEEb64F039a2c41296FcB3f5640"]
• You can get the nft tokens your agent currently has with the agent.erc721_portfolio() method (https://dojo.compasslabs.ai/tutorial/Agents#-in-built-metrics-tracking), perhaps it would be better if we renamed this to nft_portolio to be clearer 🤔
• Yes, in the case where you want to use pool_positions(), the owner is simply the agent address, which can be obtained by agent.account.address
• In the run.py example, the agent is a UniV3PoolWealthAgent, whose reward function returns the wealth of the agent in units of token0 in the pool (https://github.com/CompassLabs/dojo_examples/blob/main/agents/uniV3_pool_wealth.py#L29). In the case of the USDC/WETH pool, token0 is USDC so it should be in USDC. In other cases, you can define it precisely in whatever way you prefer by defining your agent reward() method to return a scalar of your choice 🧑‍🔬

What timespan did you run in your notebook? We can also have a look on our side :) We'll also have a look into the docs and do an update, some features to look forward to on Monday include a 5x speed increase and a load of bug fixes for better stability!

If you have any other questions as well or have a particular use case dojo doesn't yet fulfill, feel free to ask us here again! We're happy to help whatever way we can 🫡

[wwklsq]

Hey @09tangriro, thanks for the quick reply!

Upon testing new version of dojo-compass, I had following issues encountered & questions raised:

• From pypi libs trace, dojo team has already uploaded version 2.0.0 last friday, as ur message above hints a new update today, is it the latest version or new bug fixes are incoming?

• From my understanding, one of the major updates was to switch backend from hardhat to anvil, which is essentially a handy tool from foundry. Unfortunately, I believe dojo team didn´t add dependencies for the new updates, or new documentation was put in installation page. I ran following scripts to install foundry:

  curl -L https://foundry.paradigm.xyz | bash
  foundryup

  And added following workaround before the script to include anvil into python namespace:

  import os
  os.environ['PATH'] += ':'+'/root/.foundry/bin'

  After setting up the environment for foundry, the MovingAveragePolicy works for config below:

  pool = 'USDC/WETH-0.05'  # WETH/USDC
  start_time = dateparser.parse("2023-04-29 10:00:00 UTC")
  end_time = dateparser.parse("2023-04-29 16:00:00 UTC")
  demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(10_000), "WETH": Decimal(1)})
  demo_policy = MovingAveragePolicy(agent=demo_agent, short_window=200, long_window=1000)
  sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

  

• Nonetheless, PassiveConcentratedLP wasn´t pulled off smoothly. I tried four pools supported, they all refer to the missing token address in backend. Codes to reproduce the error:

  
  pool = 'USDC/WETH-0.3'
  start_time = dateparser.parse("2023-04-29 10:00:00 UTC")
  end_time = dateparser.parse("2023-04-29 16:00:00 UTC")

demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(10_000), "WETH": Decimal(1)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay", )

demo_policy = PassiveConcentratedLP(agent=demo_agent, lower_price_bound=0.95, upper_price_bound=1.05)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

Error message:

---

ValueError Traceback (most recent call last) Cell In[11], line 17 8 env = UniV3Env( 9 date_range=(start_time, end_time), 10 agents=[demo_agent], 11 pools=[pool], 12 market_impact="replay", 13 ) 15 demo_policy = PassiveConcentratedLP(agent=demo_agent, lower_price_bound=0.95, upper_price_bound=1.05) ---> 17 sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

File dojo/runners/backtest_runner.py:66, in dojo.runners.backtest_runner.backtest_run()

File /work/data/project_data/nodejs_proj_sandbox/dojo_examples/policies/passiveLP.py:68, in PassiveConcentratedLP.predict(self, obs) 66 def predict(self, obs: UniV3Obs) -> List[UniV3Action]: 67 if not self.has_invested: ---> 68 return self.inital_quote(obs) 69 return []

File /work/data/project_data/nodejs_proj_sandbox/dojo_examples/policies/passiveLP.py:43, in PassiveConcentratedLP.inital_quote(self, obs) 41 print(address0) 42 print(address1) ---> 43 decimals0 = money.get_decimals(self.agent.backend, address0) 44 decimals1 = money.get_decimals(self.agent.backend, address1) 46 lower_price_range = self.lower_price_bound * spot_price

File dojo/money/erc20.py:13, in dojo.money.erc20.get_decimals()

File dojo/network/base_backend.py:199, in dojo.network.base_backend.BaseBackend.get_contract()

ValueError: Requested contract that has not been registered: 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48

I suspect the bug lies in the new backend didn´t register token addresses as old one did, which however is compiled in source code, we cannot fix on our side. Hopefully, dojo-team can push a quick fix or guide us how to circumvent these issues. Since this time, code was blocked at an early stage, I cannot follow guidance you gave above, will proceed testing if issues are solved. Thx.

Best regards,
Wwk

[0xprofessooor]

Hey @wwklsq , yeah so I did release a new version on Friday, but it includes a bug, so I've fixed it and released a new version just now today (hence the radio silence on Friday 🤐)

You're right, the demo code you've tried to run wouldn't work with v2.0.0, I'm just about to update this repo and it should work (summary of the issue is the fact that dojo no longer requires you to deal with addresses!). I'll merge it thorugh here now, thanks for bearing with us :))

[0xprofessooor]

@wwklsq merged into main now, let me know if you have any issues now!

[wwklsq]

Hey @09tangriro,

Thanks for the quick response, after updating the example for PassiveConcentratedLP, the error message was gone. But data inspection for uncollected fees show constant 0, codes to reproduce the case:

• Policies

  class PassiveConcentratedLP(BasePolicy):
  """Provide liquidity passively to a pool in the sepcified price bounds."""
  
  def __init__(
      self, agent: BaseAgent, lower_price_bound: float, upper_price_bound: float
  ) -> None:
      """Initialize the policy.
  
      :param agent: The agent which is using this policy.
      :param lower_price_bound: The lower price bound for the tick range of the LP position to invest in.
          e.g. 0.95 means the lower price bound is 95% of the current spot price.
      :param upper_price_bound: The upper price bound for the tick range of the LP position to invest in.
          e.g. 1.05 means the upper price bound is 105% of the current spot price.
      """
      super().__init__(agent=agent)
      self.lower_price_bound = Decimal(lower_price_bound)
      self.upper_price_bound = Decimal(upper_price_bound)
      self.has_invested = False
      self.agent = agent
  
      # status tracker
      self.position_status_tracker = []
  
  def fit(self):
      pass
  
  def inital_quote(self, obs: UniV3Obs) -> List[UniV3Action]:
      pool_idx = 0
      pool = obs.pools[pool_idx]
      token0, token1 = obs.pool_tokens(pool)
      spot_price = obs.price(token0, token1, pool)
      wallet_portfolio = self.agent.erc20_portfolio()
  
      token0, token1 = obs.pool_tokens(obs.pools[pool_idx])
      decimals0 = money.get_decimals(self.agent.backend, token0)
      decimals1 = money.get_decimals(self.agent.backend, token1)
  
      lower_price_range = self.lower_price_bound * spot_price
      upper_price_range = self.upper_price_bound * spot_price
      tick_spacing = obs.tick_spacing(pool)
  
      lower_tick = uniswapV3.price_to_tick(
          lower_price_range, tick_spacing, [decimals0, decimals1]
      )
      upper_tick = uniswapV3.price_to_tick(
          upper_price_range, tick_spacing, [decimals0, decimals1]
      )
      provide_action = UniV3Action(
          agent=self.agent,
          type="quote",
          pool=pool,
          quantities=[wallet_portfolio[token0], wallet_portfolio[token1]],
          tick_range=(lower_tick, upper_tick),
      )
      self.has_invested = True
      return [provide_action]
  
  def predict(self, obs: UniV3Obs) -> List[UniV3Action]:
      pool_idx = 0
      pool = obs.pools[pool_idx]
      token0, token1 = obs.pool_tokens(pool)
      spot_price = obs.price(token0, token1, pool)
  
      token0, token1 = obs.pool_tokens(obs.pools[pool_idx])
      decimals0 = money.get_decimals(self.agent.backend, token0)
      decimals1 = money.get_decimals(self.agent.backend, token1)
  
      lower_price_range = self.lower_price_bound * spot_price
      upper_price_range = self.upper_price_bound * spot_price
      tick_spacing = obs.tick_spacing(pool)
  
      lower_tick = uniswapV3.price_to_tick(
          lower_price_range, tick_spacing, [decimals0, decimals1]
      )
      upper_tick = uniswapV3.price_to_tick(
          upper_price_range, tick_spacing, [decimals0, decimals1]
      )
      self.position_status_tracker.append(
          {
              "liquidity": obs.liquidity(pool),
              "nft_positions": [
                  obs.nft_positions(nft_id[0])
                  for key, nft_id in self.agent.erc721_portfolio().items()
              ],
              "pool_positions": obs.pool_positions(
                  pool=pool,
                  owner=self.agent.account.address,
                  tick_lower=lower_tick,
                  tick_upper=upper_tick,
              ),
          }
      )
      if not self.has_invested:
          return self.inital_quote(obs)
      return []

• Backtest run

  
  pool = 'USDC/WETH-0.3'
  start_time = dateparser.parse("2023-04-29 10:00:00 UTC")
  end_time = dateparser.parse("2023-04-29 16:00:00 UTC")

demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(10_000), "WETH": Decimal(1)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay", )

demo_policy = PassiveConcentratedLP(agent=demo_agent, lower_price_bound=0.95, upper_price_bound=1.05)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

- Code to visualize uncollected fees in positions:

uncollected_fees_nft_positions = {} uncollected_fees_pool_positions = {} for status, block in zip(demo_policy.position_status_tracker, sim_blocks): uncollected_fees_nft_positions[block] = status["nft_positions"][0]["uncollected_fees"] if len(status["nft_positions"]) > 0 else (Decimal(0), Decimal(0)) uncollected_fees_pool_positions[block] = status["pool_positions"]["uncollected_fees"] if "uncollected_fees" in status["pool_positions"] else (Decimal(0), Decimal(0))

fig, ax = plt.subplots(1,1, figsize=(8,5)) df_uncollected_nft = pd.DataFrame.from_dict(uncollected_fees_nft_positions, orient="index", columns=["token0","token1"]).astype({"token0":float, "token1":float}) df_uncollected_nft.plot(ax=ax) ax.set_ylabel("Uncollected fees") ax.set_xlabel("Block number") ax.set_title("Uncollected fees over time from NFT position")

fig, ax = plt.subplots(1,1, figsize=(8,5)) df_uncollected_nft = pd.DataFrame.from_dict(uncollected_fees_pool_positions, orient="index", columns=["token0","token1"]).astype({"token0":float, "token1":float}) df_uncollected_nft.plot(ax=ax) ax.set_ylabel("Uncollected fees") ax.set_xlabel("Block number") ax.set_title("Uncollected fees over time from pool position")

Here are my outputs:
![image](https://github.com/CompassLabs/dojo_examples/assets/112470796/84c30dda-9d51-47f3-b2d8-e99214cf83b7)
![image](https://github.com/CompassLabs/dojo_examples/assets/112470796/51eda3da-1dec-49b1-990b-c020cadefa3a)

It would make sense when Uncollected fees stagnates after price moves out of range, but it shouldn´t be 0 all time. Do you have any clue what could go wrong?

[0xprofessooor]

Hey @wwklsq , yeah this is actually a quirk of uniswap rather than dojo. The nft_positions method basically calls this, but uniswap doesn't automatically update the fee calculation, you have to trigger it manually. I believe this can be done by burning some negligible amount of liquidity. Let me do some digging in this area though and get back to you with more :)

[0xprofessooor]

Ok, I've ammended the code so calling the nft_positions gives you the uncollected fees which auto-update, I'll need to push through a minor release, which I'll do today, and I'll post back here once v2.0.2 is out :))

The below graph shows the nft fees:

[wwklsq]

Hi @09tangriro, Great, did you update the uncollected fees with this trick "I believe this can be done by burning some negligible amount of liquidity"?

[0xprofessooor]

actually no, that proved to be quite temperamental 😬, I found another fix that works though :)

[0xprofessooor]

Hi again, late in the day but I've uploaded the new version 2.0.2 which has the fixes I mentioned earlier. Now when you call nft_positions(), it should give you the correct up-to-date uncollected fee amount. There is also a new method lp_fees() for when you have multiple NFT positions: https://readthedocs.compasslabs.ai/dojo.observations.html#dojo.observations.UniV3Obs.lp_fees similar to lp_quantities()

[wwklsq]

Hi @09tangriro, thanks for the update. Three functions now return proper value as expected.

Continuing my testing with on-chain example, I found another ContractLogicError logged at below. Since this example was once available on chain, I believe, it should not violate any require statement in the contract, not after it´s minted. :

---------------------------------------------------------------------------
ContractLogicError                        Traceback (most recent call last)
Cell In[82], line 30
     21 env = UniV3Env(
     22     date_range=(start_time, end_time),
     23     agents=[demo_agent],
     24     pools=[pool],
     25     market_impact="replay",
     26 )
     28 demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)
---> 30 sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

File dojo/runners/backtest_runner.py:78, in dojo.runners.backtest_runner.backtest_run()

File dojo/environments/base_environments.py:213, in dojo.environments.base_environments.BaseEnvironment.step()

File dojo/environments/uniswapV3.py:1120, in dojo.environments.uniswapV3.UniV3Env._step()

File dojo/environments/uniswapV3.py:815, in dojo.environments.uniswapV3.UniV3Env.quote()

File dojo/environments/uniswapV3.py:513, in dojo.environments.uniswapV3.UniV3Env.increase_liquidity()

File dojo/network/base_backend.py:146, in dojo.network.base_backend.BaseBackend.contract_transact()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/contract.py:323, in ContractFunction.transact(self, transaction)
    321 def transact(self, transaction: Optional[TxParams] = None) -> HexBytes:
    322     setup_transaction = self._transact(transaction)
--> 323     return transact_with_contract_function(
    324         self.address,
    325         self.w3,
    326         self.function_identifier,
    327         setup_transaction,
    328         self.contract_abi,
    329         self.abi,
    330         *self.args,
    331         **self.kwargs,
    332     )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/utils.py:172, in transact_with_contract_function(address, w3, function_name, transaction, contract_abi, fn_abi, *args, **kwargs)
    157 """
    158 Helper function for interacting with a contract function by sending a
    159 transaction.
    160 """
    161 transact_transaction = prepare_transaction(
    162     address,
    163     w3,
   (...)
    169     fn_kwargs=kwargs,
    170 )
--> 172 txn_hash = w3.eth.send_transaction(transact_transaction)
    173 return txn_hash

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py:363, in Eth.send_transaction(self, transaction)
    362 def send_transaction(self, transaction: TxParams) -> HexBytes:
--> 363     return self._send_transaction(transaction)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/module.py:75, in retrieve_blocking_method_call_fn..caller(*args, **kwargs)
     68     return LogFilter(eth_module=module, filter_id=err.filter_id)
     70 (
     71     result_formatters,
     72     error_formatters,
     73     null_result_formatters,
     74 ) = response_formatters
---> 75 result = w3.manager.request_blocking(
     76     method_str, params, error_formatters, null_result_formatters
     77 )
     78 return apply_result_formatters(result_formatters, result)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:320, in RequestManager.request_blocking(self, method, params, error_formatters, null_result_formatters)
    310 def request_blocking(
    311     self,
    312     method: Union[RPCEndpoint, Callable[..., RPCEndpoint]],
   (...)
    315     null_result_formatters: Optional[Callable[..., Any]] = None,
    316 ) -> Any:
    317     """
    318     Make a synchronous request using the provider
    319     """
--> 320     response = self._make_request(method, params)
    321     return self.formatted_response(
    322         response, params, error_formatters, null_result_formatters
    323     )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:208, in RequestManager._make_request(self, method, params)
    204 request_func = provider.request_func(
    205     cast("Web3", self.w3), cast(MiddlewareOnion, self.middleware_onion)
    206 )
    207 self.logger.debug(f"Making request. Method: {method}")
--> 208 return request_func(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/gas_price_strategy.py:100, in gas_price_strategy_middleware..middleware(method, params)
     96     latest_block = w3.eth.get_block("latest")
     97     transaction = validate_transaction_params(
     98         transaction, latest_block, generated_gas_price
     99     )
--> 100     return make_request(method, (transaction,))
    101 return make_request(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/attrdict.py:43, in attrdict_middleware..middleware(method, params)
     42 def middleware(method: RPCEndpoint, params: Any) -> RPCResponse:
---> 43     response = make_request(method, params)
     45     if "result" in response:
     46         return assoc(
     47             response, "result", AttributeDict.recursive(response["result"])
     48         )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/buffered_gas_estimate.py:40, in buffered_gas_estimate_middleware..middleware(method, params)
     35     transaction = params[0]
     36     if "gas" not in transaction:
     37         transaction = assoc(
     38             transaction,
     39             "gas",
---> 40             hex(get_buffered_gas_estimate(w3, transaction)),
     41         )
     42         return make_request(method, [transaction])
     43 return make_request(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/_utils/transactions.py:154, in get_buffered_gas_estimate(w3, transaction, gas_buffer)
    149 def get_buffered_gas_estimate(
    150     w3: "Web3", transaction: TxParams, gas_buffer: int = 100000
    151 ) -> int:
    152     gas_estimate_transaction = cast(TxParams, dict(**transaction))
--> 154     gas_estimate = w3.eth.estimate_gas(gas_estimate_transaction)
    156     gas_limit = get_block_gas_limit(w3)
    158     if gas_estimate > gas_limit:

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py:293, in Eth.estimate_gas(self, transaction, block_identifier)
    290 def estimate_gas(
    291     self, transaction: TxParams, block_identifier: Optional[BlockIdentifier] = None
    292 ) -> int:
--> 293     return self._estimate_gas(transaction, block_identifier)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/module.py:75, in retrieve_blocking_method_call_fn..caller(*args, **kwargs)
     68     return LogFilter(eth_module=module, filter_id=err.filter_id)
     70 (
     71     result_formatters,
     72     error_formatters,
     73     null_result_formatters,
     74 ) = response_formatters
---> 75 result = w3.manager.request_blocking(
     76     method_str, params, error_formatters, null_result_formatters
     77 )
     78 return apply_result_formatters(result_formatters, result)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:321, in RequestManager.request_blocking(self, method, params, error_formatters, null_result_formatters)
    317 """
    318 Make a synchronous request using the provider
    319 """
    320 response = self._make_request(method, params)
--> 321 return self.formatted_response(
    322     response, params, error_formatters, null_result_formatters
    323 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:282, in RequestManager.formatted_response(response, params, error_formatters, null_result_formatters)
    277     if not isinstance(error.get("message"), str):
    278         _raise_bad_response_format(
    279             response, "error['message'] must be a string"
    280         )
--> 282     apply_error_formatters(error_formatters, response)
    284     raise ValueError(error)
    286 # Format and validate results

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:99, in apply_error_formatters(error_formatters, response)
     94 def apply_error_formatters(
     95     error_formatters: Callable[..., Any],
     96     response: RPCResponse,
     97 ) -> RPCResponse:
     98     if error_formatters:
---> 99         formatted_resp = pipe(response, error_formatters)
    100         return formatted_resp
    101     else:

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/cytoolz/functoolz.pyx:680, in cytoolz.functoolz.pipe()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/cytoolz/functoolz.pyx:655, in cytoolz.functoolz.c_pipe()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/_utils/contract_error_handling.py:161, in raise_contract_logic_error_on_revert(response)
    158 if message_present:
    159     # Geth Revert with error message and code 3 case:
    160     if error.get("code") == 3:
--> 161         raise ContractLogicError(message, data=data)
    162     # Geth Revert without error message case:
    163     elif "execution reverted" in message:

ContractLogicError: execution reverted: STF

Codes to reproduce Error:

• backtest runner

  
  pool = 'USDC/WETH-0.05'

start_time = dateparser.parse("Sep 3, 2023, 10:59 AM").replace(tzinfo=timezone("UTC")) end_time = dateparser.parse("2023-10-02 10:01:24.382667").replace(tzinfo=timezone("UTC"))

deposit_usdc = 12256.55110000 deposit_weth = 8.87624960 lower_price = 0.000333 upper_price = 0.00100974

demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay", )

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

- Simple_LP_Deployer, this class replaces default lower and upper price bound with real number

class Simple_LP_Deploy(BasePolicy): """Provide liquidity passively to a pool in the sepcified price bounds."""

def __init__(self, agent: BaseAgent, lower_price: float, upper_price: float) -> None:
    """Initialize the policy.

    :param agent: The agent which is using this policy.
    :param lower_price: A custom lower price.
    :param upper_price: A custom upper price.
    """
    super().__init__(agent=agent)
    self.lower_price_bound = Decimal(lower_price)
    self.upper_price_bound = Decimal(upper_price)
    self.has_invested = False

    self.agent = agent

    # status tracker
    self.position_status_tracker = []

def fit(self):
    pass

def inital_quote(self, obs: UniV3Obs) -> List[UniV3Action]:
    pool_idx = 0
    pool = obs.pools[pool_idx]
    token0, token1 = obs.pool_tokens(pool)
    wallet_portfolio = self.agent.erc20_portfolio()

    token0, token1 = obs.pool_tokens(obs.pools[pool_idx])
    decimals0 = money.get_decimals(self.agent.backend, token0)
    decimals1 = money.get_decimals(self.agent.backend, token1)

    lower_price_range = self.lower_price_bound
    upper_price_range = self.upper_price_bound
    tick_spacing = obs.tick_spacing(pool)

    lower_tick = uniswapV3.price_to_tick(
        lower_price_range, tick_spacing, [decimals0, decimals1]
    )
    upper_tick = uniswapV3.price_to_tick(
        upper_price_range, tick_spacing, [decimals0, decimals1]
    )
    provide_action = UniV3Action(
        agent=self.agent,
        type="quote",
        pool=pool,
        quantities=[wallet_portfolio[token0], wallet_portfolio[token1]],
        tick_range=(lower_tick, upper_tick),
    )
    self.has_invested = True
    return [provide_action]

def predict(self, obs: UniV3Obs) -> List[UniV3Action]:
    pool_idx = 0
    pool = obs.pools[pool_idx]
    token0, token1 = obs.pool_tokens(pool)

    token0, token1 = obs.pool_tokens(obs.pools[pool_idx])
    decimals0 = money.get_decimals(self.agent.backend, token0)
    decimals1 = money.get_decimals(self.agent.backend, token1)

    lower_price_range = self.lower_price_bound
    upper_price_range = self.upper_price_bound
    tick_spacing = obs.tick_spacing(pool)

    lower_tick = uniswapV3.price_to_tick(
        lower_price_range, tick_spacing, [decimals0, decimals1]
    )
    upper_tick = uniswapV3.price_to_tick(
        upper_price_range, tick_spacing, [decimals0, decimals1]
    )
    self.position_status_tracker.append(
        {
            "liquidity": obs.liquidity(pool),
            "nft_positions": [
                obs.nft_positions(nft_id[0])
                for key, nft_id in self.agent.erc721_portfolio().items()
            ],
            "pool_positions": obs.pool_positions(
                pool=pool,
                owner=self.agent.account.address,
                tick_lower=lower_tick,
                tick_upper=upper_tick,
            ),
            "lp_fees": [
                obs.lp_fees([nft_id[0]])
                for key, nft_id in self.agent.erc721_portfolio().items()
            ],
            "lp_quantities": [
                obs.lp_quantities([nft_id[0]])
                for key, nft_id in self.agent.erc721_portfolio().items()
            ],
        }
    )
    if not self.has_invested:
        return self.inital_quote(obs)
    return []

Apart from that, simulation broke at around 8000 blocks costing 6min, given average Ethereum block generating time to be ~12s, we can roughly calculate that it requires 6min to backtest 1day samples, if we want to simulate for one month, it would take 3h? Time cost is a little bit expensive, is there any approach we can boost the speed?

Best regards,
Wwk

[0xprofessooor]

The STF error code comes from the uniswap smart contract: https://docs.uniswap.org/contracts/v3/reference/error-codes, and indicates a failure when trying to transfer funds. This means the agent trying to increase liquidity in a position does not have enough money to fulfill the request. If you turn on the logger (https://github.com/CompassLabs/dojo_examples/blob/main/run.py#L4) you should be able to see exactly which agent failed and on what transaction. It looks like this is a problem with the MarketAgent though not having enough money to do a mint, I'll recreate here and validate.

In the meantime, one way around this is to use the 'local' backend_type in the environment, which removes communication with an archive node and means we can mint much more liquidity to the MarketAgent at setup time! Unfortunately, we're still sorting through some data in AWS so this mode only supports a start simulation date of < 2023-07-13 right now (but end date can be later). Also, for later simulation dates this mode may take longer as well to initialize the uniswap contract state.

Yeah we're aware of the speed issues and are working to resolve those next, but it might take some time since we'll be finetuning a custom EVM for simulations, and right now we're only 2 engineers :(

Thanks for bearing with us @wwklsq :)

[0xprofessooor]

A more involved way to fix the issue would be to find a whale account with lots of USDC or WETH at the time of simulation with the forked backend and use the agent fund_erc20 method (https://readthedocs.compasslabs.ai/dojo.agents.html#dojo.agents.BaseAgent.fund_erc20) to top up the MarketAgent:

env.agents[0].fund_erc20(token="USDC", source="whale_address", quantity=Decimal(1_000_000_000))

Your script would also need to change:

pool = 'USDC/WETH-0.05'

start_time = dateparser.parse("Sep 3, 2023, 10:59 AM").replace(tzinfo=timezone("UTC"))
end_time = dateparser.parse("2023-10-02 10:01:24.382667").replace(tzinfo=timezone("UTC"))

deposit_usdc = 12256.55110000
deposit_weth = 8.87624960
lower_price = 0.000333
upper_price = 0.00100974

demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)})

env = UniV3Env(
    date_range=(start_time, end_time),
    agents=[demo_agent],
    pools=[pool],
    market_impact="replay",
)

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

###### NEW LINES HERE #######
env.reset()
env.agents[0].fund_erc20(token="USDC", source="whale_address", quantity=Decimal(1_000_000_000))

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

[wwklsq]

Hi @09tangriro,

Unfortunately, two-liner fix above gave me same error, for whale address, I picked 0x5bA33e59528404DACa7319Ca186b0e000a55551E and quantity I reduced to 1_000_000 because otoh it will incur an overflow in balance error.

Detailed error msg here:

---------------------------------------------------------------------------
ContractLogicError                        Traceback (most recent call last)
Cell In[108], line 26
     23 env.reset()
     24 env.agents[0].fund_erc20(token="USDC", source="0x5bA33e59528404DACa7319Ca186b0e000a55551E", quantity=Decimal(1_000_000))
---> 26 sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

File dojo/runners/backtest_runner.py:78, in dojo.runners.backtest_runner.backtest_run()

File dojo/environments/base_environments.py:213, in dojo.environments.base_environments.BaseEnvironment.step()

File dojo/environments/uniswapV3.py:1120, in dojo.environments.uniswapV3.UniV3Env._step()

File dojo/environments/uniswapV3.py:815, in dojo.environments.uniswapV3.UniV3Env.quote()

File dojo/environments/uniswapV3.py:513, in dojo.environments.uniswapV3.UniV3Env.increase_liquidity()

File dojo/network/base_backend.py:146, in dojo.network.base_backend.BaseBackend.contract_transact()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/contract.py:323, in ContractFunction.transact(self, transaction)
    321 def transact(self, transaction: Optional[TxParams] = None) -> HexBytes:
    322     setup_transaction = self._transact(transaction)
--> 323     return transact_with_contract_function(
    324         self.address,
    325         self.w3,
    326         self.function_identifier,
    327         setup_transaction,
    328         self.contract_abi,
    329         self.abi,
    330         *self.args,
    331         **self.kwargs,
    332     )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/utils.py:172, in transact_with_contract_function(address, w3, function_name, transaction, contract_abi, fn_abi, *args, **kwargs)
    157 """
    158 Helper function for interacting with a contract function by sending a
    159 transaction.
    160 """
    161 transact_transaction = prepare_transaction(
    162     address,
    163     w3,
   (...)
    169     fn_kwargs=kwargs,
    170 )
--> 172 txn_hash = w3.eth.send_transaction(transact_transaction)
    173 return txn_hash

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py:363, in Eth.send_transaction(self, transaction)
    362 def send_transaction(self, transaction: TxParams) -> HexBytes:
--> 363     return self._send_transaction(transaction)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/module.py:75, in retrieve_blocking_method_call_fn..caller(*args, **kwargs)
     68     return LogFilter(eth_module=module, filter_id=err.filter_id)
     70 (
     71     result_formatters,
     72     error_formatters,
     73     null_result_formatters,
     74 ) = response_formatters
---> 75 result = w3.manager.request_blocking(
     76     method_str, params, error_formatters, null_result_formatters
     77 )
     78 return apply_result_formatters(result_formatters, result)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:320, in RequestManager.request_blocking(self, method, params, error_formatters, null_result_formatters)
    310 def request_blocking(
    311     self,
    312     method: Union[RPCEndpoint, Callable[..., RPCEndpoint]],
   (...)
    315     null_result_formatters: Optional[Callable[..., Any]] = None,
    316 ) -> Any:
    317     """
    318     Make a synchronous request using the provider
    319     """
--> 320     response = self._make_request(method, params)
    321     return self.formatted_response(
    322         response, params, error_formatters, null_result_formatters
    323     )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:208, in RequestManager._make_request(self, method, params)
    204 request_func = provider.request_func(
    205     cast("Web3", self.w3), cast(MiddlewareOnion, self.middleware_onion)
    206 )
    207 self.logger.debug(f"Making request. Method: {method}")
--> 208 return request_func(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/gas_price_strategy.py:100, in gas_price_strategy_middleware..middleware(method, params)
     96     latest_block = w3.eth.get_block("latest")
     97     transaction = validate_transaction_params(
     98         transaction, latest_block, generated_gas_price
     99     )
--> 100     return make_request(method, (transaction,))
    101 return make_request(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/attrdict.py:43, in attrdict_middleware..middleware(method, params)
     42 def middleware(method: RPCEndpoint, params: Any) -> RPCResponse:
---> 43     response = make_request(method, params)
     45     if "result" in response:
     46         return assoc(
     47             response, "result", AttributeDict.recursive(response["result"])
     48         )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/formatting.py:106, in construct_web3_formatting_middleware..formatter_middleware..middleware(method, params)
    104     formatter = request_formatters[method]
    105     params = formatter(params)
--> 106 response = make_request(method, params)
    108 return _apply_response_formatters(
    109     method,
    110     formatters["result_formatters"],
    111     formatters["error_formatters"],
    112     response,
    113 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/middleware/buffered_gas_estimate.py:40, in buffered_gas_estimate_middleware..middleware(method, params)
     35     transaction = params[0]
     36     if "gas" not in transaction:
     37         transaction = assoc(
     38             transaction,
     39             "gas",
---> 40             hex(get_buffered_gas_estimate(w3, transaction)),
     41         )
     42         return make_request(method, [transaction])
     43 return make_request(method, params)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/_utils/transactions.py:154, in get_buffered_gas_estimate(w3, transaction, gas_buffer)
    149 def get_buffered_gas_estimate(
    150     w3: "Web3", transaction: TxParams, gas_buffer: int = 100000
    151 ) -> int:
    152     gas_estimate_transaction = cast(TxParams, dict(**transaction))
--> 154     gas_estimate = w3.eth.estimate_gas(gas_estimate_transaction)
    156     gas_limit = get_block_gas_limit(w3)
    158     if gas_estimate > gas_limit:

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py:293, in Eth.estimate_gas(self, transaction, block_identifier)
    290 def estimate_gas(
    291     self, transaction: TxParams, block_identifier: Optional[BlockIdentifier] = None
    292 ) -> int:
--> 293     return self._estimate_gas(transaction, block_identifier)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/module.py:75, in retrieve_blocking_method_call_fn..caller(*args, **kwargs)
     68     return LogFilter(eth_module=module, filter_id=err.filter_id)
     70 (
     71     result_formatters,
     72     error_formatters,
     73     null_result_formatters,
     74 ) = response_formatters
---> 75 result = w3.manager.request_blocking(
     76     method_str, params, error_formatters, null_result_formatters
     77 )
     78 return apply_result_formatters(result_formatters, result)

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:321, in RequestManager.request_blocking(self, method, params, error_formatters, null_result_formatters)
    317 """
    318 Make a synchronous request using the provider
    319 """
    320 response = self._make_request(method, params)
--> 321 return self.formatted_response(
    322     response, params, error_formatters, null_result_formatters
    323 )

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:282, in RequestManager.formatted_response(response, params, error_formatters, null_result_formatters)
    277     if not isinstance(error.get("message"), str):
    278         _raise_bad_response_format(
    279             response, "error['message'] must be a string"
    280         )
--> 282     apply_error_formatters(error_formatters, response)
    284     raise ValueError(error)
    286 # Format and validate results

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py:99, in apply_error_formatters(error_formatters, response)
     94 def apply_error_formatters(
     95     error_formatters: Callable[..., Any],
     96     response: RPCResponse,
     97 ) -> RPCResponse:
     98     if error_formatters:
---> 99         formatted_resp = pipe(response, error_formatters)
    100         return formatted_resp
    101     else:

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/cytoolz/functoolz.pyx:680, in cytoolz.functoolz.pipe()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/cytoolz/functoolz.pyx:655, in cytoolz.functoolz.c_pipe()

File /opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/_utils/contract_error_handling.py:161, in raise_contract_logic_error_on_revert(response)
    158 if message_present:
    159     # Geth Revert with error message and code 3 case:
    160     if error.get("code") == 3:
--> 161         raise ContractLogicError(message, data=data)
    162     # Geth Revert without error message case:
    163     elif "execution reverted" in message:

ContractLogicError: execution reverted: STF

Back to reasons, there´s one thing I didn´t understand, the Simple_LP_Deploy only mints a position at initial stage, no action has ever been done in follow-up steps to increase liquidity, is this some default behavior from Dojo backend? The code broke at around 8000 block, which means it must have passed the first block to initiate a mint position, back then it has sufficient amount of tokens to do so.

[0xprofessooor]

Hmm so the Simple_LP_Deploy isn't the only agent that's running, there's also the market impact replay which is controlled by the MarketAgent (which is env.agents[0]). The lack of liquidity shouldn't be on your policy because I noticed it only mints once like you said, so if that failed it would do so right at the beginning. In my opinion, the issue is that the market agent doesn't have enough money to replay the increase liquidity action that happened in history.

One way to check is to see how much money the market agent has, env.agents[0].portfolio() (this info is also auto printed in the logger!) I was just in a meeting but will try to run now :)

[0xprofessooor]

Posting as it goes, this is what the logger says the intial state is:

2023-10-10 10:55:26,877 - ============== SETUP AGENTS ==============
2023-10-10 10:55:26,880 - MarketAgent account: 0x599978c03F8E46dBE294ACEcbdD11386F8F2CA8E
2023-10-10 10:55:29,430 -   Funded 617759178.675603 USDC from 0x40ec5B33f54e0E8A33A975908C5BA1c14e5BbbDf to MarketAgent at address 0x599978c03F8E46dBE294ACEcbdD11386F8F2CA8E
2023-10-10 10:55:30,231 -   Funded 561596.919963096017589199 WETH from 0xF04a5cC80B1E94C69B48f5ee68a08CD2F09A7c3E to MarketAgent at address 0x599978c03F8E46dBE294ACEcbdD11386F8F2CA8E
2023-10-10 10:55:30,402 - MarketAgent portfolio: {'USDC': Decimal('617759178.675603'), 'WETH': Decimal('561596.919963096017589199'), 'ETH': Decimal('500000000')}
2023-10-10 10:55:30,407 - UniV3PoolWealthAgent account: 0xd400C2CE96343B34A1DB50923bdaA04a054F4d99
2023-10-10 10:55:31,210 -   Funded 12256.55110000000058789737522602081298828125 USDC from 0x5414d89a8bF7E99d732BC52f3e6A3Ef461c0C078 to UniV3PoolWealthAgent at address 0xd400C2CE96343B34A1DB50923bdaA04a054F4d99
2023-10-10 10:55:31,679 -   Funded 8.8762495999999995177631717524491250514984130859375 WETH from 0x2F0b23f53734252Bda2277357e97e1517d6B042A to UniV3PoolWealthAgent at address 0xd400C2CE96343B34A1DB50923bdaA04a054F4d99
2023-10-10 10:55:31,832 - UniV3PoolWealthAgent portfolio: {'USDC': Decimal('12256.5511'), 'WETH': Decimal('8.876249599999999517'), 'ETH': Decimal('10')}
2023-10-10 10:55:31,833 - ==========================================

Will wait for it to fail to find exact transaction

[0xprofessooor]

This is what the logger looks like while running by the way:

2023-10-10 10:59:21,225 - ======= BLOCK 18059409: 2 ACTIONS =======
2023-10-10 10:59:21,225 - Trade made by MarketAgent of (-3483.646389 USDC, 2.131530907343109689 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,233 - Trade made by MarketAgent of (-107.488973 USDC, 0.065769166188608854 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,242 - ==========================================

2023-10-10 10:59:21,259 - ======= BLOCK 18059412: 2 ACTIONS =======
2023-10-10 10:59:21,260 - Trade made by MarketAgent of (-1634.33533 USDC, 1 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,268 - Trade made by MarketAgent of (400 USDC, -0.244503292189631137 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,277 - ==========================================

2023-10-10 10:59:21,288 - ======= BLOCK 18059413: 1 ACTIONS =======
2023-10-10 10:59:21,288 - Trade made by MarketAgent of (1099.596864 USDC, -0.672137084164245551 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,297 - ==========================================

2023-10-10 10:59:21,312 - ======= BLOCK 18059415: 1 ACTIONS =======
2023-10-10 10:59:21,313 - Trade made by MarketAgent of (1788.647135 USDC, -1.093322670265304729 WETH) to pool USDC/WETH-0.05
2023-10-10 10:59:21,322 - ==========================================

[0xprofessooor]

It looks like the issue is with this transaction, classic JIT LP:

2023-10-10 11:01:45,394 - ======= BLOCK 18063300: 3 ACTIONS =======
2023-10-10 11:01:45,395 - LP quote made by MarketAgent of (20125725.296726 USDC, 1750.467792713975896955 WETH) to pool USDC/WETH-0.05 in tick range (202350, 202360)

So at that time, the MarketAgent needs to have at least (20125725.296726 USDC, 1750.467792713975896955 WETH) to execute this transaction, unfortunately it's only funded with {'USDC': Decimal('617759178.675603'), 'WETH': Decimal('561596.919963096017589199'), 'ETH': Decimal('500000000')}. One way around is to use the local backend which will fund like billions of everything (although I need to update the data for your start date), or you can manually fund with a forked backend through various whale accounts at that simulation time.

[wwklsq]

Hi @09tangriro, thanks for the quick reply. I believe the root cause is this flash loan, although my run failed in a precedent block:

2023-10-10 10:49:15,868 - ======= BLOCK 18061849: 4 ACTIONS =======
2023-10-10 10:49:15,868 - LP quote made by MarketAgent of (0 USDC, 7121.635311481864224988 WETH) to pool USDC/WETH-0.05 in tick range (202290, 202300)

From my preliminary understanding of dojo, since marketAgent is spawned with a default portfolio, wouldn´t it solve the problem if we set the config value to be higher?

Yes, when switching to backend_type="local", it works fine for start_date before 2023-07, but it caused follow-up error for position above:

Traceback (most recent call last):
  File "/work/data/project_data/nodejs_proj_sandbox/dojo_examples/run.py", line 40, in <module>
    env.reset()
  File "dojo/environments/base_environments.py", line 143, in dojo.environments.base_environments.BaseEnvironment.reset
  File "dojo/environments/uniswapV3.py", line 323, in dojo.environments.uniswapV3.UniV3Env.initialize_state
  File "dojo/dataloaders/base_loader.py", line 198, in dojo.dataloaders.base_loader.BaseLoader.load
  File "dojo/dataloaders/s3_loader.py", line 80, in dojo.dataloaders.s3_loader.S3Loader._load_data
  File "dojo/dataloaders/s3_loader.py", line 82, in dojo.dataloaders.s3_loader.S3Loader._load_data
  File "dojo/dataloaders/s3_loader.py", line 90, in dojo.dataloaders.s3_loader.S3Loader._load_data
  File "/opt/conda/envs/dojo-env/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/envs/dojo-env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "dojo/dataloaders/s3_loader.py", line 69, in dojo.dataloaders.s3_loader.S3Loader._load_s3
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/boto3/resources/factory.py", line 580, in do_action
    response = action(self, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/boto3/resources/action.py", line 88, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/botocore/client.py", line 980, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Since JIT LP can lend milliions of tokens, it would be super hard to find a whale address that can fund such large number and it´s like a drop in the ocean. To solve the problem systematically, I would incline to update the start_date. Or can u point me to one whale address fn?

[0xprofessooor]

Yeah the issue is you can't just mint new tokens using the forked backend, so we have to transfer them from existing sources of liquidity to the MarketAgent, and the whale accounts we've chosen aren't static across time.

Yes your error with the local backend is because it can't find the data files in our AWS stack. This is expected because I need to update the files after 2023-07-12. I'll try to get round to that today. In the long term, we want to recommend only using the local backend type, but to do that we need to fix the transaction throughput speed because it'll take roughly 10 mins right now to initialize the protocol for a 2023 sim.

Last thing to note; that particular pool USDC/WETH-0.05 is the busiest pool in DeFi (which is probably why you want to simulate on it), and so it will be the slowest to run because of the average number of transactions that need to be processed per block. Other pools also put less strain on the MarketAgent, but like I said above the local backend type was designed to fix this issue.

Once again, thanks for bearing with us, I'll update the data now and post back here once it's done :)

[0xprofessooor]

Hey @wwklsq , sorry for the delay, the data should be there now for your simulation dates, it should now work with the local backend. Let me know if it anything goes wrong! :)

[0xprofessooor]

The bug you had has been solved, and we get well past where we were stuck, but another bug has popped up at least in my run, although I think I know the solution. I'll test it now and if it works I'll do another release for you as well :)

Prett sure that's fixed it, will do a release and post back here once it's out for you

[0xprofessooor]

New version out 🎉

So we were doing transactions asynchronously, but that was sometimes leading to erros where a transaction hadn't gone through completely before other code was called, so now we always await them :)

You may notice a slowdown, but actually overall for longer sims it should be faster. The time to process a transaction before would increase linearly as the simulation ran, now it's constant.

I've run your demo quite a ways in now with no issues, hopefully you find the same!

[wwklsq]

Hi @09tangriro ,

I started the script to run overnight, as for 30d on USDC-WETH it supposes to take hours. Just checked out that it failed again with STF error after two hours of running (for 8 days of transactions in total, if we inspect block number), somehow it´s truncated in my console.

2023-10-10 21:58:13,712 - ======= BLOCK 18113939: 5 ACTIONS =======
2023-10-10 21:58:13,713 - LP quote made by MarketAgent of (8523204.332033 USDC, 0 WETH) to pool USDC/WETH-0.05 in tick range (202790, 202800)
2023-10-10 21:58:15,691 - Trade made by MarketAgent of (379974.126628 USDC, -243.571757818053893287 WETH) to pool USDC/WETH-0.05
Traceback (most recent call last):
  File "/work/data/project_data/nodejs_proj_sandbox/dojo_examples/run.py", line 46, in <module>
    sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "dojo/runners/backtest_runner.py", line 78, in dojo.runners.backtest_runner.backtest_run
  File "dojo/environments/base_environments.py", line 213, in dojo.environments.base_environments.BaseEnvironment.step
  File "dojo/environments/uniswapV3.py", line 1101, in dojo.environments.uniswapV3.UniV3Env._step
  File "dojo/environments/uniswapV3.py", line 968, in dojo.environments.uniswapV3.UniV3Env.trade
  File "dojo/network/base_backend.py", line 125, in dojo.network.base_backend.BaseBackend.contract_call
  File "dojo/network/base_backend.py", line 28, in dojo.network.base_backend._contract_call
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/contract.py", line 305, in call
    return call_contract_function(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/contract/utils.py", line 96, in call_contract_function
    return_data = w3.eth.call(
                  ^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py", line 256, in call
    return self._durin_call(transaction, block_identifier, state_override)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/eth/eth.py", line 275, in _durin_call
    return self._call(transaction, block_identifier, state_override)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/module.py", line 75, in caller
    result = w3.manager.request_blocking(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py", line 321, in request_blocking
    return self.formatted_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py", line 282, in formatted_response
    apply_error_formatters(error_formatters, response)
  File "/opt/conda/envs/dojo-env/lib/python3.11/site-packages/web3/manager.py", line 99, in apply_error_formatters
    formatted_resp = pipe(response, error_formatters)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cytoolz/functoolz.pyx", line 680, in cytoolz.functoolz.pipe
  File "cytoolz/functoolz.pyx", line 655, in cytoolz.functoolz.c_pipe

Codes I ran, I skipped whale position fund, because I believe it´s not needed for now?:

import logging
from decimal import Decimal

logging.basicConfig(format="%(asctime)s - %(message)s", level=20)

from agents.uniV3_pool_wealth import UniV3PoolWealthAgent
from dateutil import parser as dateparser
from policies.moving_average import MovingAveragePolicy
from policies.passiveLP import Simple_LP_Deploy

from dojo.environments import UniV3Env
from dojo.runners import backtest_run
from pytz import timezone
import json

pool = "USDC/WETH-0.05"

start_time = dateparser.parse("Sep 3, 2023, 10:59 AM").replace(tzinfo=timezone("UTC"))
end_time = dateparser.parse("2023-10-02 10:01:24.382667").replace(tzinfo=timezone("UTC"))  #

deposit_usdc = 12256.55110000
deposit_weth = 8.87624960
lower_price = 0.000333
upper_price = 0.00100974

demo_agent = UniV3PoolWealthAgent(
    initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)}
)

env = UniV3Env(
    date_range=(start_time, end_time),
    agents=[demo_agent],
    pools=[pool],
    market_impact="replay",
    backend_type="local",
)

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

###### NEW LINES HERE #######
# env.reset()
# env.agents[0].fund_erc20(
#     token="USDC", source="0x5bA33e59528404DACa7319Ca186b0e000a55551E", quantity=Decimal(1_000_000)
# )

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=8051)

What´s the default balance for MarketAgent in this case? Is it the same issue as yesterday that we don´t have enough tokens? Since it takes quite a time to run through the code, I believe by I've run your demo quite a ways in now with no issues, u probably run past previous error block of 18063300?

[0xprofessooor]

Yeah I agree you shouldn't need to fund the agents anymore, the MarketAgent by default should be funded with 1B of every token. I can try increase the limit, although you're also right I ran to block 1807...

I'll increase the limit anyway and do another release.

[wwklsq]

Just one thought, would decimal.MAX_EMAX works? If that didn´t, probably, some logics need to be refactored. Btw, I believe, it might take longer this time to debug the scenario, can you roughly explain to me why MarketAgent would ever run out of money, as I assume it behaves more like a middleman of interpreting market transaction, but token addresses are actually holding tokens?

[0xprofessooor]

Yeah that's exactly what I've just put now, just using the max UINT256 possible.

Of course! So you're right in that the MarketAgent represents the activity of all participants in the market that aren't your agents. In the case of the 'replay' market model, it's goal is to simply replay the transactions that happened in history in your simulation. However, structurally it inherits from the same BaseAgent as your agents do, so it has it's own portfolio of funds. The token addresses actually do not hold any tokens themselves, and transactions can go funny if the sender isn't a wallet address. The MarketAgent can run out of money if there is a transaction in history that exceeds the liquidity it currently holds. Did that make sense?

[0xprofessooor]

Hey @wwklsq , thanks for waiting, I've just released v2.0.4 which should massively increase funding to market agent in setup.

Here's an example from a different demo with the local backend:

2023-10-11 12:43:24,920 - ============== SETUP AGENTS ==============
2023-10-11 12:43:24,922 - MarketAgent account: 0x689c99486BdD7A4400771008F3f36467EFe366a8
2023-10-11 12:43:24,927 -   Funded 28948022309329048855892746252171976963317496166410141009864396001978282.409983 USDC from 0x40ec5B33f54e0E8A33A975908C5BA1c14e5BbbDf to MarketAgent at address 0x689c99486BdD7A4400771008F3f36467EFe366a8
2023-10-11 12:43:24,931 -   Funded 28948022309329048855892746252171976963317496166410141009864.396001978282409983 WETH from 0xF04a5cC80B1E94C69B48f5ee68a08CD2F09A7c3E to MarketAgent at address 0x689c99486BdD7A4400771008F3f36467EFe366a8
2023-10-11 12:43:24,936 - MarketAgent portfolio: {'USDC': Decimal('28948022309329048855892746252171976963317496166410141009864396001978282.409983'), 'WETH': Decimal('28948022309329048855892746252171976963317496166410141009864.396001978282409983'), 'ETH': Decimal('500000000')}

This should definitely, definitely be enough to do almost any simulation I hope!!

[wwklsq]

Hi @09tangriro, thanks for the explanation for MarketAgent, I´m running the script again, will update outcome in this thread.

[0xprofessooor]

By the way, a thing that can help runtime a bit is to deactivate log pushes to the dashboard if you're not using it, you can do this by setting the backtest_runner port parameter to None :)

[wwklsq]

Hi @09tangriro, quick updates, the fix with UNIT256 works as the outcome matches pretty close to what we have expected, more detailed analysis is still ongoing. Besides, we are also interested into having pools other than default USDC-ETH. I believe for that we need to create our own dataloader, wondering what data source does dojo use? What preprocessing steps dojo has applied on the raw data collected? To match the performance with default pool, I deem data quality is crucial.

[0xprofessooor]

Good to hear 🎉

We've built our own datasourcing library inhouse which we call bifrost. This directly queries logs data from on-chain.

PRO: the data we collect is guaranteed to be accurate, in the past we've had accuracy issues getting data from other data vendors!

CON: collecting data in this way is quite slow.

If you'd like, you can let us know which pool(s) you want to simulate on and we can see how quickly we can get support for those pools up and running for you?

To answer your question, implementing your own dataloader would look something like this:

class YourLoader(BaseLoader):
    def __init__(self, env_name: str, date_range: Tuple[datetime, datetime], your_param1):
        super().__init__(
            env_name=env_name, date_range=date_range
        )
        self.your_param1 = your_param1

        def _load_data(self) -> List[Dict[str, Any]]:
        """Load events data.

        This should return a list of dictionaries, where each dictionary represents an event.
        Importantly, each event should obey the formats defined under `dojo.dataloaders.formats`.
        For example, https://readthedocs.compasslabs.ai/dojo.dataloaders.html#dojo.dataloaders.formats.UniV3Burn
        this is the format dojo expects to see for a burn event, which specifies the keys and value types.

        date – date of event (datetime)
        block – block number of event (int)
        log_index – log index of event (int)
        action – action type of event (str)
        pool – pool address (str)
        quantities – token quantities of the burn (list)
        tick_range – tick range of the burn (list)
        liquidity – liquidity of the burn (int)
        owner – owner address of the burn (str)

        So a burn event to be processed might look like:
        [{
            "date": datetime(...),
            "block": 1000,
            "log_index": 1,
            "action": "burn",
            "pool": "0x...", (can be lower or checksum case)
            "quantities": [1, 1],
            "tick_range": [1, 2],
            "liquidity": 1,
            "owner": "0x..." (can be lower or checksum case)
        }]

        To be clear as well, the key checking is lenient, so if you have extra
        keys in your data, you don't need to reformat it, you just need to have
        AT LEAST the keys in the formats.
        """
 All of this info is emitted by uniswap themselves: https://docs.uniswap.org/contracts/v3/reference/core/interfaces/pool/IUniswapV3PoolEvents#burn

 If anythign is unclear let us know! We'd be happy to arrange a call to help you setup as well :)

[wwklsq]

Hi @09tangriro, thx for the example. So I believe bifrost is not an open-source lib, that we could make use of ourselves? Speaking of data vendors, have you guys tried the graphql database maintained by Uniswap Team? Does that also work well for ur case?

[wwklsq]

Hi @09tangriro, three more issues are found:

1. I experienced 4x slower (8min vs. 2min) for the same position I simulated yesterday with "local" backend_type, most time consuming part is to resurrect the liquidity for given Pool. I believe it´s something default behavior from local node, to use mints and burn event to construct liquidity over time? If I use "anvil" as backend, why this step is skipped? What is "local" mode actually? Is it something other than local EVM testing node? The problem is that this process seems to repeat for the same pool with different position initialization. Is there any way we can let them "share" the pool liquidity generated?
2. Current setup cannot handle position with 0 amount token input, which yields STF error.
3. Network connected with anvil opened a port, but it didn´t release it at the end of backtesting.

[0xprofessooor]

1. Yes the local backend is just a local EVM testing node, we're working on a fix where you can specify which port you want to connect to, which will let you work with the same simulation across time.
2. We can certainly ahve a look into that bug, thanks for letting us know!
3. We are aware of that issue too, it might be what's slowing down your sim as well. We're working on a fix for that in the next version too :)

[0xprofessooor]

Unfortunately the speed issues in general are mainly limited by how fast the EVM can process transactions. To fix this would require in the order of a few hundred engineering hours, which we don't have the resources for right now :(

[0xprofessooor]

Hi @09tangriro, thx for the example. So I believe bifrost is not an open-source lib, that we could make use of ourselves? Speaking of data vendors, have you guys tried the graphql database maintained by Uniswap Team? Does that also work well for ur case?

Unfortunately, right now no bifrost is not open source, if there's enough demand for it we may consider it. No, we also haven't used the database maintained by Uniswap, we now only trust the source of truth.

[wwklsq]

Hi @09tangriro,

Thanks for the reply. Upon further testing, we found more edge cases that we want to share with dojo:

1. Another STF error with short holding period ~1d:

   
   start_time = pd.to_datetime("2023-09-24 01:44:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC"))
   end_time = pd.to_datetime("2023-09-25 02:44:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC"))
   deposit_usdc = 5606.648922
   deposit_weth = 5.93962041
   lower_price = 0.00058026
   upper_price = 0.00065752
   demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay",

backend_type = "local"

)

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=None)

2. Error code `NP` which stands for "Burn cannot be called for a position with 0 liquidity". Position to reproduce the error code:

start_time = pd.to_datetime("2023-09-21 15:35:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC")) end_time = pd.to_datetime("2023-09-23 10:33:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC")) deposit_usdc = 7527.222746 deposit_weth = 11.76244792 lower_price = 0.00063174 upper_price = 0.00063237 demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay",

backend_type = "local"

)

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=None)

3. Large discrepancy for position ran by dojo and on-chain ground-truth. For this [position](https://revert.finance/#/uniswap-position/mainnet/565097), on-chain data states that we should have earned 31 USDC and 0.018 WETH when collecting fees. However, running dojo with following codes, yields 12 USDC and 0.00937 WETH, which is far off from real number. Considering the overall liquidity in USDC-WETH pool, this position should have much less market impact to skew the result:

start_time = pd.to_datetime("2023-09-16 18:46:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC")) end_time = pd.to_datetime("2023-09-18 10:31:00+00:00").to_pydatetime().replace(tzinfo=timezone("UTC")) deposit_usdc = 11981.99022 deposit_weth = 2.47269018 lower_price = 0.00060636 upper_price = 0.00062483 demo_agent = UniV3PoolWealthAgent(initial_portfolio={"USDC": Decimal(deposit_usdc), "WETH": Decimal(deposit_weth)})

env = UniV3Env( date_range=(start_time, end_time), agents=[demo_agent], pools=[pool], market_impact="replay",

backend_type = "local"

)

demo_policy = Simple_LP_Deploy(agent=demo_agent, lower_price=lower_price, upper_price=upper_price)

sim_blocks, sim_rewards = backtest_run(env, [demo_policy], port=None)

Note: by calling `collect` at the end of `predict` function in policy doesn´t change the result:

if self.agent.done(): print("done")

burn event

provide_action = UniV3Action(
    agent=self.agent,
    type="collect",
    pool=pool,
    quantities=[self.wallet_portfolio[token0], self.wallet_portfolio[token1]],
    tick_range=(lower_tick, upper_tick),
)
return [provide_action]

Wish to get version update from previous comment and explanation for outliers discovered here. 
Best regards,

[wwklsq]

Hi @09tangriro,

Testing with pools WBTC/WETH-0.3 and DAI/USDC-0.01 failed because token contract is not registered in the backend. Error msg can be seen here: ValueError: Requested contract that has not been registered: USDC and ValueError: Requested contract that has not been registered: WETH. Here I used backend to be default anvil. Reasons for not using local backend is that it took 4x more time than anvil and quickly stuffed /tmp folder with over 16GB for one simulation. Is there anything we can do locally to get contract?

Thanks,

[0xprofessooor]

Hey @wwklsq , thanks for raising these issues! We'll have a look into them today, I'll keep you posted :)

We haven't released the version yet but now the simulation port can be set to auto-close so you don't need to do that manually between runs.

[scheuclu]

Hi @wwklsq

What exact version of dojo are you running? I think some of your issues might be related, since I have run into them myself:

Speed and memory issues First, the local backend had an issue while using anvil. Essentially, we were not awaiting transactions. What happened was that anvil was not able to process them as fast as we were sending them. This caused memory usage to rapidly increase during simulation and also slowed the simulation down significantly. It's been fixed with version 2.0.3 which has been released a week ago. You should notice that the per-block speed is initially slower but remains constant, and overall outperforms the forked backend.

STF issue In general, our medium term goal is to use the forked backend only internally for validation of the local backend via unittests. One reason is that, in the local backend, we can construct the ERC-20 tokens and therefor arbitrarily fund the agents. Otoh, on the forked backend, we have hardcoded actual whale accounts and we transfer funds from them. It just so happens that in the time-period you simulated, that particular whale actually had removed his funds. We have ideas on how to fix this. Either have time-specific-whales, or wrap the tokens so we can mint. In any case, it's the cause of your 'STF' error. Using the local backend should fix this.

Contract not instantiated For now, you should be able to fix this error by making sure the "initial_portfolio" of the agent is consistent with the pools you are simulating. We only deploy the token-contracts that are active in the pool. So if your initial portfolio contains WBTC but none of your pools is a WBTC pool, it will throw an error. It's, a bug that we will fix, of course. It can also be helpful to include all pool-tokens in the initial_portfolio and just set them to equal Decimal(0) if required.

In general, when developing a strategy, I find it most useful atm to do dev-work over short sim-time-periods with the forked backend, since I do not need to wait for the pool initialisation. Then for actual longer simulations, I change it to using the local backend.

Please let me know if anything of the above is unclear!

[wwklsq]

Hi @scheuclu,

Thanks for the detailed feedback. U can find my comments below:

• For dojo version, I´m currently using the latest 2.0.4, which I believe encompasses all the fixes so far?

  

• Yes, during simulation using anvil backend, I believe we have skipped the liquidity initialization process by downloading the liq distribution directly from ur AWS server instead of locally reconstructing them from mint & burn events if use local backend mode? The initialization unfortunately took many space in disks (over 16GB for 1d position), cannot be shared among simulations within same pool and temp files were not deleted after each simulation. All these caused an overflow in disk. For the purpose of rapid testing using short-term positions, I prefer to use anvil thereafter.

• I guess I might have wrong conception of how the STF issue was solved. By levitating the MarketAgent portfolio to maximum was not the solution to this issue. Since dynamic whales in each time period could help solve the STF error in anvil mode, why switching to local mode helps with the problem without whale account? Does that mean local mode is dojo indoor blockchain simulator, just like anvil but with more flexibility & throughput? If that´s the case, why not put liq instantiation process also in cloud and download on demand?

• Contract not instantiated was actually a mistake from my side that I didn´t spawn WealthAgent with proper tokens, adding them back solved the issues.

Overall, thanks for the quick response. Still the positionwith large deviation of ground-truth, STF issues in anvil backend mode, memory & speed issue in local mode concern me.

[scheuclu]

For the space issue. Am I understanding correct that the problem escalated with running many sims?

I think you might be right there since anvil instances were actually not closed after simulation ends. So you'd end up with many instances/instance data accumulating. I believe we recently had a change there, but not released.

[scheuclu]

Forked backend We talk to a service like Infura and all data-prior to fork date is obtained from there. All transactions after fork-date are stored by the local anvil client. To answer most questions, like what is someones balance, anvil needs to ask "What was the balance on fork-date?" to infura first, then add all relevant local transactions it stored itself, to get the final answer. This involves a lot of communitcation over the internet with that provider. You can make it much, much faster if you run a local ethereum node and talk to it instead.

Local backend We create an empty blockchain locally. No data or historic transactions. We then deploy a fresh instance of Uniswap and all the other contracts, such as the pools or tokens. Then, we run a series of transactions to get that pool to the correct state. Since we freshly deploy all contracts and tokens, we can mint as we please. There are no other historic transactions on that local chain. So, if anvil wants to know e.g. what the total balance of some account is, there is no-need to talk to infura to get the history. There is no history. Therefore the round-trip communication time is eliminated. We have internal tests to ensure that this method actually ends up at the same state than the forked backend.

[wwklsq]

Hi @scheuclu,

• Thanks for the intro to both forked and local backend, it´s much clearer now how they work.
• Yes, I think both anvil and local modes bump up memory usage for each instance/simulation I run. The major space-consuming component for local mode is the Liquidity distribution. Currently I did housekeeping manually.
• Local backend is a deployed empty chain with proper contracts spawn, so we spare the time to send requests for anvil backend.
• Speaking for the difference in position above, I´m suspecting that it could be caused by flash loan/JIT lp, which essentially call flash instead of swap in Uni smart contract. Wdyt?

Regards,

[wwklsq]

What pool are you using in the problem cases in your last bullet point? USDC-WETH-500, all positions in this comment are from this pool.

[scheuclu]

I've been running your scripts, and also get an error. But for better collaboration, would it be possible to create a branch on dojo_examples where you put your code into, and then we can directly work on that? Obviously that will be public, so don't put any sophisticated strategies there...

[wwklsq]

I've been running your scripts, and also get an error. But for better collaboration, would it be possible to create a branch on dojo_examples where you put your code into, and then we can directly work on that? Obviously that will be public, so don't put any sophisticated strategies there...

Let me ask team first.

[wwklsq]

Hi @scheuclu ,

We decided to share examples in this branch, I believe the only piece missing here is .env. Let me know if u can reproduce the issues!

Regards,

[0xprofessooor]

Hi @wwklsq, small update (note we haven't done a release just yet, although we plan to do one soon!):

• The STF error is confirmed fixed by using the local backend, although we understand the speed/memory issues you're having and are looking into it.
• The deviation from the source truth has been fixed, there was a subtle bug in the logic to load historical data, which meant some transactions weren't processed (the following screenshots are from the real-time dashboard you can spin up at any time https://dojo.compasslabs.ai/tutorial/Visualization): This has now been resolved:

Still taking a look at the final edge case raised here, but we're making progress :)