Testing contentious parameters in NullBlockChain: market.liquidity.probabilityOfTrading.tau.scaling

[Jiajia-Cui]

Task Overview

As a researcher, I would like to find out the impact from contentious parameters (market.liquidity.probabilityOfTrading.tau.scaling), so that I can set a reasonable default value for them.

Spec for this parameter is here

Input and output Metrics

• run 20 trials for each test , current value: 10; Candidate value: change from 1 to 100
• Output: required LP margin over 20 trials
• Output: LP bond account over 20 trials
• Output: average of general PnL over time over 20 trials
• Output: average of ABS(positions) over time over 20 trials

Environment for LPs:

_Env_LP001: Optimal strategy with Single LP (uninformed trading bots)

• Post "controlled" LOs close to mid
• Randomly post LOs between the "controlled" LOs and pegged order in a certain rate (mimic real market situation)
• Trade off posting MOs under the possibility from Poisson distribution
• A trial is a time series with 180 time steps, and each time step can be one epoch, trading bots can have n trades generated per epoch with poisson distribution modelling arrival rate. The current setting is 20 trades per epoch, so there are 20*180 trades each trial
• Mid-price is generated by random walk model, you can find it here
• Perhaps the volatility/spread/arrival frequency metrics should be decided randomly at the start as well as just the process generation itself

_Env_LP002: Optimal strategy with Multiple LP (uninformed trading bots)

• If there are multiple LPs with different risk aversion parameters, then the optimal strategy will be different from if there are multiple LPs with the same risk aversion parameter, I would try to run multi LP with the same risk aversion first, and then change the risk aversion parameters on some of the LPs.
• if they are multiple LPs with the same risk aversion parameters, then their "behaviour" will be different. For example, if we have 2 LPs (LP1 and LP2), and they have the same risk aversion. If the first trade happens to be with LP1 and trading bot on the market, then LP1's position will change, hence the optimal strategy for LP1 will change, etc. Therefore, LP1 and LP2 will have different "behaviour".

_Env_LP003: Optimal strategy with Single LP (proportion of informed traders)

• Average trading volume unchanged, however, informed trader accounts for a certain percentage and decide whether or not to send MOs in current time step.
• Informed trader will know midprice at next time step and only long(short) orders with price lower(higher) than midprice at next time step.
• Informed trader always trades before trading bot.
• Informed trader will close out all positions at next time step
• Currently, informed trader ignores trading fees.

_Env_LP004: Optimal strategy with Multiple LP (proportion of informed traders)

[davidsiska-vega]

"Output: required LP margin over 20 trials" is this 20 time series? Or some other statistic based on this.

[davidsiska-vega]

"Optimal strategy with Multiple LP (uninformed trading bots)" should they have exactly the same objective or different parameters e.g. aversion to holding inventory?

[davidsiska-vega]

"might need reinforcements leaning at this point, to be discussed" We need to do all this analysis assuming we've got no RL working.

The RL stuff is another, parallel stream (and it may well not go anywhere for the next few months while we work on the null-chain speed etc).

[davidsiska-vega]

@Jiajia-Cui I still don't see how, based on all the outputs, we'll decide what we think is "reasonable". What do you propose we target? Reasonable average return? VaR of return? (if so I am not sure 20 trials are enough). Average across all the "environments" or across one of them?

[Jiajia-Cui]

"Output: required LP margin over 20 trials" is this 20 time series? Or some other statistic based on this.

They are 20 time series

[TomMcL]

Thoughts distilled from my longer list on slack:

• What's 'a trial'?
  • How long is it?
  • How much trading is there?
  • How is the price process generated? (volatility etc)
  • Perhaps the volatility/spread/arrival frequency metrics should be decided randomly at the start as well as just the process generation itself?
• Should we compact the environment spec into being defined by proportion of informed traders (0, 0.25, 0.5, 0.75, 1 etc)?

[Jiajia-Cui]

"Optimal strategy with Multiple LP (uninformed trading bots)" should they have exactly the same objective or different parameters e.g. aversion to holding inventory?

If there are multiple LPs with different risk aversion parameters, then the optimal strategy will be different from if there are multiple LPs with the same risk aversion parameter, I would try to run multi LP with the same risk aversion first, and then change the risk aversion parameters on some of the LPs.

[TomMcL]

Stupid question but if the LPs have the same risk aversion and are running under the same decision algorithm, how do they differ from one another?

[Jiajia-Cui]

Thoughts distilled from my longer list on slack:

• What's 'a trial'?

  • How long is it?
  • How much trading is there?
  • How is the price process generated? (volatility etc)
  • Perhaps the volatility/spread/arrival frequency metrics should be decided randomly at the start as well as just the process generation itself?
• Should we compact the environment spec into being defined by proportion of informed traders (0, 0.25, 0.5, 0.75, 1 etc)?

• a trial is a time series with 180 time steps, and each time step can be one epoch, trading bots can have n trades generated per epoch with poisson distribution modelling arrival rate. The current setting is 20 trades per epoch, so there are 20*180 trades each trial
• mid-price is generated by random walk model, you can find it here
• Perhaps the volatility/spread/arrival frequency metrics should be decided randomly at the start as well as just the process generation itself---good point

[Jiajia-Cui]

Stupid question but if the LPs have the same risk aversion and are running under the same decision algorithm, how do they differ from one another?

For example, if we have 2 LPs (LP1 and LP2), and they have the same risk aversion. If the first trade happens to be with LP1 and trading bot on the market, then LP1's position will change, hence the optimal strategy for LP1 will change, etc. Therefore, LP1 and LP2 will have different "behaviour".

[Jiajia-Cui]

@Jiajia-Cui I still don't see how, based on all the outputs, we'll decide what we think is "reasonable". What do you propose we target? Reasonable average return? VaR of return? (if so I am not sure 20 trials are enough). Average across all the "environments" or across one of them?

If we are looking at the average return, then 20 trails is enough, if we are look at VaR, then we might need 1000 at least to get a reasonable number. The current speed is about 1.5 second per trial from my laptop, Tom has merged a faster version (with reduced wallet content) so it might be faster

[TomMcL]

Naively, 180 time steps feels like that could be quite short for observing anything that isn't immediately obvious, but perhaps most changes produce differences noticeable on that timescale? (or are not too sensitive on volatility so as you say we can just assume a timestep is an epoch?)

[davidsiska-vega]

This is now in https://github.com/vegaprotocol/vega-market-sim/