Relationship between CONFIG.TRADING.AGE_THRESHOLD and the actual latency

[fvsgit]

Hi @bmino ,

Very new to the crypto world, but as a developer always looking for ways my code can earn some money :) Firstly I would like yo say this is a job well done from a coding prespective and technical design!

I was looking at the output of the bot and noticed that the latency is printed in the console and also in the performance logs. What I would like to know are two things:

1. If for example, the latency is at 200ms, what is the correlation with the CONFIG.TRADING.AGE_THRESHOLD parameter? If i set the parameter to 100ms, does this mean that I would technically never be able to execute the trade?
2. On average, how long is an opportunity available for the trade to be successful? What is the max time the bot should take to execute the trades in order to realise a profit?

Regards, Francois

[fvsgit]

@bmino I just made some changes to the ArbitrageExecution.js file to just get more insight about the AGE_THRESHOLD value. Here is an extract of the code in the function isSafeToExecute:

logger.execution.trace(`Found potential opportunity with profit: ${calculated.percent}`);

// Age Threshold is Not Satisfied
const ageInMilliseconds = now - Math.min(calculated.depth.ab.eventTime, calculated.depth.bc.eventTime, calculated.depth.ca.eventTime);
if (isNaN(ageInMilliseconds) || ageInMilliseconds > CONFIG.TRADING.AGE_THRESHOLD){
   logger.execution.trace(`Age criteria not met: ${ageInMilliseconds}`);
   return false;
}

Note that I just added 2 lines to log some information to the execution log file before the age calculation is done and when the age threshold is not met.

Now, when there were profit opportunities with more than 0% I got my expected entries in the log file, all of them failed the age threshold check. At that point the ageInMilliseconds was on avarage: 53000. If I understand this correctly the last time we had the latest information on any of the ab bc or ac pairs was 53 seconds ago.

Here are my questions:

1. Am I correct in saying the value of 53000 is way too much and should be around 100ms?
2. How can i bring down the ageInMilliseconds value?

In a way I am expecting that running this in my local development environment would result in these kinds of slow times. But when I executed this in azure cloud, my latency was only 138ms on avaerage, so there I would expect that some trades can be made, but I assume then I also need to increase the value of CONFIG.TRADING.AGE_THRESHOLD to somthing around 150?

[bmino]

Hey! I'm always glad to see another dev interest in this project!!

To start off, we should clarify what the CONFIG.TRADING.AGE_THRESHOLD is filtering out. This doesn't necessarily filter out latency, but is sort of related to latency. Here's what I mean:

Each of the 3 tickers involved in a position (AB, BC, and CA we can call them) is being updated via websocket with the latest order book information. AB might have received an update 100 ms ago, but BC might have received an update 53000 ms (yup, that is 53 seconds owtch!) ago. The "age" of this position is determined by the oldest updated ticker which would be 53000 in this case and this would be compared against CONFIG.TRADING.AGE_THRESHOLD and likely determine it to be huuugely too old. I set my age threshold to 50 ms or so. Ideally you want this value to be 0 meaning you have the absolute newest information available, but this isn't a feasible reality.

Latency is the measure of how long it takes the /time endpoint to return. This includes time for the REST request to send, be processed on Binance, and return. It's a little different than how we get order book data because those are coming back through already opened websocket connections.

Am I correct in saying the value of 53000 is way too much and should be around 100ms?

Yup, you got it! The smaller the better

How can i bring down the ageInMilliseconds value?

Decreasing your cooldown time should have more scans and therefore catch more tickers quickly after an update, but will use more cpu resources and might bottleneck the processing of incoming/outgoing requests. Better network speeds and being located closer to the binance exchange will also help here. Latency reeeally gets you when executing 3 trades via REST requests. Your data might be 0 ms old, but if you then have to make 3 requests that take 50 ms each...

[fvsgit]

Hi @bmino ,

Thank you for the feedback. I managed to also narrow it down to infrastructure. To get started I simply containerised the app and deployed it as a container in the cloud. Obviously not using the latest greatest hardware spec. The performance log does not look good at this point:

AVG Latency: 250 ms (slow…)
Calculation Speed: 139.05ms (slow… :D home pc is faster)
API Latency: 300 to 500 ms (slow!!!!)

If we come back to your comment above of using an age threshold of 50ms, that means you must be close to the servers and that you have some serious power executing the calculations.

I guess the questions are now rather about recommended configuration for production use and infrastructure:

1. What is a good benchmark for calculation speed in ms?
2. What is a realistic latency value that I should try to achieve?
3. What is a good value for CALCULATION_COOLDOWN? I see a direct correlation here with the CPU and RAM as we will ramp up the processing if we decrease this value.
4. Where should I locate my server to be as close to the exchange as possible? I would assume binance would have multiple locations that are load balanced, so not sure where would be the best?

Regards, Francois

[bmino]

1. What is a good benchmark for calculation speed in ms?

Calculation speed will vary depending on your investment quantities and depth size. But for the defaults you should see times around the 10 - 20 ms mark.

2. What is a realistic latency value that I should try to achieve?

Latency needs to be loooow. Something less than 50 ms

3. What is a good value for CALCULATION_COOLDOWN? I see a direct correlation here with the CPU and RAM as we will ramp up the processing if we decrease this value.

I have my calculation cooldown at 50 ms, and will only decrease it if my cpu specs are trash or to test how it affects other metrics.

4. Where should I locate my server to be as close to the exchange as possible? I would assume binance would have multiple locations that are load balanced, so not sure where would be the best?

The best results I have seen are in the aws Tokyo region. I typically use a spot instance of a c5n.large for about 3 cents an hour I think

[fvsgit]

Hi @bmino ,

Thanks for all the support and feedback. Also realised this when moving to Tokyo. This seems to make the difference on the latency. Also changing my machine to a compute optimised instance made a huge difference.

Now only have one issue left when executing a trade getting the error Invalid API-Key, IP or permissions for action, but not for the original issue here. Closing it now, thanks for the support.

[fvsgit]

Just FYI, I was so stupid, I did not add my public ip in the whitelist when configuring the API Rrstrictions in Binance. Running this command on my linux machine gave me my public ip: wget -qO- http://ipecho.net/plain | xargs echo