search

Ameobea / algobot_elixir

Second iteration of the forex trader bot implemented in elixir
4 stars 2 forks source link

Race conditions when writing to database #3

Open Ameobea opened 8 years ago

Ameobea commented 8 years ago

In the current commit, there are severe race condition issues that seem to be caused by the Iset.append function.

As my VPS reached 100% cpu usage during a backtest, the server can't keep up with the pace of incoming ticks meaning that calculations on old ticks are taking longer than the time to the next received tick. This caused problems like this: https://ameobea.me/u/19o.png

The length of a set was requested in order to append an element to it, but before that element could be added another asynchronous request was made to append an item. Since the previous element had not yet been written, the length remained the same and the result was two elements being written at the same index.

In order to fix this, it may be necessary to implement Redis pipelining or scripting in order to ensure that database resources are not accessed while they are being modified.

Ameobea commented 8 years ago

Just found this: http://redis.io/topics/transactions

Looks like this would solve any issues with race conditions. I'll work on fixing that later today.

dalexj commented 8 years ago

those just like SQL transactions?

Ameobea commented 8 years ago

Seems so, yeah.

dalexj commented 8 years ago

cool cool

dalexj commented 8 years ago

do you use slack?

Ameobea commented 8 years ago

I've been in like 3 different channels in the past; don't actively use it now. I could though.

dalexj commented 8 years ago

https://elixir-slackin.herokuapp.com/