Open sam31415 opened 6 years ago
@sam31415, thanks for reporting this and opening an issue here as well. Adding a reference to the forum report. Have you been experiencing this phenomena when running not only on the entire universe?
Hi What do you mean by "running not only on the entire universe"? I originally observed this phenomenon with the algorithm I trade, and managed to reproduce it with the algorithm above, although the growth is slower with the latter. The original algorithm does a bit more computations, and obviously some ordering logic. I didn't make more extensive tests.
I meant the number of trading pairs used. Your algorithm above seems to run on all the exchange pairs (=entire universe), so I was wondering whether this behavior was observed using a small number of pairs.
Well, the function get_universe considers all the pairs on the exchange, and returns the ones with sufficiently high volume. Only those are then used in the later computations. I did not try with a small number of pairs, but it would be easy to modify the algorithm above to try.
Anyway I don't see how using a large universe would lead to the observed behavior: the function get_universe is run once per day. Then the algorithm only works with something like 10 pairs.
Just to give an update on this issue, using a large enough instance (for instance a standard one for 1-3 algos) avoids the crashing problem. The CPU and memory use has the same kind of seesaw shape over two time scales, but stays constant over sufficiently large time periods.
Hi,
Sorry for the issue spamming... Check this issue for my configuration.
I have noticed that my algorithms tend to increase their CPU and memory use little by little until they crash the virtual instance they are running on. I decided to make a little test with a very simple algorithm, included below. I just let it run on a google cloud f1-micro virtual instance (the smallest one). The CPU use is plotted below.
You see that it starts with a very modest use, and then steadily increases, with occasional drops. In the end, the algorithm crashed with a request timeout from ccxt, which I believe is unrelated to the present issue.
Did anybody notice a similar behavior? This is an issue because the algorithms I trade display the same behavior and crash once they've eaten up all the memory. The memory use increase with the simple algorithm below seems much flatter, however. My current solution is to restart them once per day, which is fine, but not ideal.
I include the algorithm below. I think it does nothing but very typical tasks. It selects a universe by volume, then fetches some price data, and performs some computations on it. (It doesn't order anything.) I see no reason why such an algorithm would lead to a steady increase in CPU load.
Best,
Samuel
Algorithm: