runtime error running example_cpu.py

[mg64ve]

Hi nice work, thanks for sharing. I am running example_cpu.py and getting lots of errors like the following

Process Process-21:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "example_cpu.py", line 72, in worker
    policy_d)
  File "example_cpu.py", line 32, in calculate_reward
    action = model.forward(state, policy) if torch.is_tensor(policy) else model.forward(state)
  File "example_cpu.py", line 144, in forward
    if torch.is_tensor(policy): return torch.matmul(state.double(), policy.double())
RuntimeError: size mismatch, m1: [1 x 518], m2: [515 x 4] at /pytorch/aten/src/TH/generic/THTensorMath.cpp:940

the code still runs but I am not sure what results I can get. Why do we have m1 and m2 with different sizes?

[kayuksel]

Hello,

Thanks!

You can just set No_Channels to 3 as I have reduced the number of assets that is being traded as follows in the latest version:

assets = ['AUDUSD', 'NZDUSD', 'USDUSD']

I will push the fix to the repository as soon as possible when I return from my vacation.

No_Channels was set to len(assets) by default. You can also instead overwrite assets as above before setting NoChannels.

The agent is basically able to learn how to trade all assets but reducing them to those three assets (AUD, NZD, USD) achieves equivalent performance while also reducing the search space of ARS significantly.

[mg64ve]

Thank you very much for your reply. I hope you are having a good vacation and relax from our stressful life :-) I have tried to make the changes you suggested, but I am still having the same issue. I think the problem is here:

    with open(root+'models/weights.pkl', 'rb') as f: weights = cPickle.load(f)
    model = ARSModel(No_Features, No_Channels, weights)
    #_, _, epoch_weights = calculate_reward(model, test_loader, No_Proccess+1, risk=True)
    #plot_function(epoch_weights)

when it loads this weights.pkl they have shape (5,515), so I think it is still the old shape. I hope you can fix once back from your vacation. In the mean time I will try to study your code and get confident with it. Thanks. Cheers.

[kayuksel]

Sorry, I need to check this on my return as I am mobile now. It might be that I havent updated weights.pkl (which is the output of the gpu version.

If you can simply run the gpu version first that may fix the error. Otherwise, you may simply comment out that part to disable weight loading and train from scratch using the cpu version (ARS)

[mg64ve]

Please enjoy your vacation and make an update when you are back. I have Linux Centos with RTX2070 and docker but I can't run example_gpu.py. I am getting the following error:

RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected

a simple pytorch snippet like:

import torch
n_devices = torch.cuda.device_count ()
a = torch.cuda.FloatTensor([1.])

works fine for me. But please take your time, enjoy vacation and let's get in touch when you are back. Thanks. Cheers

[kayuksel]

I have updated weights.pkl ; please try again and let me know in case of problems so that I can check it again on my return (in two weeks). Note: You can pull the repository from scratch.

In terms of gpu version, I believe that is related with your installation. Are you sure that you have the GPU version of torch installed? Please, also check the output of nvidia-smi in your device.

[kayuksel]

You can also rather load from policy_weights.npy which are the post-trained weights of ARS version.

[mg64ve]

Thanks for your help. For what concerns example_cpu.py I am now running it without errors. It is now running 24 processes and each taking 50% CPU thread since 30 minutes and it still not printing anything. I can't believe it is still at epoch=1 !

For what concerns example_gpu.py, sorry I am not very confident with pytorch, I have been more involved in tensorflow and keras. But I have tried simple commands from python and it seems it recognizes GPU. nvidia-smi output is ok, since I am using my machine for other ML related tasks. Since it fails at the following line:

model = nn.Linear(No_Features + No_Channels, No_Channels+1, bias = False).cuda().share_memory()

would it be possible that it does not have enough memory on GPU? My RTX2070 has 8GB and I have now 7.2GB available.

[kayuksel]

It should run No_Process = 8 on a RTX 2070 if you are not running anything else. You may try reducing this to 7 if you like. (It wouldn't also give that runtime error in case that is related with memory). Please make sure that you have the GPU version of PyTorch correctly installed.

CPU version is also quite computationally intensive, you can reduce n_process if you like. This parameter controls how many directions are averaged on each ARS step. The more directions are considered, the more accurate the convergence of the ARS algorithm would be. The original ARS actually picks only the top-k ARS directions but that is even more computationally intensive.

[kayuksel]

Ok, I now found the reason for the error. Please set following variable to '0' in shared.py

os.environ["CUDA_VISIBLE_DEVICES"] = '0'

[mg64ve]

Thanks again for your help. Setting os.environ["CUDA_VISIBLE_DEVICES"] = '0' in shared.py seems to solve the issue. I have also put No_process=4 in example_gpu.py. With value 8 it was trying to allocate memory and not finding enough available. example_cpu.py I will test it later. In the mean time I am reading about pytorch and really I am found of it. It seems different philosophy but much closer to the ML tensor/matrix calculus. May I ask you why you are considering 3 pairs only (AUD,NZD,USD) ? Is this a semplification? Could you please elaborate if you have time? Thanks. Cheers

[kayuksel]

@mg64ve Because it reaches equivalent performance to using all assets and has much less parameter space (computational time). (This is due to the strong correlation between AUD and NZD that the agent is able to exploit and also the fact that those two assets are least likely to get affected by the external news.)

[mg64ve]

@kayuksel Very interesting project. I think the key value of your project is the features extraction. Are you available to share some more details on what features you are using? I guess that 512 over 10+1 assets is not very easy to think about. I believe some regards day of month, day of week, hour of day, right?

[kayuksel]

@mg64ve All features are extracted using my proprietary DL model. I have also tried adding seasonality related features that you have mentioned but they have not shown to be very useful (at least not in FX). I am currently working on a much more advanced version of feature-extraction and also their interpretation.

[kayuksel]

@mg64ve One could also further reduce the dimensionality of search space by selecting a subset of the features using a Gradient Boosted Tree model such as XGBoost or LightGBM as not all of them may be necessary for trading these three assets (AUD, NZD, USD).

[mgazzin]

@kayuksel thanks. Could you please give an overview of what your "DL model" is?

[kayuksel]

@mgazzin Deep Learning Model (Convolutional Neural Network). I will also add features from an LSTM.

[LinuxpowerLudo]

Sound really good ! i have some question @kayuksel.

If i understand you change your size of lots of your pair NZD / AUD each hour ? (the spread and commisions have to eat a lot of profit, right?)

Have you test other pairs ? what's the return?

Why so many 512 features? Have you tried with less? Have you tried to reduce with PCA?

Have you tried pairr trading? It could reduce the time it takes to be in the market.

Thx for your feedback.

[kayuksel]

Hello @LinuxpowerLudo! Yes, the agent re-distributes the portfolio value into AUD, NZD, USD. In fact, over live-trading, I use the trained agent each 5-minute to be more responsive. The agent learns how to avoid transaction cost and it wouldn't spend it unless it is necessary to gain profit or avoid risk (in other words, unless it makes sense). This defined transaction cost includes both the commission and the costs due to spreads (I am also able to input the spread dynamically to the agent over live-trading). It is able to trade all major pairs simultaneously; however, as the risk-adjusted return of that was pretty similar I have preferred to use it only for AUD and NZD for the sake of simplicity of analyzing live-trading performance.

Why 512 features? They are basically the output of Convolutional Neural Network which produces them. I have tried multiple values and that was the most performing one. That would mean that CNN required all of these 512 features to fully describe the market situation for the reinforcement learning agent to perform well on all of those major pairs. They wouldn't probably all necessary for trading AUD and NZD only. Yes, I did try feature-reduction; you can simply run the Variational Autoencoder (or the Semi-Supervised one) in this repository to reduce the feature-space dimensionality to any number. In those mentioned examples, I have simply reduced it to 2D for being able to plot it in order to interpret what those features represent. I have found out that the agent is especially very good at detecting the right direction on large movements of AUD and NZD by this analysis. I am not very knowledgeable with pair trading but I assume that the converged policy is already something similar to that. One can also trade AUD-NZD with those features and I have tried that, but trading AUD-USD, NZD-USD and USD (cash) leads to a better result than that.

[kayuksel]

@LinuxpowerLudo You may want to check my other repository for finding best stock pairs btw: https://github.com/kayuksel/combinatorial-bandit

[LinuxpowerLudo]

Hi @kayuksel thx for your link. I have found this repo about bandit RL https://github.com/wai-i/Pair-Trading-Reinforcement-Learning i like the idea but i don't know if there a bug and it's too complex for adapt it in live or forward test :(

[mg64ve]

Hi @kayuksel , where is stock-rl-challenge ? I don't see it anymore.

[kayuksel]

@mg64ve I apologize but, I have decided to keep the stock-version private at least for the moment.