feedback request (RL IN TRADING )

[ashraf-01]

HI teddy

I would like to know as to who should I replace the following part ii , if I whant say instead want to maximize for other variant quantities say like calmar ratio

A = np.mean(R)
B = np.mean(np.square(R))
S = A / np.sqrt(B - A ** 2)    # under function gradient  

the above is just for sharpe ratio , that means that I have to write down the equivalent quantity to be maximized , then is it just enough to the following

def calc_calmar(rets):

Peaks and bottoms indexes in sequence

mins = np.ravel(argrelmin(rets))
maxs = np.ravel(argrelmax(rets))
extrema = np.concatenate((mins, maxs)) 
extrema.sort()
return -rets.median()/np.diff(rets[extrema]).min()

then the above is just the another definition just like the one you introduced in your article i.e.

def sharpe_ratio(rets): return rets.mean() / rets.std()

but now shall I add the quantity under gradient as well

mins = np.ravel(argrelmin(rets))
maxs = np.ravel(argrelmax(rets))
extrema = np.concatenate((mins, maxs)) 
extrema.sort()
return -rets.mean()/np.diff(rets[extrema]).min()

then finding derivatives w.r.t. the parameters of the calamr ratio , which are argmin and argmax ? is that right your input is highly appreciated best regards

[teddykoker]

Yes, the reward function can be replace with whatever you would like. The caveat is that, like you said, you must take the derivative of this reward function with respect to the trading model parameters. Looking at your code, I am not sure what the derivative would be, so you may want to spend some time figuring that out (you can check numerically), or alternatively use something like PyTorch, which can take the gradient of your expression automatically. Good luck!