Closed Levos-IG1075 closed 4 years ago
I'm not sure what youd expect ... there's no "obvious" lookahead / look into the future - but having few buys in backtesting, and many buys in dry-run is really the opposite signal of this - as lookahead biased strategies often have 0 buys in dry-run.
I think the main key is how you did get the weights / train the model. I don't think you can use Hyperopt for this (without some magic in your methods) - as hyperopt basically just runs backtesting over and over - always over the complete dataset. The methods (populate_buy / populate_sell / populate_indicators) are called for the whole dataset - so the strategy needs to make sure to not look into the future.
Depending on how you did the training, then the model will just look at the whole data and pick favorable points (however knowing how the future evolved).
What you'll probably need to do to properly train a model (depends on how you implement it ...)
iterate in 1unit steps through the whole data with a data-size of a few 100 (i use 300 below - assume 500 as maximum as freqtrade does not really give you more data in dry / live modes), so for example:
first step: data from index 0:300 2nd step: data from index 1:301 3rd step: data from index 2:302 ...
Neither hyperopt nor Backtesting will do this (as it would take a VERY long time).
Usually (with sklearn models) - you'd flatten the data - so all data for the first step is flattened into one very wide row, so you'd have close, close_1, close_2
... - allowing the model to look at the last 300 rows.
Obviously you'll then have 300 fewer rows in the output.
Now flattening the rows may or may not be necessary, depending on the model ... especially if you implement the model yourself from scratch, this can probably be accounted for.
It's just a guess as to why your model underperforms - but i hope it gives you some ideas.
Hi @xmatthias, thanks for the reply!
I'm not sure what youd expect ...
Over the same time period in backtest and dry-run, I'd expect the same buy/sells to occur. Maybe I misunderstood something?
I don't think you can use Hyperopt for this
It's basically what I did: I have a script that take a network shape and generate an Hyperopt class file from it. It's very ugly and, as you expected, very slow. It can barely train a neural network with up to 10 neurons realistically.
In the above code, it's a single-neuron network. The weights are coming directly from Hyperopt :)
I'm not sure what youd expect ...
Over the same time period in backtest and dry-run, I'd expect the same buy/sells to occur. Maybe I misunderstood something?
That was clear - i mean from this issue.
I don't think it's something obvious in the code you shared (doesn't mean it can't be some subtle bug) - which means it's probably something that's wrong with how the training of the network is done (or the bot opens the 30 positions in backtesting, and then keeps them open for a very long time - which would result in very few trades).
I hope you understand that we'll not spend hours trying to find what's wrong with your code.
Yes don't worry I know you won't help me regarding the Strategy code, it's fine.
What I found strange is that I got so different results. A backtest from September 1 didn't open any trades, whereas a dry run from the same time bought at least 30 times.
Maybe there is a bug regarding how the new candles are handled? (I could see the strategy having issues with partial candles, especially with 2h candles)
Sorry if there's any confusion, English is not my native language and I don't want to bother you.
-------- Message d'origine -------- Le 18 sept. 2020 à 06:07, Matthias a écrit :
I'm not sure what youd expect ...
Over the same time period in backtest and dry-run, I'd expect the same buy/sells to occur. Maybe I misunderstood something?
That was clear - i mean from this issue.
I don't think it's something obvious in the code you shared (doesn't mean it can't be some subtle bug) - which means it's probably something that's wrong with how the training of the network is done (or the bot opens the 30 positions in backtesting, and then keeps them open for a very long time - which would result in very few trades).
I hope you understand that we'll not spend hours trying to find what's wrong with your code.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
The key during dry/live operations is the last row of the dataframe. If the buy-signal is active there - then the bot will buy. All other rows are only used to generate this signal, but are irrelevant for the bot in this operation mode.
In backtesting, all rows of the datafarame are considered.
you may want to try to analyze the strategy interactively in a jupyter notebook ...
Best test how the strategy behaves if you only give it 499 candles (and look at the last few row). You can then shift the data and see how it would generate signals in other cases. You can also try to give it a longer period (the timerange you used for backtesting) - maybe you can figure out the difference wia that mode.
Thank you, I'll try to figure it out in the next days.
If anything comes out, I'll post my solution here.
Thank you, I'll try to figure it out in the next days.
If anything comes out, I'll post my solution here.
Hello. Interesting approach, did you manage to get it to work?
Describe your environment
Your question
Hi,
I'm writing an issue because I can't find a solution to my problem. I've searched and couldn't find an answer. My issue is that my custom strategy doesn't buy often in backtest, but buys as soon as possible in dry-run (leading to a lot of ugly losses)
I've checked if my Strategy could see in the future and I don't think it's the case. Could anyone help?
Here's the strategy, it's basically a single neuron neural network trained using hyperopt (nothing fancy, but the code is a bit ugly sorry) (
populate_sell_trend
not included because it's basically the same)Also, here's my config.json:
Any insight would be really appreciated. Thanks.