search

asavinov / intelligent-trading-bot

Intelligent Trading Bot: Automatically generating signals and trading based on machine learning and feature engineering
https://t.me/intelligent_trading_signals
MIT License
803 stars 179 forks source link

Warnings while working #9

Open StateException opened 1 year ago

StateException commented 1 year ago

Hello, during the process of working with an already trained model(predict_models), the following warnings occurs:

1/1 [==============================] - 0s 13ms/step
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:175: RuntimeWarning: invalid value encountered in double_scalars
  slope = ssxym / ssxm
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:189: RuntimeWarning: invalid value encountered in sqrt
  t = r * np.sqrt(df / ((1.0 - r + TINY)*(1.0 + r + TINY)))
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:192: RuntimeWarning: invalid value encountered in double_scalars
  slope_stderr = np.sqrt((1 - r**2) * ssym / ssxm / df)
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:175: RuntimeWarning: invalid value encountered in double_scalars
  slope = ssxym / ssxm
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:189: RuntimeWarning: invalid value encountered in sqrt
  t = r * np.sqrt(df / ((1.0 - r + TINY)*(1.0 + r + TINY)))
C:\Users\sanya\PycharmProjects\intelligent-trading-bot\venv\lib\site-packages\scipy\stats\_stats_mstats_common.py:192: RuntimeWarning: invalid value encountered in double_scalars
  slope_stderr = np.sqrt((1 - r**2) * ssym / ssxm / df)
1/1 [==============================] - 0s 15ms/step

What can this indicate, can this somehow affect the trained model and the whole prediction process? I see that linregress method is only used when in add_linear_trends method(class feature_generation_rollong_agg.py and in train_signal_models class which I have not touched.

Is it possible to fail at the data downloading stage? I used download_data_binance.py. By the way, such errors did not occur while downloading generating features, merging data or generate labels and while traning too.

I also use the latest version of the repository uploaded to github.

I'm new to python, that's why I'm asking such simple questions. Hope for understanding 😁

Regards!

asavinov commented 1 year ago

Hi, good observation. It seems that the cause is trying to fit linear regression to a series consisting of one point:

from scipy import stats
stats.linregress([0], [1])  # Bad -> warnings
stats.linregress([0,1,2], [1,5,1])  # Good. Slope is 0

It is not possible, so the algorithm complains. This happens because this feature (linear slope) takes a list of window sizes like [1, 5, 20]. Window 1 makes sense for some functions but does not make sense for other functions including linear trend and standard deviation. Their values will be NaN (or some other kind of undefined values) and we have to ignore them by excluding them (as well as other unnecessary features) from the "train_features" parameter.

In general, it is a matter of good parameterization. I did not want to over-parameterize the system and many things are defined in code. So now it is how it works...