search

LuckysonKhaidem / StockMarketPedicition

23 stars 24 forks source link

train_test_split -> shuffle=False #2

Open illUkc opened 4 years ago

illUkc commented 4 years ago

Hello,

biggest problem is you using shuffle data .

So if u try to predict 5,10,30,90 days .. model allready see this data in training.

If u change to shuffle False result are 50-60% max.

Best.

ericleonardo commented 3 years ago

Hello,

biggest problem is you using shuffle data .

So if u try to predict 5,10,30,90 days .. model allready see this data in training.

If u change to shuffle False result are 50-60% max.

Best.

Since this is a Classification and not a Regression task, shuffle should not interfere because what matters is the feature pattern present in each observation alone (patterns: technical indicators relative position and the label target). Also, if not shuffle, the data set may become imbalanced in terms of class number present in train and test. Ex: training data has much more class 1 and testing data has much more class 2. If you not shuffle, may lead to incorrect low accuracy because model learn to detect more class 1. See topic 4 in the Beginners Mistakes: https://towardsdatascience.com/6-amateur-mistakes-ive-made-working-with-train-test-splits-916fabb421bb

illUkc commented 3 years ago

fail_

ericleonardo commented 3 years ago

fail_

Yes... I gave up using Artificial Intelligence to predict financial/stock/forex movements. I discovered that most Academic Papers are simply wrong in their results, even those from respected universities. I tried many method following many papers, some from China (signal decomposition by VMD, CEEMD).. all were wrong because exciting results were due to data leakage when preparing the entire data (train/test) by once, and didn't work when applied to real trading. The only papers we can hope to be true are those presenting real trading results (money made). Not those presenting only predicting/classifying metrics (accuracy, rmse, etc). Sure there something that works, but I gave up my search. Now I'm focusing in learning how to trade by myself, not using AI. Common Mistakes in Stock Price Prediction.pdf

illUkc commented 3 years ago

Now I'm focusing in learning how to trade by myself, not using

Cant agree more .. Yes I find all this mistakes in people research too.