search

dataman-git / codes_for_articles

167 stars 167 forks source link

Question, trying to retrofit your code to work with adj close, and volume to predict price #1

Open thistleknot opened 3 years ago

thistleknot commented 3 years ago

I read your article on medium

https://medium.com/swlh/a-technical-guide-on-rnn-lstm-gru-for-stock-price-prediction-bce2f7f30346

Modified part of the code to accept volume


def ts_train_test(all_data,time_steps,for_periods):
    '''
    input: 
      data: dataframe with dates and price data
    output:
      X_train, y_train: data from 2013/1/1-2018/12/31
      X_test:  data from 2019 -
      sc:      insantiated MinMaxScaler object fit to the training data
    '''
    # create training and test set
    #all_data.iloc[:,[0, -1]].values
    ts_train = all_data[:'2018'].iloc[:,[0, -1]].values
    ts_test  = all_data['2019':].iloc[:,[0, -1]].values
    ts_train_len = len(ts_train)
    ts_test_len = len(ts_test)

    # create training data of s samples and t time steps
    X_train = []
    y_train = []
    y_train_stacked = []
    for i in range(time_steps,ts_train_len-1): 
        X_train.append(ts_train[i-time_steps:i,0:])
        y_train.append(ts_train[i:i+for_periods,0:])
    X_train, y_train = np.array(X_train), np.array(y_train)

    # Reshaping X_train for efficient modelling
    X_train = np.reshape(X_train, (X_train.shape[0],X_train.shape[1],2))

    inputs = pd.concat((all_data[["Adj Close","Volume"]][:'2018'], all_data[["Adj Close","Volume"]]['2019':]),axis=0).values
    inputs = inputs[len(inputs)-len(ts_test) - time_steps:]

    inputs = inputs.reshape(-1,2)
    #inputs

    # Preparing X_test
    X_test = []
    for i in range(time_steps,ts_test_len+time_steps-for_periods):
        X_test.append(inputs[i-time_steps:i,0:])

    X_test = np.array(X_test)
    X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],2))

    return X_train, y_train , X_test

X_train, y_train, X_test = ts_train_test(all_data,5,2)
X_train.shape[0],X_train.shape[1]

but when I get to this part, I'm confused how to modify the hidden layers

def simple_rnn_model(X_train, y_train, X_test):
    '''
    create single layer rnn model trained on X_train and y_train
    and make predictions on the X_test data
    '''
    # create a model
    from keras.models import Sequential
    from keras.layers import Dense, SimpleRNN

    my_rnn_model = Sequential()
    my_rnn_model.add(SimpleRNN(64, return_sequences=True))
    #my_rnn_model.add(SimpleRNN(32, return_sequences=True))
    #my_rnn_model.add(SimpleRNN(32, return_sequences=True))
    my_rnn_model.add(SimpleRNN(64))
    my_rnn_model.add(Dense(2)) # The time step of the output

    my_rnn_model.compile(optimizer='rmsprop', loss='mean_squared_error')

    # fit the RNN model
    my_rnn_model.fit(X_train, y_train, epochs=100, batch_size=150, verbose=0)

    # Finalizing predictions
    rnn_predictions = my_rnn_model.predict(X_test)

    return my_rnn_model, rnn_predictions

my_rnn_model, rnn_predictions = simple_rnn_model(X_train, y_train, X_test)
rnn_predictions[1:10]
Blessvskp commented 3 years ago

The original code of stock.py is reporting syntax error! I have raised a pull request. Since the author did not respond to you, I am not sure whether he will attend to my pull request.

Can you kindly help me ? Stock py_Capture

thistleknot commented 3 years ago

You downloaded the web page that hosts the code vs the code. I recommend copying and pasting the code vs using curl or wget.

I won't he helping w similar issues in the future. This is a rookie mistake that is easily identified by simply looking at the file your trying to run.

On Thu, Aug 26, 2021, 5:54 AM Blessvskp @.***> wrote:

The original code of stock.py is reporting syntax error! I have raised a pull request. Since the author did not respond to you, I am not sure whether he will attend to my pull request.

Can you kindly help me ? [image: Stock py_Capture] https://user-images.githubusercontent.com/87814655/130950565-659981a6-9061-44de-9474-cf6ee44f709f.JPG

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dataman-git/codes_for_articles/issues/1#issuecomment-906300817, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOXA33YLUY5YCVTREBLT6YMOPANCNFSM43DF3TPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

Blessvskp commented 3 years ago

Hello,

Thanks for your kind reply.. Kindly be sympathetic to this old man of 64 years trying to pick up new things tirelessly! Hence please cooperate.

I tried several stock market packages on Github, wasting my precious time. My observation is that on Github, 90% of packages posted are untested and contain omissions or mistakes. Hence they are unusable!

What use is a package when it is unusable?

Developers of packages, in order to make the packages usable, should give in detail, the requirements and installation method of packages posted by them on Github. This saves a lot of effort for users.

Even prior to receiving your kind reply, I copied and pasted your code and ran it. Your code, so far, has an omission. "Import ta-lib" statement is missing and as a result I came across "ta module missing" error message. I included the said statement. Then new problems cropped up. If I use "ta-lib" to run the modified package by adding the missing statement, stock.py syntax error message is appearing. "-" in ta-lib is not accepted. I renamed it to talib. "talib Not found" error message is coming. I am stuck up here. I am unable to import ta-lib and I don't know how to manage it.

I have installed the ta-lib package in the anaconda environment that is not accessible from the cmd of windows. There is only one source for whl packages on the entire web! I found 17 ta-lib packages on this single source web page and I am not sure which of these 17 packages meets my requirements. I tried to install 3 ta-lib packages from cmd. I got "NOT A SUPPORTED WHEEL ON THIS PLATFORM" Documentation for these whl packages is not available on the web.

All this is leading to colossal waste of time for the end user of the package! Package developers can help to avoid this wattage of human hours by giving instructions as stated above.

I earnestly request you to help me and others who may want to use the valuable package developed by you, by ensuring that the package is usable without making any changes to the package at the end user side.

I hope that this will bring your kind attention to the ordeal faced by end users. I request you to do the needful by modifying and help me to use your package by correcting and testing the package/code yourself first and giving usage and installation instructions.

I once again thank you for your valuable precious time.

Regards.

Dr. Sai

On Thu, 26 Aug 2021, 4:35 pm Turning out data tricks since 2006!, < @.***> wrote:

You downloaded the web page that hosts the code vs the code. I recommend copying and pasting the code vs using curl or wget.

I won't he helping w similar issues in the future. This is a rookie mistake that is easily identified by simply looking at the file your trying to run.

On Thu, Aug 26, 2021, 5:54 AM Blessvskp @.***> wrote:

The original code of stock.py is reporting syntax error! I have raised a pull request. Since the author did not respond to you, I am not sure whether he will attend to my pull request.

Can you kindly help me ? [image: Stock py_Capture] < https://user-images.githubusercontent.com/87814655/130950565-659981a6-9061-44de-9474-cf6ee44f709f.JPG

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/dataman-git/codes_for_articles/issues/1#issuecomment-906300817 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABHKKOXA33YLUY5YCVTREBLT6YMOPANCNFSM43DF3TPA

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dataman-git/codes_for_articles/issues/1#issuecomment-906307551, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU57D7ZGHU75DXAUINGKUZTT6YNX7ANCNFSM43DF3TPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

dataman-git commented 3 years ago

Dear Dr. Sai: Python dependencies can result in discontinuity if there is any update in the imported modules. For this tutorial, I incorporated the class code from ta-lib in stock2.py. It does not need to import "ta-lib" and you can replace stock.py with stock2.py. It still needs the 'yfinance' module and 'streamlit' module. You are advised to 'pip3 install streamlit.

https://github.com/dataman-git/codes_for_articles/blob/master/stock2.py