search

keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
61.88k stars 19.45k forks source link

Recurrent nerural network predicting the same answer #13945

Closed Mr-Elysium closed 3 years ago

Mr-Elysium commented 4 years ago

System information

Current Behaviour I am trying to predict if a stock price is going to go up or down the next day. I am using a pandas DataFrame that has 43 columns of which one is the y value, the y values are floats between 0 and 1, also my DataFrame has 5016 rows indexed with numbers. I made a model with some LSTM cells some Dense cells with the loss function being biary_crossentropy, but when I run the model and try to print the predictions all of the predictions are the same:

[[0.56393844]

[0.56393844]

[0.56393844]

...

[0.56393844]

[0.56393844]

[0.56393844]]

The y values are not all the same. Y values:

0

1

0

1

1

The loss and accuracy also start being the same:

Epoch 7/10

4012/4012 [==============================] - 20s 5ms/sample - loss: 0.7052 - acc: 0.5015 - val_loss:

0.6884 - val_acc: 0.5488

Epoch 8/10

4012/4012 [==============================] - 19s 5ms/sample - loss: 0.7054 - acc: 0.4980 - val_loss:

0.6907 - val_acc: 0.5488

Epoch 9/10

4012/4012 [==============================] - 18s 5ms/sample - loss: 0.7078 - acc: 0.4890 - val_loss:

0.6894 - val_acc: 0.5488

Expected Behaviour I want so that my program actually learns from the data and predict it somehwat correctly opposed to predicting the same values over and over again. don't know much about neural networks, so I don't know which layers are the most effective or how many layers I should make. If somebody knows how I can fix this so my neural network actually learns, please let me know. I you know how to make my network more efficient or better also let me know. Thanks in advance.

Code to reproduce the issue 

df.index = pd.to_numeric(df.index, errors = 'coerce')
df = df.drop(['date'], axis = 1)
print(df.shape)
x = df.loc[:, df.columns != 'result']
y = df.loc[:, df.columns == 'result']

y = y.astype(int)

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, shuffle = False)
x_train = x_train.values.reshape(x_train.shape[0], 1, x_train.shape[1])
x_test = x_test.values.reshape(x_test.shape[0], 1, x_test.shape[1])

model = Sequential()

model.add(LSTM(42, input_shape = (1, 42), activation = 'relu', return_sequences = True))
model.add(Dropout(0.2))

model.add(LSTM(42, activation = 'relu'))
model.add(Dropout(0.2))

model.add(Dense(32, activation = 'relu'))
model.add(Dropout(0.2))

model.add(Dense(1, activation = 'sigmoid'))

opt = tf.keras.optimizers.Adam(lr = 0.1, decay=1e-7)

model.compile(loss = 'binary_crossentropy', optimizer = opt, metrics = ['accuracy'])

model.fit(x_train, y_train, epochs = 10, batch_size = 1, validation_data = (x_test, y_test))

prediction = model.predict(x_test)
print(prediction)
print(x_test)
print(y_test)```

Other info / logs

WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where Train on 4012 samples, validate on 1004 samples 2020-04-08 09:41:38.096261: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

mh1337 commented 4 years ago

System information

OS: Ubuntu 18
TensorFlow version: 2.1.0
Keras version: 2.3.1
Python version: 3.6.9

Same issue when I try to predict (model.predict) any trained model. The training seems unaffected.

tilakrayal commented 3 years ago

@Mr-Elysium ,

Can you please take a look at this issue,link with similar error.It helps.Thanks

google-ml-butler[bot] commented 3 years ago

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 3 years ago

Closing as stale. Please reopen if you'd like to work on this further.