Temporal Dependencies for LSTM

[lhnguyen102]

Description

This PR introduces the temporal dependencies for LSTM layer where we include the previous cell and states to compute LSTM's output. This feature is added to both CPU and CUDA version

Changes Made

• Added the previous hidden and cell states to LSTM when seq_len=1 and batch_size=1
• Added updating kernels for the cell and hidden states
• Modified the LSTM parameter initialization. fan_in = input_size + output_size instead fan_in = input_size
• Fixed bugs in resetting states in LSTM layer
• Added input_state_update attribute to Python class 'Sequential'

Notes for Reviewer

• Run the following example for testing new feature

  python -m examples.time_series_forecasting

• Shuffling training data leads to better results for some reasons
• A single LSTM layers leads to better results
• Better parameter initialization could lead to better results but need further investigation
• Don't use multiprocessing if your hidden nodes is small i.e. net.set_threads(1)
• Currently use traditional tanh and sigmoid for lstm. Mixture tand and sigmoid might improve the results

[lhnguyen102]

@van-dai-vuong If you have any comments/questions on any changes, don't hesitate to ask through the GitHub review system. I plan to close the PR today so we can release the fix

[van-dai-vuong]

I only have one comment, other than that, I tested. It runs good. Thanks @lhnguyen102.