Fix LSTM Bug in CUDA Version

lhnguyen102 commented 1 month ago

Description

This PR fixes the bugs in cuda version for LSTM that raised in issue #67.

Changes Made

Fixed the kernel config for the concat function between intput and previous states
Initialized Jacobian matrix to one and send it to device
Refactored to backward state storing function
Added the CUDA check for debugging
Added the time series model for traffic data from van-dai-vuong

Related Issues

close #67 #66

Note for Reviewers

Sent out the invite to David and Dai but need to wait for their responses before assigning you as reviewers
Run two tests

python -m examples.time_series_forecasting
python -m examples.traffic_2008_01_14

van-dai vuong: Use save and load functionalities for save the best model here
van-dai vuong: this function is too long. It is prone to bugs and difficult to debug. The solution is break it down like a function that can run 1 time series at the time. You will then loop over it for many time series as needed.
van-dai vuong: it is better to create a dataloader that separates from c++/cuda backend. The dataloader for time series in c++ is used for testing purpose for a particular problem back then. Each time we change something in pytagi folder, we need to test and release new version which can be bloated. Based on the specific datasets, you should have your own dataloader saved in examples folder. You can use the Pytorch's dataloader like examples/mnist_bench.py for time series.

lhnguyen102 commented 1 month ago

@van-dai-vuong It is fixed for batch size of 1. In general, you dont need to approve if the issue hasn't fixed otherwise someone might accidentally merge to main. @davidwardan Could you please test it to see if it is fixed? Thanks

davidwardan commented 1 month ago

@lhnguyen102 batch_size = 1 for cuda is now working 👍🏻

lhnguyen102 commented 1 month ago

@davidwardan Awesome! could you approve my PR to merge?

lhnguyen102 / cuTAGI