Can you add a DEMO of LSTM for time series ,Please

[sharpwood]

SiaNet is great ! Thanks for your work. Can you add a DEMO of LSTM for time series . thanks!

[deepakkumar1984]

Thanks mate! sure will add to my plan

[sharpwood]

Thanks very much。 I see you mark this issue as a question, is it possible that the existing version of the function can be implemented? But I see that class Recurrent has not been cited If the existing version can be implemented, can you give a line of key code, thank you.

[deepakkumar1984]

Sorry my bad. I have to implement LSTM, GRU and RNN properly and provide examples. One the plan of development.

[sharpwood]

:+1:

[sharpwood]

LSTM has been added to the layer folder. Is it available now?

[deepakkumar1984]

I added the LSTM layer yesterday but didn't have time to test with sample data. But you are welcome to test and let me know how it goes.

On Wed, 22 Nov 2017 at 2:25 pm, sharpwood notifications@github.com wrote:

LSTM has been added to the layer folder. Is it available now?

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-346236135, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKc2MLgYR3bjzI_RljSWhHS6XSyjlks5s45sagaJpZM4Qj2f- .

[sharpwood]

I added a LSTM layer, but it was wrong when traning: Objects that can be empty must have a value

[deepakkumar1984]

Ok, Did you get any error. Could you please share the code so that I can debug at my end along with sample training data.

Thanks

On Wed, Nov 22, 2017 at 4:16 PM, sharpwood notifications@github.com wrote:

I added a LSTM layer, but it was wrong when traning: Objects that can be empty must have a value

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-346250028, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKQXYBslkCzZ1i3B4ZCRyM8dSsML9ks5s47VTgaJpZM4Qj2f- .

-- Regards, Deepak

[sharpwood]

I only try add a line : model.Add(new LSTM(dim:20,cellDim:1)); in HousingRegression BuildModel method

[deepakkumar1984]

Fixed the issue with the LSTM. Can you please check now. Although I am not sure if LSTM will work with MSE loss function. Also with Sparse input like starting with Embedding layer. Please advice?

[deepakkumar1984]

Will try to implement this example: https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

Hope to complete by tomorrow

[sharpwood]

that is great!!!

[sharpwood]

[sharpwood]

model.Add(new LSTM(20, cellDim: 5)); still in HousingRegression

[deepakkumar1984]

Oh no i checked in the code after adding the test lstm layer

On Wed, 22 Nov 2017 at 8:05 pm, sharpwood notifications@github.com wrote:

model.Add(new LSTM(20, cellDim: 5)); still in HousingRegression

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-346294729, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKUdOD1RRkeNKEd_eJ9leBe6VWvs5ks5s4-rwgaJpZM4Qj2f- .

-- Regards, Deepak

[sharpwood]

[deepakkumar1984]

Should work now! Please take the latest

[sharpwood]

That is ok. When will TimeSeriesPrediction be finished?

[deepakkumar1984]

I added the example code and testing in progress. Having some issue with training. Will try to complete by tomorrow

[deepakkumar1984]

Try out this example code with some of your sample data: https://github.com/deepakkumar1984/SiaNet/blob/master/SieNet.Examples/TimeSeriesPrediction.cs

[sharpwood]

I changed lookback to 3, how to define LSTM shape, as if it was different from Keras

[deepakkumar1984]

The lookback is the shape of the input. The number of lookback will convert the series data to tabular dataset with 3 columns which will be the input shape.

I have updated the time series example, could you please try again?

model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback)));

[sharpwood]

I have got last version, it works ,thanks. If I have multiple features in csv file, how to define the shape?

[deepakkumar1984]

Ok Its not implemented for multivariate. Let me work on that now and have another example ready for you.

[deepakkumar1984]

And yes it supports multiple LSTM..

model = new Sequential(); model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback), returnSequence: true)); model.Add(new LSTM(dim: 4, shape: Shape.Create(lookback))); model.Add(new Dense(dim: 1));

if you would like to stack the LSTM then need to use returnSequence: true for the previous layer. If returnSequence: false (y default) it will return the last sequence.

[sharpwood]

I think it may be that I don't make it clear. My English is poor. I mean not a multi-layer LSTM, but a number of features input data. For example:

[deepakkumar1984]

Do you have any example with keras or another python framework which I can use as reference? Seems like this is a general regression problem and not time series which can be solved by using Dense layer. Maybe I am wrong?

On Wed, Nov 29, 2017 at 10:28 AM, sharpwood notifications@github.com wrote:

I think it may be that I don't make it clear. My English is poor. I mean not a multi-layer LSTM, but a number of features input data. For example: [image: image] https://user-images.githubusercontent.com/7440304/33350683-0f5eaa62-d4db-11e7-89df-ed96b5d7857d.png

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-347705415, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKagoGRJsLQxqU6EY6sWoVaBiAqnoks5s7J4fgaJpZM4Qj2f- .

-- Regards, Deepak

[sharpwood]

https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/

Is this example appropriate?

[sharpwood]

https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/

[deepakkumar1984]

On Wed, 29 Nov 2017 at 8:04 pm, sharpwood notifications@github.com wrote:

https://machinelearningmastery.com/reshape-input-data-long-short-term-memory-networks-keras/

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-347803555, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKbwh5eYZXZaSIjf408_HSJK_hPbpks5s7SUXgaJpZM4Qj2f- .

Thanks mate. This example looks good. I will start the work -- Regards, Deepak

[sharpwood]

Reshape layer has been committed , Would you update the example?

[deepakkumar1984]

Yes, Will do that

On Fri, Dec 1, 2017 at 4:32 PM, sharpwood notifications@github.com wrote:

Reshape layer has been committed , Would you update the example?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-348409327, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKcMmq9gSk_GfQrMr3yr4G3yvtiAIks5s75ZYgaJpZM4Qj2f- .

-- Regards, Deepak

[bmigette]

Looks interesting :) Will follow up this project / thread.

[sharpwood]

How's the example program going?

[deepakkumar1984]

I will have something by tomorrow. Was working on other fixes on priority.

[deepakkumar1984]

Please use this example to use Reshape layer: https://github.com/deepakkumar1984/SiaNet/blob/master/SieNet.Examples/MiltiVariateTimeSeriesPrediction.cs

[bmigette]

Do you have any documentation that explains what are shapes, and why do we need a reshape layer ? Nevermind, found explanation here: https://www.tensorflow.org/programmers_guide/tensors :)

[deepakkumar1984]

I need to prepare all that will take some time because i am doing this work in my spare time. Plus some issues are there which need to be fixes on high priority. Contribution are always welcome 😉😉😉

On Thu, 7 Dec 2017 at 8:05 pm, bmigette notifications@github.com wrote:

Do you have any documentation that explains what are shapes, and why do we need a reshape layer ?

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/9#issuecomment-349913259, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKZSCqPVrZFHi_hgBnSCdFTyVWY5mks5s97FegaJpZM4Qj2f- .

-- Regards, Deepak

[sharpwood]

Normalize and ConvertTimeSeries method seems to have some problems.

Normalize method generate some NaN values , and The ConvertTimeSeries method does not seem to be able to handle multi - column data.

[deepakkumar1984]

The data frame is bit basic now. Need to enhance with missing values and load datatype other than float. For now you need to sanitize the dataset by yourself and then load it. You can use the framework like Deedle to load the csv and make it ready for training. I may plan to make use of Deedle in the project and way of converting Deedle output to CNTK value.

[deepakkumar1984]

Closing it off as there is no activity in last 180 days