search

microsoft / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
https://docs.microsoft.com/cognitive-toolkit/
Other
17.52k stars 4.28k forks source link

LSTM example doesn't make sense #2816

Open chrispugmire opened 6 years ago

chrispugmire commented 6 years ago

In this item someone asks for an example using c# and LSTM for forcasting, https://github.com/Microsoft/CNTK/issues/647 An answer is given as: https://github.com/zpbappi/simple-cntk-lstm

The example given uses a data file of this format containing sine wave data.

|t 1.0 |f 0.8414709848078965 |t 1.1 |f 0.8101885091467594 |t 1.2 |f 0.7766992383060218

Since there are no sequence numbers on the data lines, this isn't a sequence, it's a list of sequences each containing one point. So using LSTM is somewhat meaningless...

Am I wrong, it's my understanding that for a sequence you need this format:

0 |t 1.0 |f 0.8414709848078965 0 |t 1.1 |f 0.8101885091467594 0 |t 1.2 |f 0.7766992383060218

So is this example completely wrong, or is the documentation on ctf file formats incorrect? Any advice would be appreciated, it would be great if anyone could provide a working c# cntk example using LSTM

Thanks.

frankibem commented 6 years ago

From the docs:

Sequence id is a number. It can be omitted, in which case the line number will be used as the sequence id

So technically you can have sequences with a single sample. Whether or not this is useful will depend on the problem you're trying to solve - in the example in the link above, using an LSTM seems to be overkill as the data is not really 'sequential'.

Perhaps the official tutorials would provide a more suitable example for you.

chrispugmire commented 6 years ago

Thanks for confirming that, I've been banging my head against a wall trying to understand why it didn't work. Can someone remove the faulty example or at least note on the other page that it doesn't actually implement sequences so shouldn't be used as a template.

Sadly the official tutorials are mostly written in python or brainscript, and since the API is not remotely similar between c# / python versions of CNTK the examples don't really help at all. Functions have different names and/or parameters or don't exist at all. It would really be good if someone who understood the C# api went through and implemented all the examples so we had at least a minimal set to work with.

It's curious with all the investment in AI that Microsoft seems to have left the C# implementation in such a poor state, it's almost like they don't believe in their own product?

douglas125 commented 6 years ago

I would also appreciate a detailed example on LSTMs, featuring many-to-many, many-to-one and the other recurrent architectures.