RNN

[joobei]

Jorden I'm curious if you've ever used the RNN to do any actual predictions or classification. Your readme says that none of these are usable but I'm thinking that perhaps the readme is not updated. Have you successfully used the RNN in birdbrain for some task?

[jordenhill]

Hello, sorry for the late response!

I have run some very basic tests just to see if the RNN can converge to a desired output and find out if things like the backprogation-through-time algorithm work as one would expect them to, and in my experience it does appear that the basic components are in place for it to do that. I've been looking into getting a set of training data soon to observe how it performs and see if any changes or additions should be made to fix any unseen problems in the code and make it easier to use and train.

[joobei]

@jordenhill I tried training the birdbrain RNN for the following task: http://stats.stackexchange.com/questions/203272/how-can-i-find-patterns-or-irregularities-in-a-path-rnn

I feed the RNN the distance between the vertices [[x:Float,y:Float]] and whether or not that vertex is part of a "symbol" or not [[0]] or [1]. After 32000 training examples with a learning rate 0.05 the error only goes down to about 0.693147.

Could you please let me know the following:

• How many training examples did your training set have?
• How many layers did you initialize the RNN to?
• What learning rate did you use?
• What error value did the network converge to?

Many thanks in advance.

EDIT: "the basic components are in place for it to do that" So if I am to understand correctly you haven't yet tried to train the RNN in birdbrain?

[joobei]

You're welcome to try training it using my data:

Penimate

[jordenhill]

Thanks! I'll look into it.

No I haven't gotten to train an RNN with a reasonable dataset with this yet. Until recently most of the time I've put into this has been building the networks to at least have the algorithms needed for basic functionality, ensuring the algorithms behaved as expected from basic tests, and preventing the code from crashing. Now that the basics are in place, once I had more free time I was going to look into training them for things like simple classification and language modeling tasks to do more robust testing and debugging.