search

zhongkaifu / RNNSharp

RNNSharp is a toolkit of deep recurrent neural network which is widely used for many different kinds of tasks, such as sequence labeling, sequence-to-sequence and so on. It's written by C# language and based on .NET framework 4.6 or above versions. RNNSharp supports many different types of networks, such as forward and bi-directional network, sequence-to-sequence network, and different types of layers, such as LSTM, Softmax, sampled Softmax and others.
BSD 3-Clause "New" or "Revised" License
285 stars 92 forks source link

Model structure #22

Closed My-Khan closed 7 years ago

My-Khan commented 8 years ago

Hello , Can i represent DRNN in the below structure for Named entity task. any suggestion for improvement will be highly appreciated. Also is it based on Elman Architecture which is described in (Kaisheng Yao et al.,2013) ? image

zhongkaifu commented 8 years ago

Yes. This is a basic RNN model structure: single recurrent hidden layer and forward model. It can be used for NER task. For more complicated model, you could try bi-directional model, deep recurrent hidden layer, different recurrent hidden type(BPTT, LSTM), CRF output and so on.

My-Khan commented 8 years ago

many thanks for clarification. also on the home page of RNNShrap the model structure given their for deep bi-directional RNN-CRF network all the three hidden layer are labeled Hidden layer #1, Hidden layer #1 ,Hidden layer #1 . is this the typing mistake? or its real.

also the equation used from computation is this one h(t)=σ(U.x(t)+W.h(t-1) ), t=1….T and y(t)=g(Vh(t))

if yes then please explain g(Vh(t)).

is same equation is used for computation in basic RNN and DEEP RNN?

zhongkaifu commented 8 years ago

Thanks for pointing out. It's a typo. They should be "Hidden layer #1", "Hidden layer #2", "Hidden layer #3". I will fix it.

g(Vh(t)) is the function to calcuate y(t) by using the output of hidden layer and weights between hidden layer and y.

My-Khan commented 8 years ago

Many thanks. can i use your diagrams in my paper with reference?

zhongkaifu commented 8 years ago

Sure. I'm glad these diagrams is helpful for your paper.

My-Khan commented 8 years ago

so nice of you

zhongkaifu commented 8 years ago

Fixed it. Thanks for your pointing it out.

My-Khan commented 8 years ago

Appreciated , Is in future there is chance of implementation of Skip-Chain CRF in C# from you?

zhongkaifu commented 8 years ago

Maybe, but currently, I have no plan about it.

My-Khan commented 8 years ago

Ok thanks for kind reponse