Could you please give me an API that tells me how could I create a RHN?

[hfxunlp]

I have encountered in some problems in my experiments based on torch, I wonder rhn could help me and hope there could be a simple class that I could easily used to create a module and add it to a Sequential, but I can not find it in your discription and code.

[jzilly]

Thanks for being interested in RHNs. What exactly was the problem you encountered in your experiments?

What you may want to do is repurpose some of the code in: https://github.com/julian121266/RecurrentHighwayNetworks/blob/master/torch_rhn_ptb.lua This file has a local function "rhn" which you could use. To ensure proper regularization, this function also needs differently initialized dropout masks as suggested by Gal https://github.com/yaringal/BayesianRNN. These dropout masks are also created in the provided file.

For now we have not written RHNs as a "module", so I am not sure whether it would work right away with the "Sequential" command. You should nevertheless be able to reuse parts of the code to address your problem. Of course we would be more than happy if you would like to implement an RHN module. For now we have tried to keep the code as close as possible to Gal's code (linked above).

[hfxunlp]

@julian121266 Thank you for your help, I want to find a model better than lstm and gru, and try it on some tasks like sequence classification and neural machine translation. I hope I could implement this as a module based on your code.