Closed xiongma closed 5 years ago
@abisee Hi, I am a NLP worker, now I want add this model to transformer, but when I train transformer+pointer, I get nan, I try many ways, but the result still nan, I hope I can get your help very much, thx!
I believe the Tensor2Tensor library has a Transformer with pointer-generator
The pointer-generator (aka. copy mechanism) has been used with Transformer in Gehrmann et al, 2018 and is implemented using OpenNMT-py see tutorial: http://opennmt.net/OpenNMT-py/Summarization.html
@pltrdy thanks for the links! Do you know if it's possible to use sub tokens with that library? I am attempting summarization for scientific texts which have a lot of unique words, and I feel that sub tokens may be better for that.
@Santosh-Gupta Yes you can ! It's probably not embedded in the code, you should pre-process by yourself. There's probably doc about that (in the context of translation)
I am try to add this model to transformer, if anyone wanna try it, please contract me, we can discuss with each other, my wechat is 15025700935, my e-mail is mx15025700935@aliyun.com