search

abisee / pointer-generator

Code for the ACL 2017 paper "Get To The Point: Summarization with Pointer-Generator Networks"
Other
2.18k stars 813 forks source link

Do anyone wanna try to add this model to transformer? #140

Closed xiongma closed 5 years ago

xiongma commented 5 years ago

I am try to add this model to transformer, if anyone wanna try it, please contract me, we can discuss with each other, my wechat is 15025700935, my e-mail is mx15025700935@aliyun.com

xiongma commented 5 years ago

@abisee Hi, I am a NLP worker, now I want add this model to transformer, but when I train transformer+pointer, I get nan, I try many ways, but the result still nan, I hope I can get your help very much, thx!

Santosh-Gupta commented 5 years ago

I believe the Tensor2Tensor library has a Transformer with pointer-generator

pltrdy commented 5 years ago

The pointer-generator (aka. copy mechanism) has been used with Transformer in Gehrmann et al, 2018 and is implemented using OpenNMT-py see tutorial: http://opennmt.net/OpenNMT-py/Summarization.html

Santosh-Gupta commented 5 years ago

@pltrdy thanks for the links! Do you know if it's possible to use sub tokens with that library? I am attempting summarization for scientific texts which have a lot of unique words, and I feel that sub tokens may be better for that.

pltrdy commented 5 years ago

@Santosh-Gupta Yes you can ! It's probably not embedded in the code, you should pre-process by yourself. There's probably doc about that (in the context of translation)