search

kelvinguu / neural-editor

Repository for "Generating Sentences by Editing Prototypes"
328 stars 61 forks source link

Memory Usage Problem #9

Open NikcPoon opened 6 years ago

NikcPoon commented 6 years ago

Hi Kelvin

Thans for the great work! I try the default editor on personal dataset and it works well. However during the training I found that the memory usage increasingly goes up, which leads to the experiment being killed after some training steps. Is it because I run the model in a wrong way?(I just prepare the dataset and use main.py in editor model fold.) And I want to know how to fix this problem. Thanks

YadiLao commented 6 years ago

Hi NikcPoon, I try to run default editor as well, but I encounter some bugs when training. Can you tell me the way you run?

NikcPoon commented 6 years ago

@YadiLao Hi YadiLao, I just run the main.py in edit_model/. I think it will be helpful if you provide some Traceback messages

Vonzpf commented 6 years ago

@NikcPoon Hi NikcPoon, I had finished the training on onebillion dataset. My memory usage is stable during the training and it only increases quickly in data reading process. I have a question that how do you begin the test? Dose the training contain results？ I just found some outputs in neural-editor-data/edit_runs/0/stdout.txt like this:

SOURCE: an e - mail message seeking comment from the club was not immediately returned . INSERT: [u"'s", u'group', u'publicist', u'sent', u'to'] DELETE: [u'club', u'comment', u'from', u'message', u'seeking'] TARGET: an e - mail sent to the group 's publicist was not immediately returned . (0.02) an e - mail message seeking comment from the company was not immediately answered . (0.02) an e - mail message seeking comment from the company was not immediately returned . (0.02) an e - mail message seeking comment was not immediately returned . (0.01) an e - mail message seeking comment from was not immediately returned . (0.01) an e - mail message seeking comment from the company was not immediately returned .

NickPoon commented 6 years ago

@Vonzpf
Hi Vonzpf. If I remember correctly, the training will log some result from random pick data. Actually I wrote my own test script. The general idea is that you can use editor.edit() to do inference.

Vonzpf commented 6 years ago

@NikcPoon I understand it. Thank you for your help!