search

ganeshjawahar / mem_absa

Aspect Based Sentiment Analysis using End-to-End Memory Networks
MIT License
199 stars 104 forks source link

Crash during training #8

Open jurukode opened 7 years ago

jurukode commented 7 years ago

Hi @ganeshjawahar,

i got experienced that the script always got killed after seven iterations automatically. Not sure what's happening. do you have any idea why?

error

joeybose commented 7 years ago

So in my experience, there is a resource exhaustion error and thats why the process is killed. There seems to be a bug in this code that uses a lot more GPU RAM than the original MemN2N, but I'm not sure where this bug is.

jurukode commented 7 years ago

Hi @0220joey,

yeah, by the way i'm only using CPU right now and the training process eat so many RAM until my laptop got hang. Thanks for the info by the way!

joeybose commented 7 years ago

I've managed to fix the issue, its in all the tf.assign which adds more nodes to the graph. So comment out lines 156-170, 211-225 in model.py. Although, the accuracy is about 10% less than the paper.

jurukode commented 7 years ago

Thanks @0220joey,

it works well and run faster. Wondering if the commented code are vital to increase accuracy or not

joeybose commented 7 years ago

I dont think so, but the paper does a few more tricks that this code doesnt do but I cant be too sure if thats the cause of the accuracy increase.