Improve memory usage - Githubissues

kentonl / e2e-coref

End-to-end Neural Coreference Resolution

Apache License 2.0

518 stars 174 forks source link

Improve memory usage #20

Closed mkuymkuy closed 5 years ago

mkuymkuy commented 6 years ago

-Implemented a function to load data one by one instead of once for all to effectively improve the memory usage. -Apply this function to decoder.py only. -Tested on large input data about 2,000,000 and the code was running fine. The old version would crash due to too large input data.

kentonl commented 5 years ago

This can't be merged due to some recent refactoring. However, the latest commit (c7ce6c2894fd75d155323698f7f38ec93c4913c4) should fix this issue.