Something else causing fab parse to abort...

HazyResearch / bazaar

14 stars 11 forks source link

Something else causing fab parse to abort... #12

Closed ajratner closed 9 years ago

ajratner commented 9 years ago

This run seems to have gone through almost the entire process with no errors, and I still can't find the suspect error/exception in the output logs...

This time it is not due to memory or disk space as before.

ajratner commented 9 years ago

Actually this was probably due to some sort of memory error- I guess maybe with segment sizes that are too large. I suppose coreNLP has some sort of persistent state that grows in size slowly as it runs...? Either way #13 still seems useful to quickly implement before trying again...

ajratner commented 9 years ago

Or has really big spikes in memory usage? Either way I will try splitting into smaller chunks (some balance between saving on coreNLP startup time & ideally smaller seg size) and keeping a server local register for recovery