search

clulab / reach

Reach Biomedical Information Extraction
Other
96 stars 39 forks source link

Long antecedent chains will blow the stack #791

Open kwalcock opened 1 year ago

kwalcock commented 1 year ago

One 3.5MB nxml file with 50029 mentions seems to form antecedent chains long enough to blow the stack. One CorefEventMention has a "controller" argument that is a CorefTextBoundMention that has antecedents that have antecedents that have antecedents for a long, long ways so that the stack is exhausted as the json output is constructed. I followed it down manually a couple dozen steps and don't see any circles.

I wonder whether we should be recursively outputting anything. It is leading to output files of 50GB+. Can we just output an ID for mentions that are within triggers, arguments, antecedents, paths, etc.? A mention with that ID should appear elsewhere in the list, shouldn't it? Something else can then perform the recursive search.

In Eidos this was avoided by using jsonld in which we could just point to the other mentions with links. To the extent that equivalency hashes are unique (which I think is assumed), those can be used as links.

kwalcock commented 1 year ago

It recurses about 70 times with each recursion being 14 function calls, so the stack is over 980 frames deep.

MihaiSurdeanu commented 1 year ago

Nice catch! Yes, I think your solution makes sense.

kwalcock commented 1 year ago

Is there any specification of the current output formats? I don't see any.

MihaiSurdeanu commented 1 year ago

We only have doc for the FRIES format: https://github.com/clulab/reach/blob/master/doc/fries-data-representation-spec.txt