search

NorskRegnesentral / skweak

skweak: A software toolkit for weak supervision applied to NLP tasks
MIT License
917 stars 71 forks source link

Support for relation extraction #7

Open plison opened 3 years ago

plison commented 3 years ago

Right now, skweak supports two main types of NLP tasks: (token-level) sequence labelling and text classification. Both rests on the idea that labelling functions associate labels to text spans, and the role of the aggregation model is then to merge the outputs of those labelling functions such as to get unified predictions.

However, some NLP tasks cannot be easily associated to text spans. For instance, relation extraction necessitates a prediction on pairs of spans.

The question is then how to provide support for such type of tasks, for instance by implementing a RelationAnnotator that could be used to associate pairs of spans to a label.

Technically speaking, we could still encode the annotations internally as SpanGroup objects. One solution would be to only add one span of the pair in the SpanGroup, but then specify that this span is connected to a second span (SpanGroup objects allows the inclusion of JSON-serialised attributes). The method get_observation_df in the BaseAggregator class could then be extended to detect whether a span is a normal one, or is connected to a second span. If that is the case, the aggregation would then be done on pairs of spans instead of single spans.

Do get in touch if this functionality is something you need, so that we know whether we should prioritise this in our next release :-)

wjbmattingly commented 2 years ago

This is precisely what I need for multiple projects. I'm not sure what I can do to assist, but please do reach out if you need/want help.

fernandonjardim commented 2 years ago

This is totally what I need and a lot more people! I can even send you examples on what I am trying to do, with several corpus to be tagged haha

rjurney commented 2 years ago

This is something I'm very interested in. A lot of people need this.

@plison this would be great if it could integrate with flair. How much work would be involved? How hard would the implementation be that you describe?

I would love to pull this into https://github.com/Graphlet-AI/graphlet and use it for building knowledge graphs along with flair and BLINK.

Kael-DWT commented 1 year ago

I want this functionality. This is great.