verbalization

[SapaePhyu]

Hi Sainz, I am trying to reproduce results on your paper "Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning", but cannot find the method of verbalization either in the paper or code. In Label verbalization section, you mentioned that "A verbalization is generated using templates that have been manually written based on the task guidelines of each dataset.", may I know how a sentence is generated after giving the model labels, template and original sentence? For example, when filling the template of " bought something" in Figure 1, how does the model know to choose "John D. Idol" but not "hired"? Please kindly reply when you have time.

[osainz59]

Hi @SapaePhyu !

The eassiest way to reproduce the results is by running the following command:

python -m a2t.evaluation --config resources/predefined_configs/{DATASET}.arguments.config.json

Where {DATASET} can be ace or wikievents.

However, answering your question, the templates contain placeholders for different information, such as the event trigger ({trg}), trigger type ({trg_type}) and of course, the filler candidate ({arg}). So, on your example:

text = "..."
template = "{arg} bought something."
verbalization = template.format(**{"arg": "John D. Idol", "trg": "hired", "trg_type": "..."})

model_input = "{original_sentence} </s> {verbalization}".format(original_sentence=text, verbalization=verbalization)

Note that this is a very simplified example.

You can see the templates on the appendix of the paper or in the task configs at resources/predefined_configs/.

[SapaePhyu]

Thank you, Sainz! Is it correct to think that the inputs of placeholders are the outputs of NER? If so, is there any constraint of selecting the entities, or do the placeholders accept all entities one by one?

[osainz59]

Yes! The actual models perform trigger-entity classification in order to assign role fillers to the events. Depending on your evaluation you would want to use gold entities or some entities predicted from a NER model. Regarding the constraints, in the configuration file for each dataset there are defined what we call valid_conditions, that is, the trigger type entity type pairs that are allowed for a given role. Although the probabilities are computed for every trigger-entity pair, only the ones that fulfill the valid conditions are accepted as valid answers.

[SapaePhyu]

Thank you so much for your detailed explanation!!