FrameNet Based Semantic Role Annotation

[Vyso]

Hey again, @microth :) I'm sorry to keep raising issues for problems this trivial, but its really hard to find help related to this package. Its just my deficiency in understanding certain subjects I'm hoping you can help me with.

Uh, I was successful in using the complete pipeline on a sentence, and the result was this

---

---

I just wanted to know how I could get the FrameNet kind of semantic annotations, on the input text, like shown below. (Taken from the SEMAFOR Demo page.)

---

---

Like, 'born' has a Being_Born semantic frame to it, and I'd like to have my input text annotated as shown in the second picture. Is it possible? If you don't mind, could you guide me on how I could achieve this? Below are the arguments I passed

---

CompletePipelineCMDLineOptions options = new CompletePipelineCMDLineOptions(); String[] arss = {"eng", "-lemma", "/Users/vishnumohan/Desktop/LTh/PathLSTM-pre-illinois-built/src/main/java/se/lth/cs/srl/models/CoNLL2009-ST-English-ALL.anna-3.3.lemmatizer.model", "-tagger", "/Users/vishnumohan/Desktop/LTh/PathLSTM-pre-illinois-built/src/main/java/se/lth/cs/srl/models/CoNLL2009-ST-English-ALL.anna-3.3.postagger.model", "-parser", "/Users/vishnumohan/Desktop/LTh/PathLSTM-pre-illinois-built/src/main/java/se/lth/cs/srl/models/CoNLL2009-ST-English-ALL.anna-3.3.parser.model", "-srl", "/Users/vishnumohan/Desktop/LTh/PathLSTM-pre-illinois-built/srl-FN17.model", "-framenet", "/Users/vishnumohan/Desktop/LTh/fndata-1.7", "-tokenize", "-reranker", "-externalNNs", "-test", "/Users/vishnumohan/Desktop/LTh/PathLSTM-pre-illinois-built/src/main/java/se/lth/cs/srl/tesen.txt"}; options.parseCmdLineArgs(arss);

---

After reading through the docs, I also saw people mentioning a srl-ICCG16-eng.model file. Could you provide me with a link, for the same?

Best Regards, Vishnu

[microth]

Hi, you don't need the srl-ICCG16-eng.model. It is just an outdated version of the file srl-FN17.model. The arguments that you are passing on to the command line look good. What does the output look like?

If you don't see many frame and frame element assignments, it will be because the FrameNet model is trained using a different preprocessing pipeline. You can find the details here: https://github.com/microth/PathLSTM/#dependencies

Cheers, Michael

[brombach]

@microth What model for bist-parser do you use? The one downloadable from the readme? I was running into issues with that model due to the fact that it was created with boost and seemingly requires an exact match of boost version to function. The creator of bist-parser recommended training my own model on the Universal Dependencies relations data, which I've done, but is this the tagset expected?

[microth]

No, the model is not based on Universal Dependency tags/relations. I used a bist-parser model trained on OntoNotes 5.0. The model can be downloaded here: https://drive.google.com/file/d/0B5aLxfs6OvZBa21uQU9Lal9qZ28/view?usp=sharing

I forgot to add a link to this in the README. Will update it soon.

[Vyso]

Hey, @microth Uh, the output I got is what is posted above (https://drive.google.com/file/d/0B0nfa4AZPs3yT2o4OE5wa3RhUms/view?usp=sharing)

Uh, just by changing the dependency parsing technique (BIST) as recommended, will there really be a considerable difference in the semantic frames appearing in my output as to just using the default CompletePipeline with the Bohnet models?

[microth]

Yes, this is what the output will look like:

1   He  he  he  PRP PRP _   _   3   3   NSUBJPASS   NSUBJPASS   _   _   Child   _
2   was be  be  VBD VBD _   _   3   3   AUXPASS AUXPASS _   _   _   _
3   born    bear    bear    VBN VBN _   _   0   0   ROOT    ROOT    Y   Being_born  _   _
4   an  an  an  DT  DT  _   _   5   5   DET DET _   _   _   _
5   year    year    year    NN  NN  _   _   6   6   NPADVMOD    NPADVMOD    Y   Calendric_unit  _   Unit
6   ago ago ago RB  RB  _   _   3   3   TMP TMP _   _   Time    _
7   .   .   .   .   .   _   _   3   3   PUNCT   PUNCT   _   _   _   _

That is, "born" should evoke the "Being_born" frame, with the frame elements Child ("He") and Time ("an year ago"); "year" should evoke a "Calendric_unit" frame of which it fills the Unit element.

[Vyso]

Woah! That's exactly what I want. Uh, lastly before you close the issue, could you just clear one more doubt for me? The BistParser doesn't seem to accept plain sentences, and requires the user to input files formatted in CoNLL format, but which columns are supposed to be filled, in the input file? i.e., is there any class or package which transforms a text file with the sentence "He was born an year ago", to the CoNLL format that BIST parser needs, as input? How did you do that?

If y'don't mind, could you show me what your command line for the BIST parser was? (What you substituted, here.)

---

python src/parser.py --predict --outdir [results directory] --test test.conll [--extrn extrn.vectors] --model [trained model file] --params [param file generate during training]

---

Thank you for your time, @microth Best Regards, Vishnu

[microth]

Here is the command that I am using for BIST parser:

python2.7 src/parser.py --predict --outdir . --test <FILENAME> --extrn sskip.100.vectors --model model/clearpos/barchybrid.model4 --params model/clearpos/params.pickle

Unfortunately, you'll have to apply a part-of-speech tagger (I recommend NLP4J, see README) and convert your texts into the right format yourself. I don't have any scripts for all these steps and I am not going to provide step-by-step instructions. I hope you'll understand.

Cheers, Michael

[brombach]

@microth thanks for the model. do you know what version of boost it was created using? doesn't work with the latest 1.64.0_1 and dynet 1.1. Thanks for all your quick responses.

@Vyso I have a python API for bist-parser to accept raw sentences, but it is currently using a model for the graph parser (not hybrid), which I trained on Universal Dependencies data because I am having endless issues with boost/dynet for bist-parser. Thus, it's not yet working well as input for PathLSTM. Regardless, I'm happy to share something you could build off of. I hope to have a raw sentence -> bist-parser -> PathLSTM api done in the next few weeks.

[microth]

@brombach As far as I can remember, the model was trained using boost 1.58 and pycnn 1.0 (https://github.com/clab/cnn-v1).

[Vyso]

@brombach It's the same here, buddy. The dynet/bist configuration seems impossible to set. I'm still facing this, with dynet 2.0, and it seems it'd take quite a while to adapt to this version.

And yeah, that kind of an api would be a real great help, man. I'll be attempting to make it too, as soon as I have a successful run, with the hybrid parser :)

Cheers.

[brombach]

@Vyso Here is the api, plus the two files I modified, all pertaining to the graph parser. I also included a file I was using to rapidly test, which serves as a rough demo. Let me know if you have any breakthroughs on the boost/dynet issues for bist-parser. bist-api.zip

[microth]

@brombach Thank you for your efforts!

@all Due to an increasing number of requests, I decided to retrain the FrameNet model using Stanford CoreNLP. This will simplify the setup of the model and, at the same time, make it compatible with a commonly used preprocessing toolkit. The new model will be online by the end of this week.

[akb89]

@microth Would it be possible for you, while you're at it, to also provide a short README on how to (re-)train the model, for those of us who'd like to replicate some results (especially the ICCG 2016 ones) ? That would be tremendously helpful. I have a fully functional setup where I can generate das-like splits with MXPOST + MST or NLPJ + BIST. Should I use mateplus to train? If so is there any specific configuration I should use? Cheers, Alex

[microth]

@akb89 The repository contains no training code and I am currently not planning on making this available (due to time constraints). If you are interested, send me an email and I can give you the ICCG model and/or the SRL output on the test set. But keep in mind that the results are outdated anyway (see Swayamdipta et al., arXiv 2017; Yang and Mitchell, EMNLP 2017).

[microth]

I updated the code and uploaded a new model file. You can now use "-stanford" as a commandline option in order to apply the frame-semantic parser in combination with Stanford CoreNLP preprocessing (also see https://github.com/microth/PathLSTM/blob/pre-illinois-built/scripts/parse_fnet.sh).

Thanks everyone for your comments!