search

shtoshni / long-doc-coref

Code for the EMNLP 2020 paper "Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks"
23 stars 4 forks source link

about the chinese dataset of ontonotes 5.0 #2

Closed leileilin closed 2 years ago

leileilin commented 2 years ago

hey, thanks for so wonderful work. if i want to train such a experiment on chinese dataset of ontonotes 5.0, how can i do it ? thanks.

shtoshni commented 2 years ago

What step are you stuck on? I would also recommend checking out our new repo

leileilin commented 2 years ago

What step are you stuck on? I would also recommend checking out our new repo

in th mention detection training step, u use a pretrained spanbert _coref, for chinese, it does not have such that pretrained model.

leileilin commented 2 years ago

And the litbank baseline 68.1 is different from 79.3 reported by the original paper.

shtoshni commented 2 years ago

Try this: https://huggingface.co/bert-base-multilingual-cased Regarding the number, 79.3 is with gold mentions i.e. assume that the mention detector is perfect.

leileilin commented 2 years ago

Try this: https://huggingface.co/bert-base-multilingual-cased Regarding the number, 79.3 is with gold mentions i.e. assume that the mention detector is perfect.

so if you method with gold mention, the number will be what? And try m-bert can not solve the pretrained coref model, because it is a pretrained language model, not a coref model.

leileilin commented 2 years ago

and the key question is how can u get 68.1? by what method?

shtoshni commented 2 years ago

Please check Table 5 of the LitBank paper - https://arxiv.org/pdf/1912.01140.pdf - where they report 68.1 performance when using predicted mentions which is what coreference models are compared on. We didn't report performance with gold mentions because it's just a diagnostic number. After all, gold mentions are typically not provided when using the model in the wild. Regarding using M-BERT, I would suggest jointly fine-tuning it along with the remainder of the model parameters. I would also urge a more careful reading of the papers and understanding of the coreference resolution task.