On top of the provided models, the package also provides the ability to train and distill coreference models on your own data, opening the possibility for fast and accurate coreference models for additional languages and domains.
To be able to distil your own model you need:
A Large unlabeled dataset, for instance Wikipedia or any other source.
Guidelines:
Each dataset split (train/dev/test) should be in separate file.
Each file should be in jsonlines format
Each json line in the file must include at least one of:
text: str - a raw text string.
tokens: List[str] - a list of tokens (tokenized text).
sentences: List[List[str]] - a list of lists of tokens (tokenized sentences).
clusters information (see next for annotation) as a span start/end indices of the provided field text(char level) tokens(word level) sentences(word level)`.
A model to annotate the clusters. For instance, It can be the package LingMessCoref model.
from fastcoref import LingMessCoref
model = LingMessCoref()
preds = model.predict(texts=texts, output_file='train_file_with_clusters.jsonlines')
3. Train and evaluate your own `FCoref`
```python
from fastcoref import TrainingArgs, CorefTrainer
args = TrainingArgs(
output_dir='test-trainer',
overwrite_output_dir=True,
model_name_or_path='distilroberta-base',
device='cuda:2',
epochs=129,
logging_steps=100,
eval_steps=100
) # you can control other arguments such as learning head and others.
trainer = CorefTrainer(
args=args,
train_file='train_file_with_clusters.jsonlines',
dev_file='path-to-dev-file', # optional
test_file='path-to-test-file' # optional
)
trainer.train()
trainer.evaluate(test=True)
trainer.push_to_hub('your-fast-coref-model-path')
After finish training your own model, push the model the huggingface hub (or keep it local), and load your model:
from fastcoref import FCoref
model = FCoref(
model_name_or_path='your-fast-coref-model-path',
device='cuda:0'
)
Distil your own coref model
On top of the provided models, the package also provides the ability to train and distill coreference models on your own data, opening the possibility for fast and accurate coreference models for additional languages and domains.
To be able to distil your own model you need:
A Large unlabeled dataset, for instance Wikipedia or any other source.
Guidelines:
jsonlines
formattext: str
- a raw text string.tokens: List[str]
- a list of tokens (tokenized text).sentences: List[List[str]]
- a list of lists of tokens (tokenized sentences).clusters
information (see next for annotation) as a span start/end indices of the provided fieldtext
(char level)tokens
(word level)sentences
(word level)`.A model to annotate the clusters. For instance, It can be the package
LingMessCoref
model.model = LingMessCoref() preds = model.predict(texts=texts, output_file='train_file_with_clusters.jsonlines')
After finish training your own model, push the model the huggingface hub (or keep it local), and load your model: