search

AI21Labs / sense-bert

This is the code for loading the SenseBERT model, described in our paper from ACL 2020.
Apache License 2.0
42 stars 9 forks source link

source code for training of senseBert #1

Closed papasanimohansrinivas closed 3 years ago

papasanimohansrinivas commented 3 years ago

guys are u planning to release sensebert training code and dataset , if so when @yoel-zeldes @oriram

oriram commented 3 years ago

Hi @papasanimohansrinivas , thanks for your interest! Unfortunately, we'll not release the training code, as it's proprietary. Basically the code is straightforward - we add another head, parallel to BERT's Masked LM head.

The training data is exactly the same as BERT's (Wiki and BookCorpus), with the exception that we add supersense labels to masked words (according to WordNet), as described in the paper.