search

dbiir / UER-py

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo
https://github.com/dbiir/UER-py/wiki
Apache License 2.0
3.01k stars 525 forks source link

Requirement for the combination of classification + MLM target #272

Open Embedding opened 2 years ago

Embedding commented 2 years ago

It is often the case that a piece of text is attached with a label, e.g. the comments with the ratings on restaurant. It is beneficial to use the joint training of classification and MLM targets on this type of corpus. Could you add classification+MLM model in the project?

Maybe if we use the following the options to construct the above model:

--config_path models/bert/base_config.json --data_processor cls_mlm --target cls mlm