search

dmis-lab / biobert

Bioinformatics'2020: BioBERT: a pre-trained biomedical language representation model for biomedical text mining
http://doi.org/10.1093/bioinformatics/btz682
Other
1.94k stars 451 forks source link

adding additional features for QA task #27

Closed telukuntla closed 5 years ago

telukuntla commented 5 years ago

Hi, my goal is to extract contextual word embeddings from last layer of BioBert and add additional features, before performing QA fine-tuning. please suggest me how to achieve this. also I found that when I print 'final_hidden = model.get_sequence_output()' to console, tensor shape is getting printed instead of contextual embeddings. yours suggestions are appreciated.

jhyuklee commented 5 years ago

Please see https://github.com/dmis-lab/biobert/issues/23 for extracting features from BioBERT. Thank you.