search

bojone / bert4keras

keras implement of transformers for humans
https://kexue.fm/archives/6915
Apache License 2.0
5.37k stars 929 forks source link

pretrain词向量 #311

Open TyrionZK opened 3 years ago

TyrionZK commented 3 years ago

目前bert4keras可以实现仅使用encoder的预训练权重，并且随机初始化embedding层的权重吗

bojone commented 3 years ago 
embeddings = model.get_layer('Embedding-Token').embeddings
K.set_value(embeddings, value)

value是一个shape一致的numpy矩阵