search

kamalkraj / ALBERT-TF2.0

ALBERT model Pretraining and Fine Tuning using TF2.0
Apache License 2.0
199 stars 45 forks source link

Unused weights from saved model #23

Open xixiaoyao opened 4 years ago

xixiaoyao commented 4 years ago

I run convert.py to convert albert tensorhub model to TF2.0 model with following commands

MODEL_DIR=albert-base
SIZE=base

# Converting weights to TF 2.0
python converter.py --tf_hub_path=${MODEL_DIR}/ --model_type=albert_encoder --version=2 --model=${SIZE}
# Copy albert_config.json to config.json
cp ${MODEL_DIR}/assets/albert_config.json ${MODEL_DIR}/config.json
# Rename assets to vocab
mv ${MODEL_DIR}/assets/ ${MODEL_DIR}/vocab

however, at the end of converting, it shows following messages

Done loading 25 ALBERT weights from: pretrain/albert-base-v2// into <albert.AlbertModel object at 0x7f393e172b00> (prefix:albert). Count of weights not found in the checkpoint was: [0]. Count of weights with mismatched shape: [0]
Unused weights from saved model: 
    cls/predictions/output_bias
    cls/predictions/transform/LayerNorm/beta
    cls/predictions/transform/LayerNorm/gamma
    cls/predictions/transform/dense/bias
    cls/predictions/transform/dense/kernel

Is this message showed convert success?

kamalkraj commented 4 years ago

yes