yandexdataschool / nlp_course

YSDA course in Natural Language Processing
https://lena-voita.github.io/nlp_course.html
MIT License
9.84k stars 2.61k forks source link

week9_da fix: load all weights into teacher model #16

Closed Tenich closed 6 years ago

Tenich commented 6 years ago

Line utils.load(tf.trainable_variables(), 'teacher.npz') in Domain adaptation with KL penalty task from seminar.ipynb yields warning

Variables not initialized: mod/enc_fw/i2c:0 mod/enc_fw/h2c:0 mod/enc_fw/b_c:0 mod/enc_fw/i2f:0 mod/enc_fw/h2f:0 mod/enc_fw/b_f:0 mod/enc_fw/i2i:0 mod/enc_fw/h2i:0 mod/enc_fw/b_i:0 mod/enc_fw/i2o:0 mod/enc_fw/h2o:0 mod/enc_fw/b_o:0 mod/enc_bw/i2c:0 mod/enc_bw/h2c:0 mod/enc_bw/b_c:0 mod/enc_bw/i2f:0 mod/enc_bw/h2f:0 mod/enc_bw/b_f:0 mod/enc_bw/i2i:0 mod/enc_bw/h2i:0 mod/enc_bw/b_i:0 mod/enc_bw/i2o:0 mod/enc_bw/h2o:0 mod/enc_bw/b_o:0 mod/decoder/i2c:0 mod/decoder/h2c:0 mod/decoder/b_c:0 mod/decoder/i2f:0 mod/decoder/h2f:0 mod/decoder/b_f:0 mod/decoder/i2i:0 mod/decoder/h2i:0 mod/decoder/b_i:0 mod/decoder/i2o:0 mod/decoder/h2o:0 mod/decoder/b_o:0 mod/attn/dec:0 mod/attn/enc:0 mod/attn/vec:0 mod/embedding_1/embeddings:0 mod/embedding_2/embeddings:0 mod/dense_1/kernel:0 mod/dense_1/bias:0 teacher/embedding_3/embeddings:0 teacher/embedding_4/embeddings:0 teacher/dense_2/kernel:0 teacher/dense_2/bias:0

Looks like keras ignores tensorflow name spaces for embeddings and last dense layer, so teacher model is not properly loaded. Hardfixing layers' names seems to help.

justheuristic commented 6 years ago

Спасибо! мы эту часть сделали во время семинара, не заметили :(