dsindex
commented
6 years ago 'parser_trainer.py' has '--pretrained_params, --pretrained_params_names' parameters. in documentation, they are used for global training.
https://github.com/tensorflow/models/blob/master/research/syntaxnet/g3doc/syntaxnet-tutorial.md
bazel-bin/syntaxnet/parser_trainer \
--arg_prefix=brain_parser \
--batch_size=8 \
--decay_steps=100 \
--graph_builder=structured \
--hidden_layer_sizes=200,200 \
--learning_rate=0.02 \
--momentum=0.9 \
--output_path=models \
--task_context=models/brain_parser/greedy/$PARAMS/context \
--seed=0 \
--training_corpus=projectivized-training-corpus \
--tuning_corpus=tagged-tuning-corpus \
--params=200x200-0.02-100-0.9-0 \
--pretrained_params=models/brain_parser/greedy/$PARAMS/model \
--pretrained_params_names=\
embedding_matrix_0,embedding_matrix_1,embedding_matrix_2,\
bias_0,weights_0,bias_1,weights_1but i guess it could be used for retraining 'brain_tagger'. so i modified 'train.sh' for retraining 'brain_tagger' like below :
TAGGER_PARAMS=${TAGGER_HIDDEN_LAYER_PARAMS}-0.08-3600-0.9-0
function train_tagger {
${BINDIR}/parser_trainer \
--task_context=${CONTEXT} \
--arg_prefix=brain_tagger \
--compute_lexicon \
--graph_builder=greedy \
--training_corpus=training-corpus \
--tuning_corpus=tuning-corpus \
--output_path=${TMP_DIR} \
--batch_size=${BATCH_SIZE} \
--decay_steps=3600 \
--hidden_layer_sizes=${TAGGER_HIDDEN_LAYER_SIZES} \
--learning_rate=0.08 \
--momentum=0.9 \
--beam_size=1 \
--seed=0 \
--params=${TAGGER_PARAMS} \
--num_epochs=12 \
--report_every=100 \
--checkpoint_every=1000 \
--pretrained_params=${TMP_DIR}/brain_tagger/greedy/${TAGGER_PARAMS}/model \
--pretrained_params_names=embedding_matrix_0,embedding_matrix_1,embedding_matrix_2,bias_0,weights_0,bias_1,weights_1 \
--logtostderr
}and ran again, as you see, 'eval metric' is already 91.04% for epoch 1'
$ ./train.sh -v -v
2018-07-30 21:53:52.409090: I syntaxnet/reader_ops.cc:140] Starting epoch 1
2018-07-30 21:53:53.405823: I syntaxnet/reader_ops.cc:140] Starting epoch 2
INFO:tensorflow:Seconds elapsed in evaluation: 1.12, eval metric: 91.04%
INFO:tensorflow:Writing out trained parameters.
....
apurvnagvenkar
Is there a way to retrain the syntaxnet POS tagger model with new dataset?