Training loss points in .tfevent differ from num_train_steps

AdeDZY / DeepCT

DeepCT and HDCT uses BERT to generate novel, context-aware bag-of-words term weights for documents and queries.

BSD 3-Clause "New" or "Revised" License

312 stars 46 forks source link

Training loss points in .tfevent differ from num_train_steps #19

Open DankoPenko opened 2 years ago

DankoPenko commented 2 years ago

Hi!

I would like to plot the training loss of the model, but in the .tfevents-file number of tags that begin with "loss_1" differ from the calculated number of steps (num_train_steps) .

As for example, for num_train_steps = 153596, my .tfevent-file has only 1552 tags of "loss_1".

Is there any hint of how to get the losses or how its being saved in .tfevent?

Thanks in advance!