Model training loss always 0

google-research / tapas

End-to-end neural table-text understanding models.

Apache License 2.0

1.14k stars 218 forks source link

Model training loss always 0 #117

Closed Shiva-Maruth-Alapati closed 3 years ago

Shiva-Maruth-Alapati commented 3 years ago

Hello,

I am finetuning the tapas model with the COVID-19 dataset. When I am giving the 'encoding["float_answer"]=[[28756489.0],[10114.0],[2449]]' the model is always showing a training loss of 0, but when I remove one of the answers i.e encoding["float_answer"]=[[np.nan],[10114.0],[2449]] the model is getting trained. Initially, I thought the model is predicting it accurately so the loss is zero but it is not the case.

eisenjulian commented 3 years ago

Hi @Shiva-Alapati-1998 can you share a colab or script to reproduce the problem? The problem might be that there is a threshold after which the unsupervised float loss is not computed anymore. See the answer loss cutoff argument described in the paper https://arxiv.org/pdf/2004.02349.pdf

Potentially changing the answer_loss_cutoff parameter to a larger number can help in your use case.

eisenjulian commented 3 years ago

Closing for the time being, please feel free to reopen if needed