f1_score: nan

[gshpychka]

At some low threshold of precision and recall, f1_score becomes nan. I'm getting outputs like this:

2306/10000 [=====>........................] - ETA: 21:25 - loss: 0.7265 - precision: 0.5095 - recall: 0.4593 - f1_score: nan

[gshpychka]

I am not sure that it has to do with some low threshold.

Training goes something like this:

1/10000 [..............................] - ETA: 26:00 - loss: 0.4511 - precision: 0.3333 - recall: 0.3333 - f1_score: 0.3333

2/10000 [..............................] - ETA: 25:25 - loss: 0.4070 - precision: 0.4167 - recall: 0.3095 - f1_score: 0.3485

3/10000 [..............................] - ETA: 25:23 - loss: 0.4379 - precision: 0.2778 - recall: 0.2063 - f1_score: nan  

4/10000 [..............................] - ETA: 25:32 - loss: 0.4203 - precision: 0.2440 - recall: 0.4048 - f1_score: nan

And then it's 'nan's all the way down. Even if precision and recall go up to relatively decent numbers.

What kind of information would help you narrow this down?

[ybubnov]

@gshpychka, thank you a lot for posting the issue, this looks very strange. I wasn't being able to look into this issue last week, hopefully I'll figure out why is this happening on this one.

[deakkon]

Heres my two cents:

100/17172 [..............................] - ETA: 7:09 - loss: 0.0527 - acc: 0.9700 - precision: 0.8000 - recall: 0.8750 - f1_score: 0.8036 150/17172 [..............................] - ETA: 7:08 - loss: 0.0462 - acc: 0.9733 - precision: 0.8667 - recall: 0.8056 - f1_score: 0.8024 200/17172 [..............................] - ETA: 7:07 - loss: 0.0398 - acc: 0.9800 - precision: 0.9000 - recall: 0.8542 - f1_score: 0.8518 250/17172 [..............................] - ETA: 7:07 - loss: 0.0359 - acc: 0.9840 - precision: 0.9200 - recall: 0.8833 - f1_score: 0.8814 300/17172 [..............................] - ETA: 7:06 - loss: 0.0593 - acc: 0.9733 - precision: 0.8778 - recall: 0.8472 - f1_score: 0.8456 350/17172 [..............................] - ETA: 7:06 - loss: 0.0539 - acc: 0.9771 - precision: 0.8952 - recall: 0.8690 - f1_score: 0.8677 400/17172 [..............................] - ETA: 7:05 - loss: 0.0607 - acc: 0.9750 - precision: 0.9083 - recall: 0.8354 - f1_score: 0.8530 450/17172 [..............................] - ETA: 7:03 - loss: 0.0602 - acc: 0.9733 - precision: 0.9185 - recall: 0.8093 - f1_score: 0.8415 500/17172 [..............................] - ETA: 7:02 - loss: 0.0570 - acc: 0.9740 - precision: 0.8267 - recall: 0.7283 - f1_score: nan
550/17172 [..............................] - ETA: 7:00 - loss: 0.0579 - acc: 0.9727 - precision: 0.7970 - recall: 0.7076 - f1_score: nan 600/17172 [>.............................] - ETA: 6:59 - loss: 0.0565 - acc: 0.9750 - precision: 0.8139 - recall: 0.7319 - f1_score: nan 650/17172 [>.............................] - ETA: 6:58 - loss: 0.0531 - acc: 0.9769 - precision: 0.8282 - recall: 0.7526 - f1_score: nan 700/17172 [>.............................] - ETA: 6:56 - loss: 0.0535 - acc: 0.9771 - precision: 0.8405 - recall: 0.7583 - f1_score: nan 750/17172 [>.............................] - ETA: 6:55 - loss: 0.0510 - acc: 0.9787 - precision: 0.8511 - recall: 0.7744 - f1_score: nan 800/17172 [>.............................] - ETA: 6:54 - loss: 0.0503 - acc: 0.9800 - precision: 0.8604 - recall: 0.7885 - f1_score: nan 850/17172 [>.............................] - ETA: 6:53 - loss: 0.0505 - acc: 0.9800 - precision: 0.8569 - recall: 0.8010 - f1_score: nan

Not sure if its connected to P/R values or epoch numer (i.e. it gets overwritten by something/nan) or something else.

[gshpychka]

I don't actually think that it has to do with the values of P and R. Seems like it disappears after some number of iterations.

See above - it displayed fine at much lower values of P and R, before disappearing.

[deakkon]

@gshpychka just updated my comment as in later batches, with P/R > 0.8, nan was calculated as f1 score.

[dgrahn]

^ In fact, the code that keras used to use is much simpler. Might want to just copy that over.

[ybubnov]

@gshpychka, @deakkon could you, please, verify that the issue is gone in version 0.0.4?

[ybubnov]

Closing the issue, it is resolved in 0.0.5 version.