search

Sum02dean / STRINGSCORE

1 stars 1 forks source link

Stochastic behaviour (score correctness) #20

Closed Sum02dean closed 2 years ago

Sum02dean commented 2 years ago

Sometimes the cuve is exponential, sometimes it is linear. This is most likely due to random sampling of the data during model training. To address this we would like to sample data in a smarter way.

image

Sum02dean commented 2 years ago

For example, another run on human generates this plot for score correctness:

image
damianszk commented 2 years ago

Really? So these two plots are just different sampling of the data? No any other difference in parametrisation? The score distribution looks vastly different. There is enough points that they should behave consistently given the amount of data we are training and testing on. It's wild...

Sum02dean commented 2 years ago

Yes, same parameterization.