Closed Sum02dean closed 2 years ago
For example, another run on human generates this plot for score correctness:
Really? So these two plots are just different sampling of the data? No any other difference in parametrisation? The score distribution looks vastly different. There is enough points that they should behave consistently given the amount of data we are training and testing on. It's wild...
Yes, same parameterization.
Sometimes the cuve is exponential, sometimes it is linear. This is most likely due to random sampling of the data during model training. To address this we would like to sample data in a smarter way.