Beta calibration curve looks different from www

[SylvainChevalier]

beta:

www:

Let's check the maths, @lsabor.

[lsabor]

Somewhat fixed: the displayed Beta one is for the metaculus prediction, here is the recency-weighted community prediction version:

here is the updated rewrite version:

[lsabor]

I've moved this to medium priority. @SylvainChevalier feel free to move it back to high priority.

[SylvainChevalier]

As discussed yesterday, the top bin being bad and different is concerning.

[George3d6]

@SylvainChevalier are the bucket the same here ? it looks like the top is different in the new one (100 vs 95)

[lsabor]

Update: Recency Weighted main site:

Recency Weighted rewrite:

MP main site:

MP rewrite:

[lsabor]

Here's what happens when I set the forecast horizon to be from open to actual close time not scheduled close time:

What's going on here is that "coverage" in the main site for ComboPrediction is 100% if a forecast starts at open even if the question resolves early. Looks like that was our culprit...

[lsabor]

Well, that's definitely not the whole difference as my personal track record is still off:

Note the 4th and 6th bins are much closer in the main site than the rewrite.

[lsabor]

merged above changes with notes. still more to do though

[SylvainChevalier]

@lsabor what was left to do?

[lsabor]

I haven't looked at this since Sep 17. The rewrite version is still different from old, and I never got to the bottom of why. I think eventually this should be looked at seriously, and the math checked over...