search

empirical-org / Empirical-Core

Empirical Core is the mono repo for the Quill.org codebase. This repo contains both the Quill Learning Tools, such as Quill Connect, and the Quill Learning Management System, the platform that enables teachers to provide assignments and view results. (Empirical is the name of the nonprofit building Quill.org).
http://www.quill.org/
GNU Affero General Public License v3.0
25 stars 4 forks source link

Fix weighted accuracy #12512

Closed brendanshean closed 5 days ago

brendanshean commented 5 days ago

WHAT

Fix weighted accuracy calculation on Trial

WHY

The calculation was incorrect. The denominator was test_example_count but should be optimal_count and suboptimal_count

HOW

First calculate optimal_accuracy and suboptimal_accuracy. Then use them instead of recalculating the same values.

Screenshots

(If applicable. Also, please censor any sensitive data)

Notion Card Links

(Please provide links to any relevant Notion card(s) relevant to this PR.)

What have you done to QA this feature?

I tested this on staging.

PR Checklist Your Answer
Have you added and/or updated tests? No.
Have you deployed to Staging? No. Small change.
Self-Review: Have you done an initial self-review of the code below on Github? YES