Udayraj123 / OMRChecker

Evaluate OMR sheets fast and accurately using a scanner 🖨 or your phone 🤳.
MIT License
752 stars 311 forks source link

[Feature] Generate a software accuracy report #171

Open Udayraj123 opened 7 months ago

Udayraj123 commented 7 months ago

Requirements

Software accuracy report

Ideas to start with

Further ideas

This implementation can be extended to create a test benchmark for any existing OMR checking software. We need to blackbox the logic and standardise the inputs and outputs in this case.

Udayraj123 commented 1 month ago

Crosslinking discord thread: https://discord.com/channels/590134763784896514/1270815064256938183/1270816309940326572

Types of accuracy metrics
    - csv-based accuracy benchmark (using moderation portal)
    - Needs to fit in the per-directory approach
    - bubble detection/reading level accuracy
        - confidence metrics
    - preProcessor level accuracies
        - confidence metrics for the matchers
    - aggregate metric
    - speed graphs to identify

- Since we have 4-5 confidence metrics and 2-3 accuracy metrics, it'd be great to find a color spectrum that caters to these (color strips of 3 at a time if needed). Where all-white means most accurate and all-black means least accurate, and the weighted magnitude of the color gives the numeric representation.
- Can also make a Radar chart of this for explanation purposes as it's across different dimensions.