[Feature] Generate Template Layout directly from a reference OMR image

[Udayraj123]

Right now, we need to follow a tedious method of creating template.json files. There are two ways to simplify this for a start.

1. Create a drag and drop type interface that let's us make a template overlay and export it as a json file.
2. Input a single fully-bubbled omr sheet to the code, it generates the bubble coordinates by itself.(scope of this task)

Check on this Discord message for further details .

[05Alston]

Hey, is someone working on this issue? If not, I'd like to try it out. Some question, what language is this frontend supposed to be? a web based HTML/React or something like PyQt? Or is the issue only talking about creating a logic for creating templates?

[Udayraj123]

@05Alston This is quite open as it can be an optional/separate application module. We'll be ultimately moving the whole OMRChecker experience into the browser (https://github.com/Udayraj123/OMRChecker/issues/41), so yeah React is preferred. But it can be in any language of your choice as final deployed version can be used by the users.

The issue is for generating final template.json file (may choose to take user's modifications before generating final json)

[Udayraj123]

Adding a related link:

https://www.cse.iitd.ac.in/~aseth/udai/OMRProj/README.html

[Udayraj123]

Adding a resource library that can make snapping to a grid easy:

https://shapely.readthedocs.io/en/stable/manual.html#snapping

[Udayraj123]

Since grouping of bubbles will need user intervention, this method is not making the process any simpler. Will recommend https://github.com/Udayraj123/OMRChecker/issues/126 as an alternative.

[Udayraj123]

Reopening by changing the scope of the problem to allow people to try out ML/AI methods for generating template.json/ bubble positions in the template.json using segmentation or custom object detection methods like YOLO etc.

[Udayraj123]

Adding a screenshot here(from a discord user) that attempts a segmentation approach

[Udayraj123]

LLMs are getting good at outputting jsons with adherence to a schema structure. Useful for generating jsons from user inputs

https://platform.openai.com/docs/guides/structured-outputs/supported-schemas

Started discord thread here: https://discord.com/channels/590134763784896514/1270873010529763450/1270873016774955028