Add maker-checker feature

[rogerhutchings]

Adds in a maker-checker feature to PFE.

How it works

Annotations should be added to a subject's metadata - if present, they'll be rendered as the worker moves between tasks.

Example subject (note the metadata.classification property)

{
    "id": "66463188",
    "metadata": {
        "Filename": "1.jpeg",
        "classification": {
            "annotations": [
                {
                    "task": "T0",
                    "value": 1
                },
                {
                    "task": "T1",
                    "value": [
                        {
                            "x": 342.68145751953125,
                            "y": 140.88299560546875,
                            "tool": 0,
                            "frame": 0,
                            "details": []
                        },
                        {
                            "x": 271.272705078125,
                            "y": 203.1796875,
                            "tool": 0,
                            "frame": 0,
                            "details": []
                        },
                        {
                            "x": 387.2652893066406,
                            "y": 268.668212890625,
                            "tool": 0,
                            "frame": 0,
                            "details": []
                        }
                    ]
                }
            ]
        }
    },
    "locations": [
        {
            "image/jpeg": "https://panoptes-uploads.zooniverse.org/subject_location/8b1fd886-e7ef-4c6a-850a-56c27d0289df.jpeg"
        }
    ],
    "zooniverse_id": null,
    "created_at": "2021-08-16T14:46:51.382Z",
    "updated_at": "2021-08-16T14:54:26.038Z",
    "href": "/subjects/66463188",
    "selected_at": "2021-09-03T12:42:05.376Z",
    "retired": false,
    "already_seen": false,
    "finished_workflow": false,
    "user_has_finished_workflow": false,
    "favorite": false,
    "selection_state": "normal"
}

If a subject contains existing annotations for a task on a workflow, they'll be added to the annotations state.

Currently works for

• Points
• Questions

Other tool types may require adding additional code, since we're trying to mimic the output from the actual drawing task in order to make this work.

To do

• Check it works as expected when moving between tasks (I think it does, but we should check with more complex workflows as required)

[vercel[bot]]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/1715labs/1715labs-pfe/Bk8yANQVigLnGxKxUxzBzjvY5psh
✅ Preview: https://1715labs-pfe-git-maker-checker-1715labs.vercel.app

[tingard]

First thoughts:

• How does a classification behave on the "wrong" workflow (i.e. annotations that don't match the current workflow)?
• Code does not fail gracefully if "classification" metadata is not found (Cannot read property 'annotations' of undefined)

[tingard]

I've also noticed that "persist annotations" is now ignored? i.e. I would imagine that with "persist annotations" enabled then going back to a previous task then forwards again would preserve the changes I've made to the default classification, whereas going backwards + forwards without "persist annotations" would reset the values back to the default for that task?

Edit: Also - I've checked and persist annotations ignores any changes to the value of a single-choice question, which is counter-intuitive (I'd have imagined that it would persist the choice made, rather than revert to the default)

[tingard]

The only breaking issue I've spotted is that we need to handle the case where "classification" is not present in the subject metadata, and so no default should be specified".

[tingard]

It also doesn't yet support "allow multiple" in categorisation tasks, which we do sometimes use

[rogerhutchings]

persist annotations

I haven't tested this scenario, but I've intentionally kept the code dumb, so I suspect it would try and render a mark with the wrong tool and fall over. To be honest, I think it should error out to prevent wasted work

Missing classification property

This is a bug, I'll put a fix in. Thanks for spotting it!

allow multiple

That shouldn't be too hard to add; if I had to guess, value for this task changes type (quiet facepalm) to array, so we just need to handle that.

[tingard]

That shouldn't be too hard to add; if I had to guess, value for this task changes type (quiet facepalm) to array, so we just need to handle that.

That's exactly it - you get an array of selected answer indices