New data model

[ztl8702]

Entry and Tasks

• Entry An Entry is the thing that has Fields which require crowd-sourced workers to work on.

  • Entry's have Metadata which describes the object being worked on, such as links to scanned images and/or coordinates of regions on an image that need work.
  • Entry's have Fields. Fields are revisioned.

• EntryState An Entry can be in different states. EntryState can be a vector composite of several sub-states. For example, (ENG, BUC, CHI) => (ENG=Not Transcribed, ENG-transcribing=2, BUC=Reviewed, CHI=Transcribed). Entry's can be filtered based on their states to create tasks.

• Field Fields are the units of output which crowd-sourced workers are required to produce. It is the missing "blanks" on each Entry which need to be filled in. For example, a scanned image of a dictionary entry might have 3 text Fields for the word, the pronunciation and the explanation respectively.

  • FieldFormatters are applied to each field to sanitise the data before it is stored.

• Task Tasks are the minimal units of work that need to be performed on Entry's. Each Task definition correspond to one user interface. A Task is always associated with an Entry.

  • The effect a Task can have on an Entry can be different, depending on the TaskDefinition. For example, a "Transcribe" Task will add a new revision to the Entry, making changes to the Fields. Alternatively, a "Review" Task might change the state of an Entry to "Reviewed" or to "Needs Rework".

• Tasks can be created from TaskTemplates, which defines the input and output field types of the Task.

Queues and the assignment of Tasks

• Assignment An assignment is a Task being assigned to a worker. A Task can only be assigned once. Task has some inputs, and Assignment has some outputs.
  • Assignment needs to be completed within the time limit of the Task.
  • Or it could be abandoned and another assignment is created later.
  • A Task can be in assignable/non-assignable/assigned/completed states. Only assignable tasks are assigned. Assignable tasks go into different queues.
• Queues are linked lists that contain pending assignable tasks. It is usually "First In First Out", but items can be removed from middle.
• Assignable Tasks can be assigned by pull or by push.
  • "By pull" is when the user proactively requests a task. "By push" is when the system pushes the task to some worker (usually automatic) to complete.
  • "By push" can be implemented using a loop that repeatedly queries and consumes tasks from the Queue.

Pipeline and Events

Pipelines are configurable workflows. A Task can have a few pipelines:

• BeforeTaskPipeline which takes void and returns void. It make changes to the task.
• AfterTaskPipeline which takes AssignmentResult and returns TaskProcessDecision (whether the assignment is accepted or not).
  • TransformMiddleware: configurable middleware that transforms (modifies) the AssignmentResult before it is stored. May call FieldFormatters for each field.
  • ValidateMiddleware: validates the assignment result. If the results violate some rules, the AssignmentResult will be rejected and goes back to the worker. (safeguarding)
  • DiffLastValidateMiddleware: ensures that there are actual changes.
  • CheckEntangledTaskResultsMiddleware
  • ScoreEntangledAssignmentsMiddleware
  • CommitMiddleware

An Entry can also have Pipelines which functions as hooks as certain events happen:

• BeforeEntryCommitPipeline which takes FieldsValue and returns void.
• AfterEntryFieldChange which takes void and returns void. This is where to reassign state.
  • ClassifierMiddleware: (re-assign state) change state of the entry
• AfterEntryStateChange: create new tasks based on the new state.

Credit and Assessment System

How to count credits and determine the quality of workers?

• By tasks completed
• By field edits that go into final revision

Task / assignment has a score. Can be scored manually or by automatic reviews.

Permissions

• Task is only assignable to certain roles (permission on queue)
• Roles:
  • Admin
  • User
  • Team Manager
  • Project Manager

Misc

• FieldFormatters

[ztl8702]

introduce task or entry level locks if some operation needs to be exclusive

[ztl8702]

Entry field storage can be multiple types:

• RevisionedFieldsStorage
• MultipleResultsStorage (for collecting sensing data)