Progress report per annotator (doc name, annotator id, date of finishing)

[khyurri]

General algorithm

User will be able to get report by period. Report must contain:

By annotator:

• Finished tasks + AS + (time_start, time_finish to calculate how much time annotator worked in single document)
• Tasks in progress + job start time

Example:

annotator_id, task_id, task_status, time_start (earliest event), time_finish (draft or task_finish), agreement score (if task finished and job with extensive coverage and AGREEMENT_SCORE_SERVICE_URL was set)

Front-end:

• [x] send event (post) when document is open by annotator or validator during task
• [x] add page to download result by user and date_from, date_to

Back-end:

• [x] add endpoint to save document open timestamp, we need to create new table: annotation_statistics with fields: job_id, task_id, event_type, event_date (current timestamp), additional_data (json). event_type must be: document_opened
• [x] add new config value (annotation service): AGREEMENT_SCORE_SERVICE_URL. In case of extensive coverage task, when validator finishes validation task, create additional POST request to AGREEMENT_SCORE_SERVICE_URL (if set) to get agreement score. BadgerDoc must send: annotators_id, job_id, task_id, document url (S3 resource), manifests urls (S3 resource) for each annotator
• [x] Store all metrics between annotator and validator. Each annotator has own AS for each task over validator.

[MagicTearsAsunder]

AGREEMENT_SCORE_SERVICE_URL: POST REQUEST Body

[
    {
        "annotator_id": "f0474853-f733-41c0-b897-90b788b822e3",
        "job_id": 1,
        "task_id": 1,
        "s3_file_path": "file1.pdf",
        "s3_file_bucket": "s3://bucket",
        "manifest_url": "url1",
    },
    {
        "annotator_id": "4e9c5839-f63b-49c8-b918-614b87813e53",
        "job_id": 1,
        "task_id": 1,
        "s3_file_path": "file2.pdf",
        "s3_file_bucket": "s3://bucket",
        "manifest_url": "url2",
    }
]

EXPECTED RESPONSE: Body

[
    {
        "annotator_id": "f0474853-f733-41c0-b897-90b788b822e3",
        "job_id": 1,
        "task_id": 1,
        "agreement_score": {"attr1": "value1"}
    },
    {
        "annotator_id": "4e9c5839-f63b-49c8-b918-614b87813e53",
        "job_id": 1,
        "task_id": 1,
        "agreement_score": {"attr1": "value1"}
    },
]

[Nathicanaa]

Expected response of agreement score service from my point @khyurri @MagicTearsAsunder

[
  {
    "annotator_id": "5a33e3b104ed42bd8b947dbb4e9b1ac6",
    "job_id": 1,
    "task_id": 1,
    "agreement_score": {
      "scores": [
        {
          "task_id": 2,
          "agreement_score": 0.99
        },
        {
          "task_id": 3,
          "agreement_score": 0.81
        },
        {
          "task_id": 4,
          "agreement_score": 0.85
        }
      ]
    }
  },
  {
    "annotator_id": "9c82f7592b804a44aed4911ec7f5362a",
    "job_id": 1,
    "task_id": 2,
    "agreement_score": {
      "scores": [
        {
          "task_id": 1,
          "agreement_score": 0.99
        },
        {
          "task_id": 4,
          "agreement_score": 0.89
        },
        {
          "task_id": 3,
          "agreement_score": 0.86
        }
      ]
    }
  },
  {
    "annotator_id": "7ed28729a90a4117887f5011bb050a2b",
    "job_id": 1,
    "task_id": 3,
    "agreement_score": {
      "scores": [
        {
          "task_id": 4,
          "agreement_score": 0.92
        },
        {
          "task_id": 2,
          "agreement_score": 0.86
        },
        {
          "task_id": 1,
          "agreement_score": 0.81
        }
      ]
    }
  },
  {
    "annotator_id": "e292ef3b80f44c49930fccde9170dd79",
    "job_id": 1,
    "task_id": 4,
    "agreement_score": {
      "scores": [
        {
          "task_id": 1,
          "agreement_score": 0.85
        },
        {
          "task_id": 2,
          "agreement_score": 0.89
        },
        {
          "task_id": 3,
          "agreement_score": 0.92
        }
      ]
    }
  }
]