Issue Description

When seed analysts use Nachet, they should be able to give their retroaction on the result. A pipeline of action needs to be integrated from the Frontend to the database to be able to register the user feedback. The possible feedbacks types are:

No seed: The user indicates that the seed detected by the model is not a seed (soil peds for example)
Wrong classification: The user indicates that the seed is wrongfully classified by selecting the right seed or guess.
Perfect Inference: The guess of the model was correct and the box hasn't been changed.

Architecture:


---
title: Nachet Architecture for Inference
---
erDiagram
    seed{
        uuid id
        text name
    }
    inference{
        uuid id PK
        json inference 
        uuid picture_id FK
        uuid user_id FK
        timestamp upload_date
    }
    object{
        uuid id PK
        json box_metadata
        uuid inference_id FK
        integer type_id
        boolean verified
        boolean modified
        uuid top_guess FK
        timestamp upload_date
        timestamp updated_at
    }
    seed_object{
        uuid id PK
        uuid seed_id FK 
        uuid object_id FK
        float score
    }

  user ||--o{ inference: requests
  inference ||--|| picture: infers
  inference }o--|| pipeline: creates
  inference ||--o{ object: detects
  object ||--o{ seed_object: is
  seed_object }o--|| seed: is

Work to do

1. Tweak the current classification/inference process in the Backend to use the Datastore.

Sequence of saving the inference:

Note the picture must of already been uploaded and registered in the DB


sequenceDiagram;
  actor User
  box grey Ai-Lab services
  participant Frontend
  participant Backend
  participant Datastore
  participant ML
  end
  box grey Storage services
  participant PostgreSQL Database
  participant Azure Storage
  end

    User ->> Frontend: Classify picture
    Frontend -) Backend: Classify_picture(user_id,picture_id,pipeline_id)
    Backend -) Datastore: get_picture_url(picture_id)
    Datastore ->> Backend : picture_url
    Backend -) ML: inference_request(pipeline,picture)
    ML ->> Backend : inference.json
    Backend -) Datastore: register_inference_result(inference,user_id,picture_id,pipeline_id)
    Datastore ->> Datastore: trim_inference
    Datastore -) PostgreSQL Database: new_inference(trimmed_inference)
    Datastore ->> Datastore: Add {inference_id: uuid}
    loop each box 
        Datastore ->> Datastore: build_box_metadata(box_metadata)
        Datastore ->> PostgreSQL: new_inference_object(box_metadata)
        Datastore ->> Datastore: Add {box_id: uuid}
        loop each guess
            Datastore -) PostgreSQL: get_seed_id(seed_name)
            Datastore ->> PostgreSQL: new_seed_object(box_id,seed_id)
            Datastore ->> Datastore: Add {object_id: uuid}
        end
        Datastore  ->> PostgreSQL: set_inference_object_top_id(object_id, top_seed_object_id)
    end
  Datastore ->> Backend: inference_with_id.json
  Backend ->> Frontend: Display picture with inference results

2. Implement a process to enable users to submit their inference feedback/validation

Sequence of saving the inference feedback


sequenceDiagram;
  actor User
  participant Frontend
  participant Backend
  participant Datastore
  participant Database

    User ->> Frontend: Validate inference
    alt Perfect Inference
    Frontend -) Backend: Inference result positive (user_id,inference_id)
    Backend -) Datastore: Inference result positive (user_id,inference_id)
    Datastore ->> Database: Set each object.verified = True & object.modified=False
    else Annotated Inference
    Frontend -) Backend: Inference feedback (inference_feedback.json,user_id,inference_id)
    Backend ->> Datastore: Inference feedback (inference_feedback.json, user_id, inference_id)
    Datastore -> Database: Get Inference_result(inference_id)
        loop each Boxes
            alt box has an id value
                alt inference_feedback.box.verified= False
                    Datastore --> Datastore: Next box & flag_all_box_verified=False
                else
                    Datastore -) Database: Set object.verified=True & object.verified_by=user_id
                    Datastore -) Datastore: Compare label & box coordinate
                    alt label value empty
                        Datastore -) Database: Set object.top_inference=Null
                        Datastore -) Database: Set object.modified=False                   
                    else label or box coordinate are changed & not empty
                        Datastore -) Database: Update object.top_inference & object.box_metadata
                        Note over Datastore,Database: if the top label is not part of the seed_object guesses, <br>we will need to create a new instance of seed_object.
                        Datastore -) Database: Set object.modified=true
                    else label and box haven't changed
                        Datastore -) Database: Set object.modified=False
                    end
                end
            else box has no id value
                Datastore -) Database: Create new object and seed_object
            end
        end
        alt if flag_all_box_verified=True
            Datastore -) Database: Set Inference.verified=true
        end
    end

Acceptance Criteria

Users can annotate their retroaction on the model results
Retroaction is saved into the database and helps data scientists with the model training
Data from retroaction is recorded and saved in the database
An endpoint in the backend is set up and allows the transit of the data from the frontend to the database and the model

Tasks

Frontend:

[x] Implement the ability for users to give their retroaction in the frontend #129
- [x] Implement the ability for users to give their retroaction in the frontend
- Related to https://github.com/ai-cfia/nachet-frontend/issues/40
- [x] Provide positive feedback to accept an inference as good data
- [x] ~~Provide negative feedback to reject an inference box completely~~
- [x] Provide negative feedback with a correction to the inference box position and dimensions
- [x] Provide negative feedback with a correction to the species label (species in training set)
- [x] Provide negative feedback with a correction to the species label (species not in training set)
- [x] Provide negative feedback no seed in the inference box
- [x] Provide negative feedback seed not detected
- [x] Add API calls for uuid and seed list
- [x] New Component Tests
- [x] Update Documentation

Backend:

[x] #87
[x] https://github.com/ai-cfia/nachet-backend/issues/88

Database:

[ ] Create inference structure for the Classification and feedback #23

ai-cfia / nachet-backend

Analyst Feedback on Model Result #68