[DESIGN] Finalise optimal JSON structure for answers payload

[JBurkinshaw]

Determine an optimal JSON structure for sending answers payloads to the API.

Describe the solution you'd like <More details to follow, here or in comments>

Describe alternatives you've considered <A clear and concise description of any alternative solutions or features you've considered.>

Additional context

[ghelobytes]

This is the proposed payload for the answers endpoint: https://viewer.diagrams.net/?page-id=z5HBLktNAcPlfK06zZkv&page-id=z5HBLktNAcPlfK06zZkv&target=blank&highlight=0000ff&nav=1&hide-pages=1#G17NrR06_ZNPUmfwvVBoKOmFXpj7ThOhDm

cc: @JBurkinshaw @julia-rom

[JBurkinshaw]

A few thoughts on this:

1. The API route contains form type (e.g. .../registration_answers, .../intervention_answers, .../monitoring_sequence). I don't see a reason why question_form_type needs to be included in the request body because of this.
2. Maybe we only need question_id and answer_value within an answer object (possibly answer_other as well but I am struggling to remember its purpose). Does the API need to be aware of the other property values?
   • Can the UI be responsible for ensuring the answers in the payload are for the questions related to a specific form section?
   • Does the UI need to tell the API what the question_parent_id and question_type are or can the API be trusted to know that based on what the question_id is?
3. I might be wrong on this, but would it make sense to use PUT to submit all answers and POST only to create a new monitoring sequence? Other than creating a new monitoring sequence, does POST do something PUT cannot? From these docs:

   The HTTP PUT request method creates a new resource or replaces a representation of the target resource with the request payload. The difference between PUT and POST is that PUT is idempotent: calling it once or several times successively has the same effect (that is no side effect), whereas successive identical POST requests may have additional effects, akin to placing an order several times.

[ghelobytes]

1. That's a fair point. I forgot to remove question_form_type when I split the forms into their own tables.
2. Details:
   • Purpose of the fields:

     question_form_section: Will be removed,
     question_id: Stores the reference to Question ID
     question_parent_id: Stores the reference to the parent Question. Can be useful for validation.
     question_type: Can be any of string, float, integer, or array
     answer_value: Stores the actual answer to the question. Multiple and single choice is stored as array. 
     answer_other: If the answer_value happens to be a choice and one of the choice is "Other"

   • If we agree backend will be the source of truth as far as the definition of the questions is concerned, then we can reduce the fields into:

     question_id: Stores the reference to Question ID
     answer_value: Stores the actual answer to the question. Multiple and single choice is stored as array. 
     answer_other: If the answer_value happens to be a choice and one of the choice is "Other"

3. Let's do that (use PUT). This is one of the reason I feel this isn't really restful as it doesn't really fit into the usual pattern of HTTP verbs. The supposed attributes are actually represented as rows (instead of columns).

[ghelobytes]

If we agree on #2, please note that I will still store the values this way:

[julia-rom]

just catching up on this thread...

@ghelobytes for #2 ... you have

question_id: Stores the reference to Question ID answer_value: Stores the actual answer to the question. Multiple and single choice is stored as array. answer_other: If the answer_value happens to be a choice and one of the choice is "Other"

In that case, would I just be sending an array of answer objects with those three properties for each form section? I'm also struggling to understand why we need the 'other' part. Do you know if we have this kind of question?

[julia-rom]

isn't the second form type just called monitoring? Was intervention_answers specified as a name? Also agree that we don't really need the form section name, as we can infer it from the question number/id

[ghelobytes]

@julia-rom From our discussion with @JBurkinshaw , we can eliminate answer_other and instead pass the selection/choices like this:

{
    "question_id: "Q2.1",
    "answer_value": [
      {
        "choice": "Visitors/Ecotourist",
        "value": "name of org"
      },
      {
        "choice": "Other",
        "value": "name of other"
      }
    ]
}

We were also thinking if multiple choice doesn't require additional answers, we can just pass answers like this:

{
    "question_id: "Q1.3",
    "answer_value": ["Malaysia", "Indonesia", "Thailand"]
}

[ghelobytes]

isn't the second form type just called monitoring? Was intervention_answers specified as a name? Also agree that we don't really need the form section name, as we can infer it from the question number/id

The forms are Registration, Intervention, and Monitoring. Registration and Intervention maps to 1 set of answers only. Monitoring is recurring so there will be multiple set of answers.

[julia-rom]

maybe we can discuss a bit more in todays meet. This would require extra logic/complexity on the front end to always check the type of question, and whether it includes other. At the moment react hook form just preps the data as {questionLabel: answerValue, questionLabel2: answerValue2 ....}

I feel like we are splitting the question form logic between f/e and b/e at the moment

[julia-rom]

is there a route I can test out with the answers payload (basic structure for now) @ghelobytes ?

[ghelobytes]

@JBurkinshaw @julia-rom You can download and extract the 2 files in the archive here and import in either Thunder Client or Postman.

Link: request_example.zip

[ghelobytes]

In gist:

curl --location --request PUT 'https://mrtt-api-test.herokuapp.com/api/v2/sites/1/registration_answers' \
--header 'Content-Type: application/json' \
--data-raw '[
  {
    "question_id": "Q1.1",
    "answer_value": "Bakawan Restoration Project"
  },
  {
    "question_id": "Q1.5",
    "answer_value": {
      "area_km": 10,
      "geojson": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              30.0,
              10.0
            ],
            [
              40.0,
              40.0
            ],
            [
              20.0,
              40.0
            ],
            [
              10.0,
              20.0
            ],
            [
              30.0,
              10.0
            ]
          ]
        ]
      }
    }
  },
  {
    "question_id": "Q2.5",
    "answer_value": [
      {
        "choice": "Full protection"
      },
      {
        "choice": "No management"
      },
      {
        "choice": "Other",
        "value": "Community protected"
      }
    ]
  }
]'

[julia-rom]

I'm a little confused at how to implement this with axios. Wouldn't the first submit be a POST? Would I use 'https://mrtt-api-test.herokuapp.com/api/v2/sites/1/registration_answers' in the axios call? @ghelobtyes

[JBurkinshaw]

We decided early this week that PUT would be suitable as it can be used to create or update records. One example (there are many more and I haven't followed this one): https://stackabuse.com/how-to-make-put-http-request-with-axios/

[julia-rom]

@ghelobytes for some reason my tag wasn't working in the previous comment - could you confirm whether https://mrtt-api-test.herokuapp.com/api/v2/sites/1/registration_answers is the correct route to use when testing the PUT call?

[ghelobytes]

@julia-rom This is an example on how you would use axios to send the answers:

<html>
  <!-- python -m http.server 9000 -->
  <head>
    <script src="https://unpkg.com/axios/dist/axios.min.js"></script>
    <script>
      function send_answers() {
        var url =
          "https://mrtt-api-test.herokuapp.com/api/v2/sites/1/registration_answers";
        var payload = [
          {
            question_id: "Q1.1",
            answer_value: "Bakawan Restoration Project",
          },
          {
            question_id: "Q1.5",
            answer_value: {
              area_km: 10,
              geojson: {
                type: "Polygon",
                coordinates: [
                  [
                    [30.0, 10.0],
                    [40.0, 40.0],
                    [20.0, 40.0],
                    [10.0, 20.0],
                    [30.0, 10.0],
                  ],
                ],
              },
            },
          },
          {
            question_id: "Q2.5",
            answer_value: [
              {
                choice: "Full protection",
              },
              {
                choice: "No management",
              },
              {
                choice: "Other",
                value: "Community protected",
              },
            ],
          },
        ];

        axios.put(url, payload).then(function (response) {
          console.log(response.data);
          console.log(response.status);
        });
      }
    </script>
  </head>
  <body>
    <button type="button" onclick="send_answers()">Send answers</button>
  </body>
</html>

cc: @JBurkinshaw

[julia-rom]

getting a 200 👍

[julia-rom]

[julia-rom]

it looks like items 0-3 are being added onto the response. Is this seed data that you used in the api @ghelobytes? I'm only adding 3 - 7. I also have to update the question ids from the descriptive titles

[ghelobytes]

Yes. Q1.5, Q1.1, Q2.5 were example questions.

[ghelobytes]

it looks like items 0-3 are being added onto the response. Is this seed data that you used in the api @ghelobytes? I'm only adding 3 - 7. I also have to update the question ids from the descriptive titles

@julia-rom

This made me realize the current behaviour should be a PATCH (partial updates). I'll reimplement PUT so that the records will stay true to the payload (i.e. if you remove an answer to the question in the payload, that get's delete in the DB too).

[julia-rom]

ok, I think that's ok for now since we haven't implemented editing yet

[ghelobytes]

In general, answers payload will take the form of:

[
    {
        "question_id": "Q1",
        "answer_value": [ "CHOICE A", "CHOICE B"]
    },
    {
        "question_id": "Q2",
        "answer_value": [ "CHOICE A", "CHOICE B"]
    },
    ...
]

Where answer_value will be validated against a jsonschema defined for the question_id.

Related ticket: https://github.com/globalmangrovewatch/gmw-api/issues/40

[ghelobytes]

Example Postman and Thunder client requests: mrtt_requests.zip