Show caddisfly results data in meaningful way in dashboard

[mtwestra]

The Caddisfly results data is in JSON format, and includes an image in base64 encoding. This needs to be properly handled in the dashboard:

1) when a point is selected on the dashboard map, the image needs to be shown, together with the key / value pairs 2) when a formInstance is selected, the key/value pairs need to be shown, but not editable 3) when data is exported, the image data needs to be excluded.

These need to be taken care of:

• maps tab
• inspect data tab
• monitoring tab
• and import and export of reports.

A good template is the signature question, for which a similar workflow is needed: https://github.com/akvo/akvo-flow/issues/1453

The format of the data is described here: https://github.com/akvo/akvo-caddisfly/issues/60

[mtwestra]

Test plan

1) install the development version of Caddisfly, which contains the strip test code 2) create a free text question in FLOW ending in '- STRIP' 3) using this survey, collect some data with the strip test method (by clicking on the External Sources button) and submit it

An alternative for this process is to use the data in flowaglimmerofhope-hrd instance, survey 1.mark/caddisfly strip test/strip test, which has formId 38534000

4) both the inspect data tab and the monitoring tab should show the strip test data as follows (can be a single image in the case of a 2-patch strip test):

5) the exported data should have the form: {"result":[{"unit":"ppm","name":"Total Chlorine (ppm)","value":7},{"unit":"ppm","name":"Free Chlorine (ppm)","value":0.45}],"name":"Chlorine and Free Chlorine","brand":"HACH-27450-50","code":"strip-CL-HACH-27450","type":"caddisfly-strip"}

So a valid JSON object with the keys "type", "code", 'brand', and 'result', where the result is a JSON array with keys unit, name, and value. The image data is not part of the exported data.

6) when reuploaded, the strip test data should not influence the upload process. (it is quite likely that the current code will fail on this)

[muloem]

I believe this issue was the proof of concept, first pilot trial issue. We used that separately but I believe @mtwestra you have been working on the final solution that includes the caddisfly question type (#1577) So can we close this one? cc/ @janagombitova

[mtwestra]

For the exported data, we need a flat format. The proposal is to have a column for each result, and to put other information in the header. For example, say we have the following result for a chlorine test:

{"result":[{"name":"Total Chlorine (ppm)","value":10,"unit":"ppm","id":0},{"name":"Free Chlorine (ppm)","value":0.5,"unit":"ppm","id":1}],"type":"caddisfly","name":"Chlorine and Free Chlorine","uuid":"bf1c19c0-9788-4e26-999e-1b5c6ca28111","image":"b3893f16-6a02-4e92-a13e-fce25223a0c5.png"}

Say the question was named 'Chlorine test'. In that case, we can export the following columns

Chlorine test - Free Chlorine (ppm) Chlorine test - Total Chlorine (ppm) Chlorine test - Image URL

Which contain the values for the two tests, and for the complete image URL. These columns should be skipped when data is uploaded.

[janagombitova]

Test

Based on https://github.com/akvo/akvo-flow/pull/1723

Test plan

1. Collect data for surveys that hold Caddisfly questions
2. Go to Inspect data tab and cross check that the submission are shown properly (no json strings)
3. Go to Monitoring tab and cross check that the submissions are shown properly.
4. Go to Map tab and cross check that the data is shown properly
5. Go to public map and cross check that the data is shown properly

Confirmation of old implementation

1. Enable external sources on the dashboard
2. Create a survey with the new Caddisfly questions and the old implementation, publish the survey and assign to the phone
3. Collect data
4. In the dashboard in the Inspect data tab, Monitoring tab, both maps check that all data show properly

[janagombitova]

Notes from test round 1

1) size of result image

Caddisfly results show the answers in the expected way. However, I find the size of the image large in relation to the size of the font, especially in the Monitoring tab.

What do you think @loicsans and @mtwestra - would it be ok to make the image smaller?

2) Rendering in Carto maps

In Carto maps the results are rendered as a string. The answers need to be shown in the same way across the entire dashboard. Ideally in the same way they are shown on non-Carto maps.

(rendering on non-Carto maps)

[mtwestra]

@janagombitova, 1) I have no opinion on the size of the image, it can be smaller if you prefer. In that case, only the width of the image should be changed, because the height is variable. 2) Ah, good point. Perhaps you can show me tomorrow - apparently the code that does the carto maps is different from the non-carto maps.

[janagombitova]

@mtwestra 1) smaller - yes please. If you can change it it would be wonderful 2) you can find the data points here https://uat1.akvoflow.org/admin/ > Map tab > 1.9.11 UAT Folder > Jana UAT tests > 1.9.11 UAT Non MF Survey (All questions) 1st test > points are located in Slovakia in the North part of the country

[janagombitova]

Based on the discussion with @mtwestra I have created two separate issues to resolve the points that came out of the testing: 1) Change size of Caddisfly answer image https://github.com/akvo/akvo-flow/issues/1801 2) Redering in Carto maps https://github.com/akvo/akvo-flow/issues/1800

As the only two notes from this issue will be dealt with separately, we can consider the test plan passed 👍

[loicsans]

Just committed a fix (see: https://github.com/akvo/akvo-flow/issues/1801)