Update job to handle scenarios when `no` consent provided in Kobo

aleksa-krolls commented 3 years ago

Background

Every hour, OpenFn syncs "rural consumption" Kobo forms to a structured database configured on Postgres for the Wildmeat initiative. See the repo's README for more info.

Change request

Update the job so that if "consent_checklist": "no" in Kobo... only create 1 row in the DB table swm_transaction. This is to record that the survey was created... but we don't want to create data in the other tables, because no other information was provided.

https://github.com/OpenFn/wcs-wildmeat/blob/5477d6bc31b6a97783cae31295dfe5649432c655/jobs/rural-consumption-to-postgres.js#L116-L128

expression.js

jobs/rural-consumption-to-postgres.js See here: https://openfn.org/projects/1767/jobs/3603

state.json

Kobo: See LP for openfn_kobo DB: See LP for WCS Wildmeat Transitional

Test with the following Message states:

This Message is where "consent_checklist": "no", so ONLY a swm_transaction row should be inserted: https://openfn.org/projects/1767/messages/10197240
This Message contains lots of data like in a typical survey, so rows should be inserted across the different DB tables we are mapping to incl. tbl_household: https://openfn.org/projects/1767/messages/9824945

output

Both versions of state should run successfully to upsert Kobo data in the DB.

Configuration on production only contains test data, it is safe to test and re-run the failure. If any questions, ping @aleksa-krolls and @chaiwa-berian.

aleksa-krolls commented 3 years ago

@chaiwa-berian Is this something you can try implementing? cc @taylordowns2000

aleksa-krolls commented 3 years ago

@chaiwa-berian For the below instances mapping... if "consent_checklist": "no", can we return the string {"uuid", ${state.data.body._uuid}, "consent": "no"} (..instead of state.data)? https://github.com/OpenFn/wcs-wildmeat/blob/e2daa91a226b0ec4c5b58b96ff181fbd89ba786a/jobs/rural-consumption-to-postgres.js#L138

So for this message (https://openfn.org/projects/1767/messages/10197240), this should return... instances: '{"uuid": "f3f6ca03-b14f-4d8a-a0b3-a84d26d73e65", "consent": "no"}'

For all other surveys, continue with the existing mapping... instances: JSON.stringify(state.data),

aleksa-krolls commented 3 years ago

@chaiwa-berian were you able to test on your side last night? Please move to Review/QA if ready for me :)

aleksa-krolls commented 3 years ago

@chaiwa-berian I just ran this message (https://openfn.org/projects/1767/messages/10213111) and it fails with this error: https://openfn.org/projects/1767/runs/10597551

OpenFn / wcs-wildmeat