Repeated mappings in Pre and Post jobs - return whichever value is present

aleksa-krolls commented 3 years ago

Background, context, and business value

See issue #1 for original specs and mapping specs.

Request

Apologies we did not catch this before in the mapping specs... but we are seeing the same Salesforce destination field referenced multiple times in 1 job, which is not allowed.

Depending on how the CommCare user completes the form, the data value for a SF field might come from 1 of 3 different source question ( if the user is in Nigeria, they will be asked a different question than a Zambia user... but the data should map to the same destination field in SF). Therefore please implement logic to return whichever data value is defined in CommCare.

Example: search the job upsertPreChallenges.js to see how Pre_1__c is referenced multiple times. Please update the job mapping logic so that it checks all 3 questions....

Pre_1__c: dataValue( 
'form.pre_challenge_nigeria_-_advanced_skills.i_have_talked_about_hiv_with_an_adult_in_the_past_two_months_outside_skillz' ||
'form.pre_challenge_zambia_-_skillz_core_zambia.i_know_what_i_am_good_at' ||
'form.pre_challenge_zambia_-_plus_mh_enhanced_zambia.in_general_i_feel_good_about_myself')

expression

Pre Challenges: https://github.com/OpenFn/grassroot-soccer/blob/master/jobs/upsertPreChallenges.js
- See repeated fields like Pre_1__c, Pre_2__c`, etc.
Post: https://github.com/OpenFn/grassroot-soccer/blob/master/jobs/upsertPostChallenges.js
- See repeated fields like Post_1__c, Post_2__c`, etc.
CSV: https://github.com/OpenFn/grassroot-soccer/blob/master/jobs/2.e.upsertCoach.js
- See commented out fields that are repeated (e.g., 'Accurate_Information__c')

state

Expected behavior

Please ensure Salesforce destination fields (e.g., Pre_1__c) are only reference one time in the jobs. Current mapping example:


Pre_1__c: value1, 
Pre_1__c: value2

Requested mapping example: 
`Pre_1__c: value1 || value2,`

## Testing notes
If you see an error message from Salesforce that says something like `JSON_PARSER_ERROR: Cannot deserialize instance of double`... we probably still have duplicate references in the job. 

If you see other errors, that might be additional mapping issues we need to look at... so keep us posted.

aleksa-krolls commented 3 years ago

@chaiwa-berian please start work and let @daissatou2 if you have any mapping questions.

aleksa-krolls commented 3 years ago

@chaiwa-berian In response to your slack comments (https://openfn.slack.com/archives/G2LJCFFDW/p1620733998017300)...

It's okay if all the fields are not present in state. We expect that sometimes fields will not be present if the user has not been prompted to answer that question in CommCare... therefore some fields might return undefined until an answer in CommCare is provided.... so please rely on the mapping specification for how the paths to the fields should look.

Can you let me and @daissatou2 know if these changes are done (assuming we know you won't be able to test every mapping)?

chaiwa-berian commented 3 years ago

@aleksa-krolls Yes this has been addressed, @daissatou2 will test this and recommend if anything else. Thanks.

daissatou2 commented 3 years ago

@chaiwa-berian We are now seeing a new error with the Pre Challenges and the Post Challenges forms. The mappings for the pre and post questions on the forms are not matching with the answer choices in Salesforce. The transformations required are in the mapping sheet. in green. Please let me know if you have any questions.

Here’s a link to the latest run.

chaiwa-berian commented 3 years ago

@daissatou2 I have written the transform function as below but there are other choices that are still un-mapped:

 function transform(value) {
    switch (value) {
      case 'true':
        return 1;
      case 'false':
        return 2;
      case 'i_dont_know':
        return 3;
      case 'unanswered':
        return 4;
      case 'not_applicable':
        return 5;
      default:
        return value;
    }
  }

Here is the output:

  Pre_1__c: 3,
  Pre_2__c: 'yes',
  Pre_3__c: 'yes',
  Pre_4__c: 'agree',
  Pre_5__c: 'agree',
  Pre_6__c: 'yes',
  Pre_7__c: 'no',
  Pre_8__c: 'yes',
  Pre_9__c: 'yes',
  Pre_10__c: 1,
  Pre_11__c: 'agree',
  Pre_12__c: 2,
  Pre_13__c: 'no',
  Pre_14__c: 'agree',
  Pre_15__c: 2,
  Pre_16__c: 3,
  Pre_17__c: undefined,
  Pre_18__c: undefined,
  Pre_19__c: undefined,
  Pre_20__c: undefined

If answer is yes, no, or agree. What should we do?

chaiwa-berian commented 3 years ago

@aleksa-krolls , @daissatou2 and I cleaned-up the two jobs and ready for review!

OpenFn / grassroot-soccer