Update f2-2.sendReferralToDTP.js to output the full run log for errors received from DTP

[daissatou2]

Describe the bug

The run logs for f2-2.sendReferralToDTP.js are not printing out the full error message when the first GET request to the Primero /users endpoint fails.

When Primero threw a 500 error response, we found that the full error message is being truncated in the OpenFn run logs. See L312 in the log - it's showing data: { errors: [Array] }, instead of printing what is in the errors: [Array] (which contains detailed error messages).

Can you see if it's possible to print the full error message? Does this need to be an HTTP adaptor change?

To Reproduce

1. Here is a link to a failed run on OpenFn.org which is indicative of the bug: See line 312 of this log.

expression.js

Link to the job itself in Github: https://github.com/OpenFn/primero-progres/blob/master/jobs/PROD/f2-2.sendReferralToDTP.js

🚨 Please make changes on a separate branch and do not commit to master until approved. 🚨

state.json

f2-j1

{
  "configuration": ["SEE LAST PASS: 'Primero Gambella Demo - Get Referrals (open_function user)'"]
}

f2-j2

{
  "configuration": ["SEE LAST PASS: 'DTP UAT - ReceiveIncomingReferral (open_function)'"],
  "data": {"data is generated by f2-j1 job"}
}

To test/resolve

After the desired output is working locally while connected with the DEV environment (from the CLI), please [push commits to a NEW BRANCH || open a pull request].

[daissatou2]

@aleksa-krolls please confirm these are the correct staging credentials.

[aleksa-krolls]

@daissatou2 it turns out this 500 error was actually thrown by Primero (not DTP) when we send this first GET request to the /users endpoint to query the caseworker's contact info. See related step: https://github.com/OpenFn/primero-progres/blob/master/jobs/PROD/f2-2.sendReferralToDTP.js#L10-L24

In tests, we're unable to replicate the 500 error again. I added some initial logs to help with troubleshooting and make it clear when the error is from Primero vs. Progresv4, and @mtuchi @taylordowns2000 will look into why this errors [Array] wasn't fully printed.

[aleksa-krolls]

@taylordowns2000 Immediate fix pushed to help with logging for enhanced troubleshooting, but can we please discuss this and whether its an adaptor issue?

[aleksa-krolls]

@taylordowns2000 can you revisit this later this week or next so that we can determine whether to take action or close this issue? Thanks!

[taylordowns2000]

@aleksa-krolls , apologies for the delay here. The proper thing to do here would be to handle the response before that error is thrown. The adaptor allows this, using an array of transformResponse functions. See implementation below and a link to the job library.

get(
  'https://jsonplaceholder.typicode.com/wrongUrl', // the url
  {
    transformResponse: [
      data => {
        console.log("The data from the response, BEFORE the error is thrown");
        console.log(JSON.stringify(data, null, 2))
        return data;
      }
    ],
  }, // no additional options
  state => { // the callback
    console.log('the data', state.data)
    return state; 
  }
);