OpenFn / primero-progres

Interoperability solution for UNICEF <> UNHCR interagency referrals between Primero and Progres v4 systems.
https://openfn.github.io/primero-progres/
0 stars 2 forks source link

New date filter for `getCases(...)` jobs #17

Closed aleksa-krolls closed 3 years ago

aleksa-krolls commented 3 years ago

Background, context, and business value

See issue #14 . If we filter by a specific datetime, this will only extract cases with the same exact datetime stamp.

The specific request, in as few words as possible

We want to extract cases greater than the datetime of the last fetch. This can be done with the Primero API by querying... GET /api/v2/cases?last_updated_at=2021-06-02T20:00:00.587Z.. (where we add .. after the datetime stamp)

Can you please add this .. and update the job so that the datetime stamp will be set to the datetime of the last job run OR a default manual cursor date?

last_updated_at: `${lastRunDateTime}..` || '2021-06-01T00:00:00.000Z..'

See example pseudo code above where lastRunDateTime is updated every time the job runs, and 2021-06-01T00:00:00.000Z is the manual cursor for when state is cleared.

state.json

See LP for Primero Dadaab creds

expression.js

Please implement on both get Primero jobs... https://github.com/OpenFn/primero-progres/blob/master/jobs/2.a.getCases.js https://github.com/OpenFn/primero-progres/blob/master/jobs/1.b.sendReferrals.js

output

Only cases updated after the datetime stamp we query by will be fetched.

aleksa-krolls commented 3 years ago

@lakhassane fyi, I was just on a testing call with unhcr and made some edits to these job for that testing... please feel free to overwrite my edits with any changes you make.

lakhassane commented 3 years ago

noted @aleksa-krolls I made the corrections for both.