Adjust 1-get-cases job to only get for updates today

[taylordowns2000]

@lakhassane , do you know how to filter getCases in primero to only return those cases that were updated_at some time today()?

The next job in the flow is idempotent, so we don't actually need to keep a trigger, but we'd love your eyes on this bit here and some clarification on how the Primero v2 api works.

Thank you!

(CC: @elias-ba )

[lakhassane]

@taylordowns2000 we should be able to have a manualCursor. Something like:

// set up version control first!
const manualCursor = new Date().toISOString();
// const anotherTypeofCursorValue = "2021-09-24T00:00:00.000Z";
getCases({
  remote: true,
  // TODO: make sure date filter grabs anything updated TODAY!
  last_updated_at: manualCursor,
});

Since it's always TODAY() we want this above can do it. Otherwise we can have another cursor using the date of previous run (fetch all cases since previous run)

[taylordowns2000]

thanks, @lakhassane . @elias-ba , try inlining this to keep it short and clear... if we already had another operation then the manualCursor idea would be great. but since we don't, try out:

getCases({
  remote: true,
  last_updated_at: new Date().toISOString(),
});

[elias-ba]

Okay @taylordowns2000.

[taylordowns2000]

@lakhassane , I think this is something that's always confused me about Primero though... if I were a Javascript developer and I read that expression, I'd assume that it would only return cases who's last_updated_at date matched the output of new Date().toISOString(). Would I be wrong? We want all cases updated at any time today. Is that what the above code gives us from the Primero API? (And do you have a link to the Primero API docs?)

[lakhassane]

Here is the doc in case you need it but I doubt you find anything that you don't already know https://github.com/primeroIMS/primero/blob/development_v2/app/controllers/api/v2/docs/cases/get.md

[taylordowns2000]

@elias-ba , per @lakhassane 's slack suggestion please try out:

last_updated_at: `${new Date(new Date().setHours(0, 0, 0, 0))}..`

[aleksa-krolls]

@elias-ba @taylordowns2000 here is an example primero job where we have successfully filtered by date before: https://github.com/OpenFn/primero-progres/blob/master/jobs/2.a.getCases.js#L9

[elias-ba]

Thanks @aleksa-krolls and @taylordowns2000 this is helpful.

[aleksa-krolls]

@taylordowns2000 @elias-ba See my comments in the job for what date filters seem to work and which don't: https://github.com/OpenFn/primero-demo/blob/6165a9f13d636d2744cb2fba7bbc5e40bbac19b6/jobs/1-get-cases.js#L1-L4

I think the timezone of OpenFn runtime is not aligned with the Primero instance... so we end up querying for cases with a "future" date. If we filter the query by today's date (with no time T00:00:00.000Z) then it should work, no issue.

[aleksa-krolls]

@elias-ba we can discuss in our standup tomorrow... but please hold on making any changes. For the demo, I'm happy with the filters I've added for now... https://www.openfn.org/projects/p5ypvq/jobs/jy365a