search

medic / cht-core

The CHT Core Framework makes it faster to build responsive, offline-first digital health apps that equip health workers to provide better care in their communities. It is a central resource of the Community Health Toolkit.
https://communityhealthtoolkit.org
GNU Affero General Public License v3.0
438 stars 209 forks source link

Improve DHIS2 automation #7469

Open mrjones-plip opened 2 years ago

mrjones-plip commented 2 years ago

Is your feature request related to a problem? Please describe. It is currently a manual process to get DHIS2 integration to work on a regular basis. It looks something like this:

  1. Hit a URL in the CHT to download the data.
  2. Log into DHIS2 in your browser.
  3. Upload the data from step 1 into DHIS2 via an import screen.

Describe the solution you'd like Automate the above steps so no humans need to be involved for reliable import into DHIS2.

Describe alternatives you've considered The is a "no dev" solution that exists using Apache NiFi, but a CHT native solution would be much better.

Additional context A number of partners have requested DHIS2 integration and if it's not automated it will slow or prevent adoption.

Why Now? Partner(s): Guava Pineapple Reason(s): A partner will be blocked by this for a major release or expansion

jdndiaye commented 2 years ago

Our feature request is not related to a problem. The manual process you described works well, the partners would like it to be automated in the future so that no one needs to be involved for a reliable import into DHIS2. For now, they agree to start with the manual process for the first phase.

mrjones-plip commented 2 years ago

Thanks for the update @jdndiaye!

@craig-landry - since this is not a pressing feature, I'm going to remove it from 3.15 in hopes of keeping a lean release. If this isn't right, bring it back or ping me on slack so we can discuss!

jdndiaye commented 2 years ago

Hi @mrjones-plip Sorry, I spoke with Fatou and the MOH information systems manager. And it is clear that automation is now becoming a priority in this project. Please take this into account in the first phase.

Best,

mrjones-plip commented 2 years ago

Thanks for the update @jdndiaye ! Let's put it back in 3.15 then :rocket:

mrjones-plip commented 2 years ago

Noting that there's now the App Services effort which, after configuration, is a single CLI call which automates pulling results directly from postgres in materialized views to DHIS.

cc @garethbowen for any architectural concerns and duplication of efforts

magp18 commented 2 years ago

Noting that there's now the App Services effort which, after configuration, is a single CLI call which automates pulling results directly from postgres in materialized views to DHIS.

cc @garethbowen for any architectural concerns and duplication of efforts

The link seems to be broken. Can this not be done by building a DHIS2 app that pulls data from CHT?

mrjones-plip commented 2 years ago

Hey @magp18 ! Thanks for your comment.

Sorry for the bad link to the "App services effort" - GitHub shows resources you don't have permissions for (like a private repo in this case) as "404" which isn't that helpful.

Further, I'll see if I'm allowed to post a copy of the solution I linked to. What was done on that link was a node app that would do an ETL. The difference from this ticket is that it leveraged and existing couch2pg deployment which, in turn, had materialized views run in PostgreSQL. These views were close to what was needed in DHIS2, such that an export from PostgreSQL -> light transform -> import into DHIS2 is what was done in the node app.

Speaking of an ETL process - I don't know the details of how to write a DHIS2 app, but as long as it can reach the CHT instance, can authenticate and query the right APIs, do any transforms if needed, then yes - a DHIS2 app could be written!

mrjones-plip commented 2 years ago

Follow up - I got the OK to post the code for the node app that pulls from PostgreSQL -> DHIS2. The PostgreSQL data was seeded by couch2pg and then enhance with materialized views. cc @magp18 .

kennsippell commented 2 years ago

Can this not be done by building a DHIS2 app that pulls data from CHT?

@magp18 Yes. It is very possible to integrate via a DHIS2 App. But I believe this would make a "one click integration with CHT from within the DHIS2 experience" - if that is what you desire, it would be a great option.

This ticket intends to make a fully automated integration (no clicks in either platform). I'm not sure how to do this as a DHIS2 app but would be interested to learn if that is possible.

magp18 commented 2 years ago

Thanks @mrjones-plip and @kennsippell for the input and clarification. Will have a look at the material provided too. Yes I am not sure how to do this as a DHIS2 app but heard about the possibility so thought about looking into it.

garethbowen commented 2 years ago

Not a blocker for 3.15.0