Form filters and ANC analytics dashboard not working as expected

medic / cht-core

The CHT Core Framework makes it faster to build responsive, offline-first digital health apps that equip health workers to provide better care in their communities. It is a central resource of the Community Health Toolkit.

https://communityhealthtoolkit.org

GNU Affero General Public License v3.0

468 stars 217 forks source link

Form filters and ANC analytics dashboard not working as expected #3070

Closed abbyad closed 6 years ago

abbyad commented 7 years ago

We are seeing an issue in a project where filters and analytics are not working properly on a v0.4 instance.

We have not yet had other reports of this issue. The project in question is using Nepali characters for form codes - which may be responsible of the issue or a red herring.

garethbowen commented 7 years ago

I agree with @bishwas-medic - this module needs a complete rewrite either using postgres (via couch2pg) or using targets (2.x only). I don't think we have time to do this in this iteration.

sglangevin commented 7 years ago

I think this should be bumped to the next iteration as we are more than halfway through the current iteration. Let's hold off for now and consider options for either fixing this bug or increasing the priority of the re-write.

sglangevin commented 7 years ago

Thinking about this a bit more, let's see if there is something simple we can do to fix this. If not, we would have to hold off until a later release.

abbyad commented 7 years ago

Just want to highlight that there are potentially two issues here, and that they are related only in that they both rely on getting subsets of forms with Nepali form codes: 1) Form filters in the Reports tab are not returning the correct forms 2) Analytics widget showing incorrect values

I think it makes sense to deal with Analytics during a rewrite, but the form filters not working is a more immediate concern. Also, there is a chance that solving it may also lead to a quick fix to Analytics without it needing a rewrite.

garethbowen commented 7 years ago

This could be an internationalisation issue if filtering forms by unicode characters isn't working as expected...

garethbowen commented 7 years ago

Ok, I've done some investigation and can't see a quick fix.

I can't reproduce the reports filter bug and it looks to be caused by changes not being filtered so as changes come in they gradually pollute your filtering. I vaguely recall us fixing a bug for that but can't find it now... As this instance is top priority for migration to 2.x I propose getting the upgrade out and trying to reproduce the error with recent code. Perhaps we should add this to the migration testing for this instance?

I did see the ANC Analytics issue. That may also be fixed by upgrading to 2.x but I'm not confident that the in-app analytics are production ready regardless of this specific bug. Can we get them using klipfolio instead, perhaps via couch2pg? It's not in-app but it's the next best thing. We have some ideas about replacing in-app online-only analytics with either embedded single sign-on klipfolio or direct queries against postgres.

@abbyad Thoughts?

abbyad commented 7 years ago

Thanks Gareth, let's revisit the filtering and analytics post upgrade to v2.

garethbowen commented 7 years ago

Closing until it happens in 2.x

abbyad commented 7 years ago

Added AT to this issue so that we do come back to it on v2.

ngaruko commented 7 years ago

Are we able to AT this at this stage @abbyad @sglangevin ?

sglangevin commented 7 years ago

This should be tested on the cloned/upgraded version of the instance where it was originally reported, if possible. We should coordinate with @bishwas-medic

bishwasBhatta commented 7 years ago

Did a quick check with the ANC widgets in a clone instance running v2.11.0-alpha. I can confirm that the issues still persists. Here are few issues that I could verify:

The Active Pregnancies number is way off.
The Monthly Pregnancies Registered chart is showing very low numbers than actual numbers
The Total Births number is higher than actual numbers
The Visits Completed During Pregnancy widget doesn't show any data
The Reported Delivery Locations is also empty. (I know the reason for this. We don't have anywhere where we specify the codes for different delivery locations, so the code for this widget might be looking at the standard codes in English while the codes are in Nepalese)

Reopening the issue as it is happening on v2.x too.

sglangevin commented 7 years ago

@bishwas-medic please test the filtering issue as well and feedback on whether or not that is working. Which project were you testing on the cloned instance?

bishwasBhatta commented 7 years ago

All the filters for Reports seems to be working fine. I will DM you the name of the instance since this thread is public.

abbyad commented 6 years ago

Closing as the original issue is resolved:

We are not supporting this ANC dashboard in v2, so I'm closing this issue as the report filter piece is resolved.