[Support Request] Plan data is not reflecting on the map or dashboard

[pongthepm]

Hi, Please see the below detail. we found problems. Could you please fix it? Thank you, Pongthep

plan: A2 ไอปือลง (9609010701) 2023-05-10 id plan: 28f4267d-c791-59b3-a58d-7c90cbe74d06 https://mhealth.ddc.moph.go.th/plans/update/28f4267d-c791-59b3-a58d-7c90cbe74d06

user: 12.4.2-1 finished the blood draw activity for 53 patients (15 households), finished 1 BCC (image 1) however, when BVBD checked in the web, they found a problem that no data on the map (image 2) and no data on metabase(dashboard), image 3.

[pongthepm]

Also, I have uploaded the video and local DB. https://tmbiophics-my.sharepoint.com/:f:/g/personal/pongthepm_biophics_org/Et2Ze2RIOZdCqlcYVRB0ABsBy7d_PzTLp6Es8LFdzrGkkQ?e=GFgbxL

[AngelaKabari]

Hello @pongthepm I confirm receipt of this ticket.

@truongnt - Kindly provide the go ahead for Ona to action this?

[AngelaKabari]

@truongnt It will take an Ona engineer approximately 4 hours to identify what the cause of issue is as well as the Level Of Effort (LOE) estimate for resolving both this and issue #31. Please confirm that we can proceed with this?

[peterMuriuki]

tasks are missing. The request to get this data either times out with a 504 or returns 0 records for the applied filters.

the affected slice endpoint is 8. It pulls data from the thailand_prod.task_structures_geojson_slice data source.

The slice is a wrapper for task_structures_materialized_view materialized view.

Ran the following query on superset to confirm that the data wasn't in the view as well.

SELECT * from thailand_prod.task_structures_materialized_view where plan_id = '28f4267d-c791-59b3-a58d-7c90cbe74d06'; The query returned no data

note: I ran the queries on superset, under the reasonable assumption that superset was in sync with the db.

This can be escalated to the data engineering team. cc @lincmba

[lincmba]

Tried to replicate the above in at database and got that the slice doesn't have the plan data

eal1=> SELECT task_identifier AS task_identifier,
reveal1->        plan_id AS plan_id,
reveal1->        jurisdiction_id AS jurisdiction_id,
reveal1->        goal_id AS goal_id,
reveal1->        geojson AS geojson
reveal1-> FROM thailand_prod.task_structures_geojson_slice
reveal1-> WHERE plan_id='28f4267d-c791-59b3-a58d-7c90cbe74d06'
reveal1-> LIMIT 100;
 task_identifier | plan_id | jurisdiction_id | goal_id | geojson
-----------------+---------+-----------------+---------+---------
(0 rows)

Then checked for the same data from the materialized view that the above slice picks its data from and it too lacked the tasks for the plan

reveal1=> select count(*) FROM thailand_prod.task_structures_materialized_view WHERE plan_id='28f4267d-c791-59b3-a58d-7c90cbe74d06';
 count 
-------
     0
(1 row)

Realized that the tasks exist in the database based on the following query

reveal1=> select count(*) FROM thailand_prod.tasks where plan_identifier='28f4267d-c791-59b3-a58d-7c90cbe74d06';
 count 
-------
   613
(1 row)

Next step was to refresh the mateialized view, this however errored with the following error

reveal1=> REFRESH MATERIALIZED VIEW thailand_prod.task_structures_materialized_view;

ERROR:  could not extend file "base/18871/381744094.1": No space left on device
HINT:  Check free disk space.
reveal1=> 

[truongnt]

@lincmba Which server is out of space?

[lincmba]

@truongnt this is the database server. Here's how the disk utilization looks like

mhealth@database-server:~$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                               3.9G     0  3.9G   0% /dev
tmpfs                              798M  2.0M  796M   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv  122G  113G  2.9G  98% /
tmpfs                              3.9G   48K  3.9G   1% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                              3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda2                          974M  151M  757M  17% /boot
tmpfs                              798M     0  798M   0% /run/user/1000
mhealth@database-server:~$ 

[pongthepm]

Dear Lincoln, We found there are many backup files consuming the disk space (please see the attached file). Can we delete the backup files of 2022? Thank you, Pongthep

From: Lincoln Simba @.> Sent: Monday, June 5, 2023 8:41 PM To: onaio/Reveal-Thailand @.> Cc: Pongthep Meankaew @.>; Mention @.> Subject: Re: [onaio/Reveal-Thailand] [Support Request] Plan data is not reflecting on the map or dashboard (Issue #32)

@truongnthttps://github.com/truongnt this is the database server. Here's how the disk utilization looks like

@.***:~$ df -h

Filesystem Size Used Avail Use% Mounted on

udev 3.9G 0 3.9G 0% /dev

tmpfs 798M 2.0M 796M 1% /run

/dev/mapper/ubuntu--vg-ubuntu--lv 122G 113G 2.9G 98% /

tmpfs 3.9G 48K 3.9G 1% /dev/shm

tmpfs 5.0M 0 5.0M 0% /run/lock

tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup

/dev/sda2 974M 151M 757M 17% /boot

tmpfs 798M 0 798M 0% /run/user/1000

@.***:~$

— Reply to this email directly, view it on GitHubhttps://github.com/onaio/Reveal-Thailand/issues/32#issuecomment-1576830473, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A4MDDNMHR4EPUF3YUFD5QBLXJXOXPANCNFSM6AAAAAAYPUODEM. You are receiving this because you were mentioned.Message ID: @.**@.>>

[AngelaKabari]

Hello @pongthepm please consult with DVBD to find out whether they would like to delete the 2022 back up files or allocate more storage space, as this is their decision to make and not Ona's.

[pongthepm]

Hi @AngelaKabari Are these backup files? that I can delete?

[pongthepm]

We are going to add more disk space tonight.

Thank you, Pongthep

[AngelaKabari]

Hello @pongthepm,

I'm following up on this - did you and the DVBD team complete the addition of disk space?

[pongthepm]

Dear Angela, IT team of BVBD has added disk space already. Thank you, Pongthep

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: AngelaKabari @.> Sent: Monday, June 12, 2023 11:24:34 PM To: onaio/Reveal-Thailand @.> Cc: Pongthep Meankaew @.>; Mention @.> Subject: Re: [onaio/Reveal-Thailand] [Support Request] Plan data is not reflecting on the map or dashboard (Issue #32)

Hello @pongthepmhttps://github.com/pongthepm,

I'm following up on this - did you and the DVBD team complete the addition of disk space?

— Reply to this email directly, view it on GitHubhttps://github.com/onaio/Reveal-Thailand/issues/32#issuecomment-1587663265, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A4MDDNIIVKA2BRV3DBN7NETXK47EFANCNFSM6AAAAAAYPUODEM. You are receiving this because you were mentioned.Message ID: @.***>

[AngelaKabari]

Thanks for the confirmation @pongthepm.

@lincmba could you please check whether the slice now has the plan data and refresh the materialised view too if necessary?

[HusnaHariz]

The matview that failed to be refreshed last time is up to date and has records linked to plan_id='28f4267d-c791-59b3-a58d-7c90cbe74d06'

reveal1=> select count(*) FROM thailand_prod.task_structures_materialized_view WHERE plan_id='28f4267d-c791-59b3-a58d-7c90cbe74d06';
 count
-------
   613
(1 row)

SELECT count(*)
FROM thailand_prod.task_structures_geojson_slice
WHERE plan_id='28f4267d-c791-59b3-a58d-7c90cbe74d06'
LIMIT 100;
 count
-------
   613
(1 row)

[AngelaKabari]

Hello @pongthepm the points are now showing on the map.

Please note that by default the activity whose points are shown is that of Family Registration, but since we do not have that for this plan there are no map points data shown initially. One has to click the present activities links, in this case blood draw activities, to see the associated map points data.

For health education activities (BCC), there was one task but without geometry, therefore it is not showing on the map. Is this deliberate?

Please ask DVBD to look into at this issue and confirm that it has been satisfactorily been resolved? If yes, please close out this issue.

[AngelaKabari]

Hello @pongthepm, please confirm that this issue has been satisfactorily resolved so that we can close it out?

[pongthepm]

Hi @AngelaKabari You can close this issue. Thank you, Pongthep