medic / cht-couch2pg

Software for creating read-only replicas of data from CHT applications, using PostgreSQL v9.4
Apache License 2.0
4 stars 8 forks source link

Cast period_start column on useview_feedback to have date time data type. #102

Open 1yuv opened 3 years ago

1yuv commented 3 years ago

The following line:

https://github.com/medic/cht-couch2pg/blob/3e38727075c0132fad3f3e41c4a4461b202519de/libs/medic-users-meta/migrations/202102191153.do.54-create-couchdb-users-meta-table.sql#L10

Stores period_start as text by default, but it has to be date time field so that it will be used as expected.

But to avoid the hiccups like previous couch2pg run, we should ensure that for any supported cht version, doc->>{'meta','time'} can always be converted to valid date time.

Why we need this change? We are now seeing that some partners have more than 10 million feedback docs. We'll want to filter those date by date time for analytical purpose and probably create index to optimize the query. But, with current view structure, it is not possible to do so.