Control DE dates not registering time

[ifoche]

@gtrsmith @rodmelia @cristinaelle I've reviewed in server ken.malariaotss.org to see why dates were not being stored with their time. Apparently (I've confirmed this with eds-dev server) we were using Text type for these DEs, so they were being stored without problems even if we were sending both but not in Date&Time DHIS proper format (when this happened, there was a bug in Date&Time format so we used text type). Now I see that ken.malariaotss.org control DEs are configured with Date type. This means that time is not accepted by the server. To do so, we would need to change it to Date&Time time, but I've done a test and it generates a conflict when we do it (the format is not correct). So it is ok if I generate a new version for both HNQIS and EDS sending control DEs with Date&Time format? This would imply to change those DEs from the servers were the apps are working from Date to Date&Time type, and to update users so they don't have conflicts. In fact we've discovered that most of the duplicates come from conflicts, so an update of the users is something I would encourage to do in any case. What do you think?

[rodmelia]

I'm cc'ing Jose - Jose: I would like your opinion.

On 19 July 2016 at 19:08, Ignacio Foche notifications@github.com wrote:

@gtrsmith https://github.com/gtrsmith @rodmelia https://github.com/rodmelia I've reviewed in server ken.malariaotss.org to see why dates were not being stored with their time. Apparently (I've confirmed this with eds-dev server) we were using Text type for these DEs, so they were being stored without problems even if we were sending both but not in Date&Time DHIS proper format (when this happened, there was a bug in Date&Time format so we used text type). Now I see that ken.malariaotss.org control DEs are configured with Date type. This means that time is not accepted by the server. To do so, we would need to change it to Date&Time time, but I've done a test and it generates a conflict when we do it (the format is not correct). So it is ok if I generate a new version for both HNQIS and EDS sending control DEs with Date&Time format? This would imply to change those DEs from the servers were the apps are working from Date to Date&Time type, and to update users so they don't have conflicts. In fact we've discovered that most of the duplicates come from conflicts, so an update of the users is something I would encourage to do in any case. What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EyeSeeTea/EDSApp/issues/79#issuecomment-233717506, or mute the thread https://github.com/notifications/unsubscribe-auth/AQRv6mFstqpJeeus4fr7ydYvnnnWErFgks5qXRKTgaJpZM4JP_3l .

[josemp10]

Hi guys,

sorry I'm travelling today (in 30 minutes I'm taking a bus) and don't have much time to look at this until tomorrow Thursday (Asia time zone). I think, it would be nice to have also the time for the Control DE, but, I'm not so sure if they are already being supported (or if they are working properly). In PSI, I believe we have never used a DATETIME dataelement, but if we try to add a trackeddatavalue for a DATETIME DE using the web interface the value is not being stored in the database (if you test it, you need to open a new window in incognito mode), so, if this is just a fron-end problem or also the back-end, I dont know, I can test it tomorrow.

The tomcat log does not display any error...

Talk to you tomorrow

Regards Jose

On Tue, Jul 19, 2016 at 8:17 PM, rodmelia notifications@github.com wrote:

I'm cc'ing Jose - Jose: I would like your opinion.

On 19 July 2016 at 19:08, Ignacio Foche notifications@github.com wrote:

@gtrsmith https://github.com/gtrsmith @rodmelia https://github.com/rodmelia I've reviewed in server ken.malariaotss.org to see why dates were not being stored with their time. Apparently (I've confirmed this with eds-dev server) we were using Text type for these DEs, so they were being stored without problems even if we were sending both but not in Date&Time DHIS proper format (when this happened, there was a bug in Date&Time format so we used text type). Now I see that ken.malariaotss.org control DEs are configured with Date type. This means that time is not accepted by the server. To do so, we would need to change it to Date&Time time, but I've done a test and it generates a conflict when we do it (the format is not correct). So it is ok if I generate a new version for both HNQIS and EDS sending control DEs with Date&Time format? This would imply to change those DEs from the servers were the apps are working from Date to Date&Time type, and to update users so they don't have conflicts. In fact we've discovered that most of the duplicates come from conflicts, so an update of the users is something I would encourage to do in any case. What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EyeSeeTea/EDSApp/issues/79#issuecomment-233717506, or mute the thread < https://github.com/notifications/unsubscribe-auth/AQRv6mFstqpJeeus4fr7ydYvnnnWErFgks5qXRKTgaJpZM4JP_3l

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/EyeSeeTea/EDSApp/issues/79#issuecomment-233720157, or mute the thread https://github.com/notifications/unsubscribe-auth/AD9HZw3ze5SUVomuRotBM_-bifpSCMCwks5qXRS4gaJpZM4JP_3l .

[ifoche]

Hi @josemp10, all. Just in case. We're already using Date&Time DEs in MCS Cambodia. Just in case you want to test it there. I remember they reported some problems in Event Reports app regarding these DEs, but the time is being stored there.

[josemp10]

Right, my bad, I completely forgot about MCS, so it is a problem with the front end then. Thanks!

I think it may be good idea to change the control dataelement type to DateTime, BUT I agree with Ignacio that this is something that we may need to coordinate with the users (if so, @gtrsmith https://github.com/gtrsmith to decide if it is pertinent or not. There are hundred of users in EDS ...). The questions that I have:

1) We are only talking about control DataElements right? As far as I know the _Push Date Control DE always had its valueType as Date... I know there are other DataElements storing dates as text, but they are Questions

2) What does it happen when a user pushes just the date for a DateTime dataelement using the SDK/web API? ( Using the web interface the date is not being stored ...) @ifoche, I'm assuming from your email that it is generating a conflict? (for example, just submitting '2016-04-01' to a DATETIME dataelement)

Thanks Jose

On Wed, Jul 20, 2016 at 9:21 AM, Ignacio Foche notifications@github.com wrote:

Hi @josemp10 https://github.com/josemp10, all. Just in case. We're already using Date&Time DEs in MCS Cambodia. Just in case you want to test it there. I remember they reported some problems in Event Reports app regarding these DEs, but the time is being stored there.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EyeSeeTea/EDSApp/issues/79#issuecomment-233863515, or mute the thread https://github.com/notifications/unsubscribe-auth/AD9HZ7Hg5L9gu29eLsxdhgK0fabhObd0ks5qXcxlgaJpZM4JP_3l .

[ifoche]

Hi @josemp10, in fact...I think is the other way around. There are questions that are using Date type (like KE ADH-1040-TPT-Pat. 18 - Record Date), but in development server we used always Text type for the control DE (we changed it recently to test it). However we always used a Date-compatible format as we had tested Date&Time format before (long time ago), realizing that it didn't work. Now that it's working we can move forward and send it with time enabled. The tests we did some days ago changing to Date&Time the control DEs that were previously set as Date produced a conflict. So the only way to store the time and not produce a conflict is to set it as Date&Time.

The only other possibility that I can imagine to avoid the conflict and keep a staging situation with both users of new and old versions is to create a new control DE for the users of the new version, and remove the old control DE only when we're sure no user is still using the old version. That moves the complexity from the users to server side...but otherwise I don't see any other solution apart synchronizing all the users at once.

[josemp10]

Ok, sorry, I was talking about the EDS production server https://eds.psi-mis.org/api/dataElements/cyT9X5y15lj (maybe the databases, eds and eds-dev, were desync at some point in April for the Control DEs)...

In any case, I agree, @gtrsmith to confirm, but I think it is going to be pretty difficult to sync here hundreds of users from 5 different countries at once. So, as @ifoche mentioned, we should create a temp solution for a period of time (months maybe?). Not sure if it is worth (in hnqis we had a similar problem, we changed several DEs to a Text valueType as a temp solution, but it was solved in a couple of days)

On Thu, Jul 21, 2016 at 10:45 AM, Ignacio Foche notifications@github.com wrote:

Hi @josemp10 https://github.com/josemp10, in fact...I think is the other way around. There are questions that are using Date type (like KE ADH-1040-TPT-Pat. 18 - Record Date), but in development server we used always Text type for the control DE (we changed it recently to test it). However we always used a Date-compatible format as we had tested Date&Time format before (long time ago), realizing that it didn't work. Now that it's working we can move forward and send it with time enabled. The tests we did some days ago changing to Date&Time the control DEs that were previously set as Date produced a conflict. So the only way to store the time and not produce a conflict is to set it as Date&Time.

The only other possibility that I can imagine to avoid the conflict and keep a staging situation with both users of new and old versions is to create a new control DE for the users of the new version, and remove the old control DE only when we're sure no user is still using the old version. That moves the complexity from the users to server side...but otherwise I don't see any other solution apart synchronizing all the users at once.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EyeSeeTea/EDSApp/issues/79#issuecomment-234193300, or mute the thread https://github.com/notifications/unsubscribe-auth/AD9HZ_cBWrGZD4vk2pLaeNwBgEhJ9nIJks5qXzGrgaJpZM4JP_3l .