"Queued for submission" message problem

[ghost]

As data collection is moving increasingly online at this time, I noticed that there is a second part to the GUI issue which especially affects people in lower bandwidth locations but also others and gives the impression of "lost data" when in fact the process is very reliable. I thought I would quickly open this issue before the GUI change from earlier in the month goes live on KoBo and other platforms.

While currently there are two messages indicating Record queued for submission or Record sucessfully submitted, the messages appear in the same location at the top of the screen, in the same colour and have similar amounts of text. As the rest of the screen doesn't change and there is no pop-up message when closing the tab, its common for the user to close the tab without realising the data has not yet been sent and not return later to re-submit.

I think there are several possible solutions and as before I am sure that those with more experience in this area will be able to improve upon them:

• Styling the Record queued for submission pop-up message like the Finalize record later info box (prominent, centre of screen, on top of the form, click OK to continue). It would need to have a title indicating that the process is not complete and a sentence warning against closing the browser tab. Preferably it would include some kind of "processing symbol" to indicate that its a temporary message. I am not sure if these messages can disappear or change when the record is submitted.
• Add a warning pop-up message appearing from the browser itself on clicking to close the tab while there is a submission in queue (the way some websites use a pop-up to encourage the user to logout first or to verify that they do want to leave even if some process is partially complete). This would I think work very well but I am not sure if its compatible with any website or how dynamic that type of pop-up is.
• Keeping the current Record queued for submission message but filling a larger part of the screen, having a different colour and making it more clear that this message is the most important thing on the screen at that moment.

The first GUI update is already very useful and will simplify data collection for many organisations. I hope that it will be possible to resolve this issue and if there's anything I can do to assist, whether it be on translations or anything else, please do let me know as the combination of the two changes would be very effective in avoiding human error during the data submission process.

[MartijnR]

Thanks @NoelCartONG. Very useful feedback! Let's figure out what we can do. I'm hoping others will chime in as well.

The current UI is really assuming that users open the side bar regularly to actively keep track of outstanding submissions (or at least keep an eye on the number in the top left). It wouldn't surprise me that some users never do this, as it's not really obvious enough.

The second solution is technically problematic (doesn't work reliably), and I quite dislike this behavior myself (when it does work). Also I believe, that browsers are planning to remove this functionality.

[ghost]

Thanks very much for the quick response! I think you are correct about users not checking the top left, especially where a form has a large number of respondents, some of which may be new users or send data infrequently and fail to notice the queued records until after the survey is complete. Combined with the recent GUI update I hope a good all round solution will be possible.

[ghost]

Hi, can I ask if its possible that a solution could be included before the next Enketo update on kobo.unhcr.org? Users providing data as part of the Covid response are reporting significant difficulties in submitting data. In some cases users are worried that they have submitted the data multiple times but in fact none of their attempts were successful. Removing this issue would significantly facilitate these data collections.

Many thanks for the change that was already implemented which has already removed one half of the problem!

[MartijnR]

Thanks for following up @NoelCartONG. I've quickly cobbled something together using existing text strings. Please have a thorough look at https://odk.enke.to/x/YYY5 and let me know what you think.

I don't have a whole lot of time at the moment, so for something more involved, I hope the KoBo people can jump in, if they consider it a priority.

Happy to make small adjustments though.

cc @jnm

[ghost]

Thanks very much for already making a mock-up, I understand that this is a very busy time for you. Hopefully at some point in the future it will be possible. If Kobo are interested in the data collections on kobo.unhcr.org that are currently facing this obstacle, I am happy to discuss outside of github if relevant.

It looks (as far as I can tell) like the box appears almost immediately and in the centre of the screen, which is great as its now clear that something is happening. If possible it would look like the below images taken from Enketo but with the text adjusted so that it still makes sense for users who may submit data more than once.

I have since spoken with some colleagues about the possibility of using volunteer time to translate strings for the Enketo system so we will now start the process of identifying volunteers who speak useful languages such as Turkish, Persian or Bengali as well as of course Spanish, Arabic, French etc. in order to be able to make a contribution to the general translation effort via transifex. Once we have a little more information I will give an update on how much translation is likely to be possible and hopefully it will be possible to contribute to the widespread accessibility of the system.

[MartijnR]

Hi @NoelCartONG,

That's really great news about possibly helping out with more translations!

I think I misunderstood what you meant initially. For offline-capable webform views saving and submitting are very different actions that are assumed to take place with hours/days/weeks in between.

I think you're proposing some kind of hybrid solution but only for records not marked as draft, where an attempt to submit would be made immediately and the result is shown in the dialog.

That's a pretty big change, but I agree it would be very nice for users (that still require an offline-capable view) when they're online.

[ghost]

No worries, its mostly thanks to my colleagues who organise volunteer activities as they have adapted the tasks so that they can be done online during restrictions and still reach volunteers.

Yes I think you have summarised it. Usually the Enketo data collections I have been involved with have no fully offline data collection but include offline capability due to offices in more remote locations with slow or unreliable internet connections. What I find is that if I click submit and then close the browser tab, the data is never submitted, no matter how long I wait. It is only submitted if I re-open a tab with the same form and leave it open for a minute or two. Most users of course will not know this and immediately close their tab. Can I check if this is the expected process for the offline capable forms?

The big solution you mention would be ideal and I hope its possible some day but I understand that's a different scale of change. A simple dialog box like the mock-up you created but with new text, informing the user that submission may be in progress for some time and that its best not to close the current browser tab until they see a zero in the top left of their screen, would already cut the rate of problems in half. The message might be triggered with the same logic as the current yellow bar at the top of the screen but would be more effective from a user perspective.

[MartijnR]

What I find is that if I click submit and then close the browser tab, the data is never submitted, no matter how long I wait. It is only submitted if I re-open a tab with the same form and leave it open for a minute or two.

That's definitely not right. A submission attempt (of the whole queue) is made within 10 seconds after clicking submit. I think this indicates a different issue. It could be the data/form backend server that is not responding (I know this can be the case with ODK Aggregate, but am not aware of KoBoToolbox having that issue). Once a submission fails, it would take 5 minutes before the next attempt. I think that's what's happening in your case. It probably times out.

There is an additional attempt to submit after a page load.

Are you experiencing the same with https://odk.enke.to/x/YYY5 (ODK Aggregate) and https://enke.to/x/widgets (Ona.io)?