Improve copying of surveys between instances

[mtwestra]

At the moment, when a survey is copied from one instance to another, the original ids of the objects are used in the new instance. This is fine when surveys are copied to an empty instance, but risky when surveys are copied to an existing instance.

Proposed solution: 1) implement a save copy of surveys 2) check if translations are copied as well

[iperdomo]

@mtwestra we have discussed the idea of doing it on FLOW services - https://github.com/akvo/akvo-flow-services/issues/7

[stellanl]

Remapping ids during import is simple in concept but has a few unsolved issues.

Currently there are two cases: A. Copying the entire instance. This should work fine, as all the id's are processed together. B. Copying one Form. This becomes problematic when you want to copy a multiple-form (monitored) survey. You will have to copy each form separately, so they will end up in separate trees of folders+survey, but with the same names.

a possible fix is to introduce a third copying mode: C. Copying a Survey, or more generally a Folder. This solves the multiple-form survey case, but the problem now becomes how to know the id of the survey/folder. They are not displayed in the UI and would have to be dug out of the instance DB console one by one.

[stellanl]

Do we want newly copied surveys to start out unpublished?

[stellanl]

The id remap currently is done only for single-survey copying, but all forms in the same survey as that which is indicated by the parameter are copied. The copied survey is always created in the root folder.

[janagombitova]

@stellanl I found three things that did not get copied well into the copied survey on dev2:

• settings for geoshape questions - it any of the checkboxes was selected in the original survey, it was not shown in the new form
• repeatable group - if a group was set as repeatable, it was not set so in the copied form
• option codes - if options did not have codes added, in the copied form they all show code "null"

[stellanl]

It seems JsonObject.optString() does the wrong thing with null values; it returns the string "null" which is not what we want. There are probably more places in the code where this could cause problems. We could check the datastores for "null" strings to get an idea of where. See http://stackoverflow.com/questions/18226288/json-jsonobject-optstring-returns-string-null

[stellanl]

There is one improvement still possible; remap id's when copying a whole instance. But that requires a a lot of work to recreate the folder structure.

[janagombitova]

Code tested with 180+ SNV surveys. 👍 can be considered passing test

We will not release this as a full user functionality for now (no UI changes), but as a support service which 2nd line of support will handle in a controlled way (creating the copying link, cross checking that all questions got moved properly, options and settings and informing the user). There will be a questionnaire added to Flow support to easily guide to user through the copying request.

Secondly we will track what types of requests we receive (motivations of copying a survey across dashboards) to evaluate the need to support 'best practice/standardised surveys'. In January 2017 we will review the findings and decide on next steps.

[janagombitova]

Test plan

1. Find two surveys on any existing dashboard - preferable survey with over 100 questions and multiple options. One survey should be non monitored and then other monitored.
2. Create the script to copy the non monitored survey from the dashboard and copy into UAT1 dashboard
3. Repeat with monitored survey
4. Cross check that surveys are copied in properly, all questions are in, all options are in, survey setting are set up ok (monitoring, registration forms)

[janagombitova]

Implementation passes tests plan 👍