Remove _copy from question ID after duplicating a survey.

moloscripts commented 9 years ago

problem After duplicating a form, the text in the question ID is accompanied by __copy_

screen

Proposed Solution After duplicating a survey, the questions do not come with the text __copy_. Is it possible to use the same logic in the question ID?

muloem commented 9 years ago

@mtwestra This depends on the scope of question ID uniqueness. If it is still system-wide as I recall, then this solution cannot work. The options left are either to not copy the question ID or leave it with the _copy suffix.

mtwestra commented 9 years ago

@muloem question-IDs are only unique inside a survey (meaning, at most among the forms inside a single survey), so this should not be a problem.

mtwestra commented 9 years ago

We will take the simple approach and not add the "_copy". For copying a survey, this will work fine.

The only occasion where this might not work is when we copy a form inside a survey, something we do not support at the moment. However, it is not clear in what case people would want to have two copies of a form in a single survey.

Furthermore, this edge case could be tackled by the foreseen 'survey integrity check' that we want to build in.

mtwestra commented 9 years ago

Test plan: 1) create a survey, and include some questionIds 2) copy the survey 3) the question ids in the questions should be identical to the source questions.

janagombitova commented 7 years ago

Hi @muloem and @mtwestra what is the status of this fix? There seems to be some referenced code but the issue is not fixed yet.

Secondly I would like to clarify for future reference: does the question ID have to be unique (1) only in a form but can be repeated in another form in the same survey or does it have to be unique (2) across all forms of the survey? Thank you both

janagombitova commented 7 years ago

Created via Reamaze:

Link: https://flowhelp.reamaze.com/admin/conversations/copying-survey-question-id-changes Assignee: Unassigned

janagombitova commented 7 years ago

Problem

As a dashboard user I want to reuse a survey we have implemented in one region to collect data in another region. We import the collected data into our national database where we link the data set to our structure using question IDs. I want to make a copy of the original survey (with all its forms and questions with question IDs) to create a new survey for the 2nd region. Now if I do so, I have to then go through all the questions and remove the "_copy" from the question IDs.

Current rules in place

As there is a bit of unclarity on how the question IDs are currently set up and what the rules are, I have ran a test on UAT to understand what the current implementation is from the user's point of view:

only allowed characters are: a-z; A-Z; 0-9; -; _
we make a distinction between upper case and lower case letters ("test-1" does not equal "Test-1" and they are considered unique)
the question ID must be unique across the entire survey, so across all forms in that survey
if the user tries to save a question that has a question ID in it and it does not fit these criteria, then he cannot save the question and gets a info dialogue

Proposal

The idea is to remove the "_copy" from the question ID if a survey is copied. (Not the word "copy" from the title of the survey) As we copy the entire survey with all its forms, the rule that questions IDs must be unique across the entire survey (so across all its forms) will still be met, as in the original survey this rule was validated when creating the IDs.

Risk

The risk with this implementation is once we enable copying one form (or question group) into the same survey or into another survey.

Why would a user want to reuse the same form in the same survey?

We have cases where partners create:

the same form for registering the data point and for monitoring the point, where some questions are removed from the monitoring form, but most of the questions are identical
the same monitoring form from an existing monitoring form and change a few questions but some are identical
users re-use a group of questions from one monitoring form in another monitoring form in the same survey

The forms are now created manually but we receive many requests to enable the copying as a feature, or to at least enable copying a question group across a form. In both cases we risk the situation that the question IDs will not be unique anymore.

How are question IDs used in cases where questions are identical across forms of a survey?

(Answer from Sam from CWSA Ghana, where they use the identical structure of the registration form as the monitoring form as well. The have question IDs created for each question as they upload Flow data into their national Dimes database)

"...yes all question IDs are unique across both forms in this case we are able to know the difference between a baseline data and a monitoring data. So for example, in a baseline form a question iD could be like "functional" while in a monitoring form [for the same question] it will be "functional_m"..."

Steps forward

As we do not currently support copying a form (or question group) within the same survey or as another form in another survey this risk currently does not exists. We need to however take the rules of question IDs into account when thinking about this opportunity. https://github.com/akvo/akvo-product-design/issues/178

For resolving this current issue we need to ensure that:

when the user copies a survey which has question IDs we do not add "_copy" to the question ID
we do not break the current rules in place

@muloem and @stellanl lets talk a bit more about this issue in the next planning session

janagombitova commented 7 years ago

Created via Reamaze:

Link: https://flowhelp.reamaze.com/admin/conversations/akvoflow-support-request-number-44720-from-sarah-dot-bailey-at-sendacow-dot-org

stellanl commented 7 years ago

This solves the basic problem. However, the case of copying a question group more than once within the same survey still remains. How far do we want to go to ensure no id duplications ever happen?

janagombitova commented 7 years ago

@stellanl and @muloem so I tested how the copying currently works with a survey which has 2 forms and all questions have IDs.

The validation of if an ID is unique or not happens in the ID field and if the user still wants to save the question with a non-unique ID, the saving is not possible. There is no validation of this kind when the user publishes the survey.

Current copying patterns with questions with IDs

Case 1: Copying an entire survey with question IDs

all IDs get "_copy" added to the IDs

Case 2: Copying a question group (or a single question) in one form

if the user copies an existing question group with question IDs and does not make any edits to the questions (so copies and want to directly publish)
the survey gets published
the copied question group with the copied IDs have "_copy" added to the ID
if a question is copied twice it the gets another "_copy" added to the ID (so it can become "ID-1_copy_copy")

Changes

Case 1: Copying an entire survey with question IDs

here we said that we will remove the "_copy" in the IDs to stop forcing users to have to delete that from the question ID
the risk of duplicated IDs is not in place here as the entire survey is copied

Case 2: Copying a question group (or a single question) on the survey form with IDs

in this case the IDs get duplicated and then are not unique
currently we add the "_copy" to the ID to ensure that each ID is unique
we have to assume that the user might copy the question group and not make any further edits, will not see the questions and their details
we can either leave the "_copy" in place for copying question groups and single questions within the same form to not have to add any UI elements to warn the user about the duplication of the ID
or we full remove the rule to add "_copy" to the question ID, but then we need to indicate to the user that the ID in a question is not unique (either when publishing, or somewhere else) > I feel this is a larger change also affecting other improvements

@muloem and @stellanl how would you deal with this issue?

stellanl commented 7 years ago

Case 2 has two sub-cases for repeated copies: 2a. Copy group A to group B, then copy group B to group C, etc. "_copy" is added for each step; ids remain unique, if a little ugly: "foo_copy_copy" 2b. Copy group A to group B, then copy group A to group C without changing ids in B first. B and C will then both have ids like foo_copy

stellanl commented 7 years ago

The copying code could ensure that case 2 always sets unique ids, maybe by incrementing a number like _2, _3 etc, at the cost of loading a bit more data. Ensuring it is unique across all forms of the survey adds another performance hit, but I don't think this is one of our bottlenecks...

janagombitova commented 7 years ago

@stellanl that is a good point 2a and 2b. I did not think of it. I did a cross check on how things work currently and we do not handle 2b well. It simply gives the "_copy" to the ID what results in have two groups with identical IDs. If you do not go to edit the question and directly publish the survey, it is all possible. Publishing works fine and the survey gets to the phone. I did not try to collect data or export it. However, if you edit the questions you see the validation that the ID is not unique, but can simply cancel the edits and move on. If you want to save changes, then you need to also change the ID with "_copy".

Stellan do you propose to add _copy_1for situations happening following case 2b?

stellanl commented 7 years ago

Basically, I have code that will detect and avoid a collision. Right now it does it by adding _1, _2, _3... etc until it is unique, but it can be anything at all. Tell me what would be best.

janagombitova commented 7 years ago

I think that is a good approach from the users point of view 👍

stellanl commented 7 years ago

Test plan: Set id on a question. Copy the question group. Check the questions in the copy; they should have different ids ending in _1 etc Copy an entire survey. The questions in the copy should have unchanged ids

janagombitova commented 7 years ago

👍 made copies of a first copy of a question group and then multiple copies of a copied question in a copy of a copy of a question group and IDs remained unique, while copying the entire survey did not change the IDs as wanted. Nicely done @stellanl. This one is ready to go

akvo / akvo-flow