Questionnaires broken due to changes made to code lists

[HayleyMills]

The questions, constructs and doc view are not loading for nshd_77_q. This happened at midday on Monday 20th May.

[HayleyMills]

alspac_91_pq has also broken.

JS and I believe this is caused when a code list which is used by a question is changed from a response domain to not a response domain, this causes issues with the way the code list is attached to the question which breaks the question, which then breaks the construct and doc view if the questions are also added to constructs.

Restoring from the back-up will be the safest thing to do and reload into Archivist. Although this may have problems with the import/export due to orphaned items (#280)

In future we must ensure that is is not possible to untick the response domain option in the code lists if it is being used by a question already. The code list must be removed from the question first and then it can be edited. In the mean-time I will pass this information on to Archivist users.

[charlesdebarros]

When those instruments' code lists were edited, for some reason I still need to find out, they lost their response_domain id reference (even though the ids were still available in the database).

I have added a layer of checks to tell Archivist what to do in those cases. Unless an response_domain id is null, then just display the content as usual. That makes Archivist see those response_domain ids and display then accordingly.

This correction worked on both nshd_77_q and alspac_91_pq.

Waiting for @HayleyMills to QA and sign it off.

[charlesdebarros]

alspac_91_pq has also broken.

JS and I believe this is caused when a code list which is used by a question is changed from a response domain to not a response domain, this causes issues with the way the code list is attached to the question which breaks the question, which then breaks the construct and doc view if the questions are also added to constructs.

It seems the issue was that, when changing from response_domain to non-response_domain, the reference to that response_domain was kept and, when Archivist reached that reference (which would return null), it dit not know how to handle it.

In a first instance, I created a layer of checking to make sure Archivist knows how to handle those 'null'. My intention now is to make sure those references are removed when such changes are made.

[HayleyMills]

@charlesdebarros can you clarify what requires testing please? Should alspac_91_pq and nshd_77_q now be working? Do you want me to test what happens when a code list which is being used is no longer a response domain?

[charlesdebarros]

Should alspac_91_pq and nshd_77_q now be working? Yes. They should. You may try and load them.

The kind of problem that generated this issue in the first place, i.e., editing code lists, should now be sorted.

[HayleyMills]

Tested on staging all broken instruments are now loading. Tested changing a code list from a response domain which was used in a question using hm_test_1 and it saves the code list (must refresh to see this) and it removed the response domain from the question.

[HayleyMills]

Tested in staging by JS and it does not save the code list without a response domain the first time, it saves the second time.

There may also be more work to be done to resolve the knock on effects. 1) The question the response domain was initially attached to cannot be edited correctly. For example if you add a new response domain (because the old one has been removed) it will not allow it to be saved. If you edit anything else, you still cannot save it, but it does save when you click cancel.

2) If you turn the code list back to a response domain- it gives an error message "Code list failed to update! undefined" when you save it, although it looks like it does save.

[simonreed]

@HayleyMills I'm adding in validations to ensure that min_responses and max_responses are present for a Response Domain. Is there any validation that they should have e.g. an integer or within a range.

[HayleyMills]

I don't think there is any validation currently, but if you are adding some then- they shouldn't have a value which is larger than the number of categories. Min can have 0, but max should be greater than 1. - does that make sense?

[simonreed]

This is on staging @HayleyMills

[HayleyMills]

Tested on staging using hm_test_2- It does let you edit the question that the original response domain was attached to (grid or item) which is great. There are two outaiding issues:

a) It doesn't save the first time when you want to untick a code list as a response domain.

b) When changing the code list back to a response domain by ticking it gives the error message "Code list failed to update! Response domain code min responses can't be blank, Response domain code min responses is not a number, Response domain code max responses can't be blank, and Response domain code max responses is not a number"

[simonreed]

a) It doesn't save the first time when you want to untick a code list as a response domain.

I'll fix

b) When changing the code list back to a response domain by ticking it gives the error message "Code list failed to update! Response domain code min responses can't be blank, Response domain code min responses is not a number, Response domain code max responses can't be blank, and Response domain code max responses is not a number"

That error message is saying that you're trying to create a response domain with a blank min and max responses. Did you get this error message when you submitted the form?

Or are you saying you want to improve the error message, unfortunately the error messages while descriptive are actually not that easy to change the format of.

[HayleyMills]

That error message is saying that you're trying to create a response domain with a blank min and max responses. Did you get this error message when you submitted the form?

Or are you saying you want to improve the error message, unfortunately the error messages while descriptive are actually not that easy to change the format of.

I got the error message when I saved it, but it did have min and max responses (the default 1,1). The error message itself is fine.

[simonreed]

@HayleyMills this is now on staging and working as in the gif below (note that I refresh the page in the gif to demonstrate that it has been persisted).

I got the error message when I saved it, but it did have min and max responses (the default 1,1). The error message itself is fine.

Did you manually enter the min/max responses when creating the response domain again. The reason i ask is that those input fields have placeholders to demonstrate an example of what you could enter (in this case they're both 1). While it might look like 1 has been entered the placeholders aren't submitted they're only for display purposes so if you the user doesn't enter their own values then they will be submitted as blank, hence the error messages.

[HayleyMills]

No I didn't add any min and max, as the usual behaviour is that if you don't enter a min and max then the default is 1 and 1, rather than it being an example, this is what it should save. In most cases, the min and max will be 1 and 1.

[simonreed]

Ok I will change to this behaviour

[HayleyMills]

Testing on staging using hm_test_2 and this is now behaving as expected with the default 1 1 saving.