Getting duplicates for write-in other answers

[kjhenner]

I've noticed that calling response.parsed_answers results in two answer objects for questions with a write-in other field, one with answer_text "Other" and one with the actual write-in value from the response. Because these share the survey_id, response_id, question_id, and option_id fields, I end up with errors down the line when I try to insert this data into my database. Is this the expected behavior?

[jarthod]

Mm, that doesn't sound expected, no. You implemented this @apurvis, any idea ?

[apurvis]

this is sort of how SG output actually works, though it would be helpful if you could post the debug output.

from the README: "Other" text for some questions is parsed to Answer#other_text; all other answers to Answer#answer_text

so IIRC answer_text has "other" but other_text has the real write in answer.

however it's possible some small tweak on SG's side means that now things are sort of different than they were when i implemented this.

[kjhenner]

Here are a couple examples of the JSON for answers returned from the API:

"[question(54), option(10210)]": "",
"[question(54), option(10211)]": "I already use Puppet for job related tasks, and wanted to develop my skills.",
"[question(54), option(10212)]": "",
"[question(54), option(10213)]": "",
"[question(54), option(10214)]": "I work with people who use with Puppet, and wanted a better understanding of what they do.",
"[question(54), option(\"10215-other\")]": "",
"[question(54), option(10215)]": "",
[...]
"[question(54), option(10210)]": "",
"[question(54), option(10211)]": "",
"[question(54), option(10212)]": "",
"[question(54), option(10213)]": "",
"[question(54), option(10214)]": "",
"[question(54), option(\"10215-other\")]": "Wanted to learn it",
"[question(54), option(10215)]": "Other",
[...]
"[question(73)]": "Systems Engineer",
"[question(73), option(\"10241-other\")]": "",

This confirms that the double result is initially coming from SG itself.

I'm trying to understand the difference in how the "10215-other" and "1025" results are parsed by the gem. They each have an "answer_text" that corresponds to the value in the example above (i.e. the write-in answer "Wanted to learn it" and the string associated with the check-box itself "Other") should one or both of these have an "other_text" with that write-in value as well?

[apurvis]

i think so - at least this is how i am using it in my app:

answer = response.parsed_answers.first.to_hash
answer[:answer_text] = answer.delete(:other_text) unless answer[:other_text].blank?

basically just copying values at the other_text into answer_text, but i didn't do that inside the gem just in case someone needs access to both values.

ps are you sure answer_text in your example is "Wanted to learn it." and not just like "other" or ""

[apurvis]

it appears that your responses from SG don't quite match what the code is expecting: the code should only be giving you one Answer - the other_text

i didn't dive into the details of your case but the specs for this behavior are pretty thorough - maybe you can figure out what's different about your responses looking over there?

[kjhenner]

Ok, thanks, I'll have a look when I get a chance.

[apurvis]

the code should be throwing out the response that doesn't have the actual info in it because of that regex and you should just be left with one Answer to abstract away the grossness of how SG does this, but yeah something is slightly tweaked about the format you are getting responses in and things aren't quite working correctly.

[kjhenner]

May not be the cleanest fix, but I think this demonstrates the problem: https://github.com/jarthod/survey-gizmo-ruby/pull/75

[kjhenner]

Seems like the escaping thing was a red herring. Changing the filter logic as in https://github.com/jarthod/survey-gizmo-ruby/pull/77 resolves the issue for me, though I honestly have no idea why. I tried a few possibilities that seemed logically equivalent and got different results.

[kjhenner]

If I modify the parsed_answers method to print the answers before and after applying that filter, I get stuff like the following.

Here, the "Other" option is correctly removed.

UNPARSED ANSWERS
{
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(54), option(10215)]": "Other",
  "[question(73)]": "System Administrator",
  "[question(77)]": "7",
  [...]
}
PARSED ANSWERS
{
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(73)]": "System Administrator",
  "[question(77)]": "7",
  [...]
}

But for the same question and option in a different response, it stays in:

UNPARSED ANSWERS
{
  "[question(54), option(10210)]": "We're considering Puppet as a configuration management solution, and I wanted to learn more about it.",
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(54), option(10215)]": "Other",
  "[question(73)]": "System Administrator",
  "[question(77)]": "10",
  [...]
}
PARSED ANSWERS
{
  "[question(54), option(10210)]": "We're considering Puppet as a configuration management solution, and I wanted to learn more about it.",
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(54), option(10215)]": "Other",
  "[question(73)]": "System Administrator",
  "[question(77)]": "10",
  [...]
}

UNPARSED ANSWERS
{
  "[question(54), option(10211)]": "I already use Puppet for job related tasks, and wanted to develop my skills.",
  "[question(54), option(10214)]": "I work with people who use with Puppet, and wanted a better understanding of what they do.",
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(54), option(10215)]": "Other",
  "[question(73)]": "Other (please specify)",
  "[question(73), option(\"10241-other\")]": "Helpdesk Intern",
  "[question(77)]": "7",
  [...]
}
PARSED ANSWERS
{
  "[question(54), option(10211)]": "I already use Puppet for job related tasks, and wanted to develop my skills.",
  "[question(54), option(10214)]": "I work with people who use with Puppet, and wanted a better understanding of what they do.",
  "[question(54), option(\"10215-other\")]": "<WRITE IN RESPONSE>",
  "[question(54), option(10215)]": "Other",
  "[question(73)]": "Other (please specify)",
  "[question(73), option(\"10241-other\")]": "Helpdesk Intern",
  "[question(77)]": "7",
  [...]
}

[apurvis]

those parsed an unparsed look the same? did you copy/paste the same?

[kjhenner]

Nope, that's what I was getting.

I think I figured it out for real this time, though. Updated the PR: https://github.com/jarthod/survey-gizmo-ruby/pull/77/files

[apurvis]

i'm confused because the parsed answers are arrays of Answer objects and don't have the crazy [question(54), option(10215)] format?

[kjhenner]

Oh yeah, I was pulling it out of the answers method itself before that .map block at the end.

Edit: Sorry, not very clear. It's from within the parsed_answers method for the Response class, so just comparing before and after filtering the key value pairs without converting them into Answers.

[kjhenner]

Cool, looks like this is resolved with https://github.com/jarthod/survey-gizmo-ruby/pull/77. Thanks for your help wtih this @jarthod and @apurvis, and for being patient with my false starts! 👍

[jarthod]

Great, thanks for the feedback.

[apurvis]

thx for fixing a pretty hard to find bug!