Review of submission 124chazette

[anonreporev]

Hello 124chazette,

I'm going through your paper and artifact.

At a high-level, I understand the contribution of the survey raw data from the paper.

In detail, I have some questions:

1) To view the CSV file, I can either do so online on GitHub or via converting in Excel, delimiting by white space. In either format the file looks relatively normal until line 26. Then there are 4 lines of long-form data. Which responses do theses correspond to? Which respondent and which question? The same type of lines appear later in the file (line 64, etc.) Even GitHub is giving a formatting issue "We can make this file beautiful and searchable if this error is corrected: Illegal quoting in line 1."

2) Some of the question answers are blank? E.g., what is your occupation? And some of the Likert answers are blank. Why? Are some of the questions optional? And if so, can the authors indicate which questions were optional. My understanding is that only completed survey responses (107) are included.

3) It's great that the non-English responses were translated, but this raises more questions. I believe both the original response and the translation are in the CSV file. This can be made more clear, I see only TRANSLATION in the first few columns, not in later columns. Was the translation done by only one person or checked by several? The presence of the original text makes this less important, but perhaps the translation process can be described briefly in the readme.

Thank you for addressing my questions.

nonreporev

[larissamp]

Hello!

1. That's odd. I didn't notice this problem when I was checking the csv file before submitting but I guess I must had overseen it. Sorry for that. Looks like a problem during the conversion from xlsx to csv. The original was an excel file. At the moment, I'm travelling and only have my smartphone with me, which is quite limiting. I have the original xlsx and I converted again using mobile excel to csv. I tried to attach the csv here but github doesn't allow it. Maybe you can take a look and, if needed, try to convert again to csv? RAW_Results_Survey.xlsx
2. Yes, all open-ended questions were optional. Therefore the blank spaces. You can also identify the open-ended questions because they are the only questions with possible blank spaces. Only completed survey responses are included as well: 107.
3. Only columns that needed to be translated to English were translated. Other columns were already in English and, therefore, not translated. The translation was done by one person and checked by another.

Hope that helps!

[anonreporev]

Thanks for your response. The xls looks better but is still a bit tricky to read.

Will you be back to your laptop before the end of the week? I don't think I should change what's in the repo. I would recommend the following:

Converting to and from CSV to check that the conversion works. I think something in the excel is adding some strange delimitation to the CSV file.

What is the heading of column CJ and CK? Some hexidecimal? Fix or explain in the readme.

Highlight in the readme that the original response and the translation is available.

As the Excel is hard to read, also check in a text or md version of the survey questions and all the options (Likert options, sub-lists). Even html would be fine. Then we can map back and forth from the questions to the data much easier.

Perhaps, even though it is not as universal, also upload the Excel in addition to the fixed CSV.

Hope you or your co-authors are able to make these small fixes.

A further thought, you've provided the raw data, but not the coding results from the paper. Is this easy also to share? Further spreadsheets or some in vivo files? This would also be interesting for the community.

Best,

anonreporev

[larissamp]

Hello anonreporev,

First, thanks for the significant feedback!

I could arrange to borrow a computer for half of the day to do the changes. I am afraid this was my only opportunity, though. I will only have access to another computer again on Wednesday, the 17th, in time for any (if accepted) camera ready changes.

But now I think I could meet all your requests. I hope it works! Here are the changes I made:

1. First, regarding the csv file... The problem was that, during the conversion, commas in the questions or answers were also (of course) considered as separators. Therefore, the weird line breaks. I replaced the commas in the text by / and it worked out fine.
2. I corrected the headings.
3. The readme now contains more details, including the requested information about the translation.
4. The questionnaire is available in three versions: pdf, html and rtfd
5. The .xlsx version of the raw data is also now available
6. The coding results and a separate file with more details about the analysis + instructions about how to read the spreadsheet are also now there.

Best Regards, larissamp

[anonreporev]

Great, thanks for your efforts, I'm happy with the changes.

The only thing I noticed is that the README.md file cuts off mid-sentence.

Best,

anonreporev

[anonreporev]

Review

After reviewing the artifact, I can see potential utility for reuse of the raw survey results. The questions in the survey are now clear and easy to read. The presence of the authors codes makes the qualitative analysis comparable at a detailed level, in case future researchers want to compare their codes.

The authors addressed formatting issues with the file and added more supplementary material, making the artifacts easier to understand and use.

As requested, I recommend a badge of REUSABLE.

[larissamp]

Hello @anonreporev ,

Thank you for your helpful feedback. I fixed the problems you indicated with the readme file. I'm glad that the effort was worth it and resulted in the recommendation for the badge reusable.

Is there still something that needs to be done from my side? Please let me know.

Best, larissamp

[timm]

@neilernst : please concur with my badge of "REUSABLE" or change it "SecondReview".