Questions to UKRN

[agboch]

Our response to UKRN survey call and questions:

We have now met with the rest of the team at Norwegian Reproducibility Network (NORRN) and here are our answers to your questions: 1) Yes we would like to be involved 2) We will roll out the survey in all universities and colleges in Norway though NORRN's local nodes and other points of contact. We will make sure the sample is as representative and multidisciplinary as possible. 3) We will be translating the survey to Norwegian to be able to offer the possibility of answering either in Norwegian or English. We also cannot use Qualtrics since this survey is collecting indirectly identifying personal information (combined background variables might single out researchers in Norway) so we will need to move survey questions to a Norwegian survey tool (but it will look relatively similar to the original one). Because of that, could you send us the whole survey content in an editable format (e.g. word or excel doc, we only got qualtrics link)?

We also have a few follow up questions: a) What is the meaning of "Research Groups" in the question "Are you a member of a Research Group at this university/institution?" Are you asking whether they are affiliated with the lab, for example? Or is "Research Group" a specific term in the UK (since you capitalized it). This might be unclear to many researchers here. b) There are no Open Research Working Groups in Norway. Would it be ok to ask this question more generally, e.g. "Are you involved in any open research initiatives, e.g. ReproducibiliTea or open research groups at your institution?" c) Can we use the subset of data that we will collect in Norway for our own research purposes, e.g. to publish a separate piece on open research practices in Norway? d) Could we get an example data output from Qualtrics so that we can compare it with the output we get after we move survey content to our Norwegian survey tool (to make sure they are interoperable)? e) Could we add a couple of additional questions at the very end of they survey (and on a separate page), once participants are done with the part that is set and preregistered? It would be an opportunity for us to collect some more data on the local context for NORRN's future work, e.g. questions such us “Do you want your institution to focus more on Open Science?” or similar.

[agboch]

Response to the questions that we got from UKRN, input from all very welcome (otherwise we will discuss this with @Tamara-Kalan and try to respond to them next week)

"Just to say that we have got ethical approval here for the global dissemination of this survey, so you could use Qualtrics under our ethics and your role would simply be to disseminate the link (like you might with any online survey you see with ethical clearance). We would then clean the data for you and send over. For the translated version, you could amend this in a shared Qualtrics we could provide you, and again this would be covered (i.e., myself, Kait and Emma’s names are the guarantors of the research). However, if this isn’t suitable then of course we can share an offline version for you. Can you please let me know your final thoughts on this so we can plan our next steps?

Thank you also for your important points below. With regards to Point a) and b): we will revise the survey with this in mind and reach out if we need any clarification. Once the survey is finalised we’ll send it round to you with an update.

With regards to Point c): Yes, we can share with you an anonymised version of your countries data for your own training and research practices. It would be nice if Kait, Emma and I could be involved in this, if possible.

d), Yes, we can generate a pilot test response from Qualtrics for checking.

e), There are two issues here. The first and biggest one is that we do not have ethical clearance to include additional questions to any of the surveys, so this would preclude us from doing so anyway. However, the second reason is based on our explicit rationale for the survey, written in the UKRN dissemination: We want the survey to remain brief (< 5 mins) to ensure it is not burdensome for researchers to complete (also considering how many open research surveys there are at the moment). Because of this, we were able to collect over 1000 responses to our UKRN Survey. For this reason, we won’t be adding any additional questions to the survey (but of course, you can build upon this research yourself in future years, using your data as a basis to form new research questions). "

[Tamara-Kalan]

Here are my thoughts: Regarding using Qualtrics under their ethics: I have no experience with using Qualtrics and doing a study where ethics permission is issued in another country. Unless you, @agboch, or someone else have experience, I think we should contact SIKT and ask?

Points a), b) c) and d) are fine, I guess.

d) I completely understand reasoning here. Yet while many surveys are distributed in the UK, Norway is different in that sense. But I'm ok to not add questions this time and, as Charlotte says, we can develop this survey further.

[agboch]

Thanks, Tamara! Here is what we responded now to UKRN:

"We did understand that you have ethical approval for global dissemination but in the survey information and consent part you say the data will be collected anonymously - this is unfortunately not the case for a small country like Norway where in some cases indicating even just a university and department affiliation or discipline together with open research proficiency might single out individuals. So 1) we can't promise complete anonymity in data collection if the survey is structured the way it is now with the current background variables; instead we might need to say something like: "We will not collect any directly identifying information about you (no name or contact information). There might be rare cases when some collected information will be indirectly identifying survey participants (e.g. a combination of answers to some questions" and 2) due to possible indirect identification we probably need to get a green light from Norwegian data protection services before we start data collection, including permission to release the data to the UK. This second point should not be too difficult and shouldn't take longer than one month, but before we do that we need to move the survey from Qualtrics to Norwegian Nettskjema (together with a translation to Norwegian). If we want to share the data from the survey openly we probably also need to include information about that in the information/consent part, so that participants are aware that their data (even if anonymized) will be shared.

So the short answer is yes, it would be great if you could share a document/offline version with the survey content so that we can move it to a local survey tool and translate. We were also curious if other countries/RNs are only disseminating the Qualtrics survey or they have similar issues with privacy like we do? It would be very interesting to know.

Great about revising questions mentioned in a) and b)

About c), thanks for letting us know, we would be happy to collaborate on this.

Finally, thank you for elaborating on e), we understand that it's important to keep the survey short. We will keep additional questions for follow-up surveys.

We also noticed that in the current survey it still says: "Your responses will be used to inform the UKRN's understanding of current awareness and use of Open Research in its Local Network institutions." - that should be maybe changed to "Global RN's understanding"?"

[agboch]

The final email from UKRN where we were informed about data collection procedures:

The below email outlines how to begin data collection for the international Reproducibility Network survey into open research practices, ways to distribute this survey, and authorship requirements. Please start data collection on Monday 26th June and continue until 1st October 2023. An example invite to the survey is attached to this email.

Dear Leads of the Reproducibility Network,

Thank you for agreeing to be part of our international survey of open research practices across Reproducibility Networks (RNs). We have 12 countries signed up to this exciting project, which will provide some very rich data. You will be pleased to know that we are now at the stage where data collection can begin and so in the below email I outline everything you will need to start this process, as well as some guidance and actions. Apologies in advance for the lengthy email, but I hope this contains all of the information you will need. But, of course, do feel free to send any additional questions!

Data collection: First of all, here is the survey link: https://uwe.eu.qualtrics.com/jfe/form/SV_0ibDSD34iOPWvFI

It is entirely up to you how you distribute your survey; whatever is most appropriate for you and your organisation. Some recommendations could be to distribute this link through newsletters and mailing lists, Heads of Departments/Research, Open Research Working Groups, and/or seminars/talks. You may also wish to ask individual/group members of your RN to pass it out within their institutions or departments, but importantly, if they support our data collection, then they would be eligible for co-authorship (see below). An example invite to this survey is attached to this email. Data collection will begin now (June 2023) and end by October 2023, meaning 12-weeks in total. You may want to send periodic email reminders to your colleagues.

Co-authorship agreement: Any person who contributes to the data collection process by distributing the survey link and provides feedback and approval on the final draft of the manuscript will be eligible for co-authorship. You will see from the email thread that there are a lot of UK co-authors as there are typically Institutional and Local Network Leads for the RN at UK institutions; for our international collaborators, feel free to bring other collaborators on board who can help with the survey distribution – if they lead the distribution with you, then they will be categorised as a co-author. Please complete this spreadsheet to include your co-author details: this is essential as we will not know that you contributed if you do not complete this sheet. At this point it is also worth noting that you may also conduct secondary data analyses on your specific country-level data, which we will send to you in an anonymised format, but we ask that you kindly notify us of your plans.

Ethics: This study is approved by both Brunel University and Aston University, as well as other universities which require their own local ethical approval. Please do remind your colleagues of the voluntary nature of this survey – respondents have a choice whether to respond or not.

Inclusion criteria & anonymity: This survey is open to researchers from any discipline and all academic staff, researchers and doctoral researchers (Ph.D. level and above), whether they are aware or have any experience of using open research practices or not. Survey data is completely anonymised at the point of data collection.

We’d like to take this opportunity to thank you for your support on this project, and for progressing open research! Please do get in touch if you have any questions, and we will be in touch periodically to give you updates on the data collection and manuscript progress.

Best wishes,

Charlotte, Kait, and Emma (Research Leads)

[agboch]

UKRN also informed us that in the meantime two other volunteers from UiO signed up to help with the roll out of the survey in Norway and so they joined the survey task force with me and Tamara.