Campaign proposal: Open Code Pledge

[rmib200]

• Campaign co-creator/s: Manuel Illanes @rmib200
• Rationale: Many areas of Neuroscience are critically dependent on computational tools for things like analysis of large volumes of data, tasks, visualization, etc. Right now, we can say that sharing code is the exception when new research is published. Only certain journals require papers to include a statement on whether programs are available. We belive that all jounals should adopt policies that strongly encourage or mandate the sharing of code. Sharing all code and data needed to allow others to reproduce our work should be considered a regular practice. At least the key parts when this is not possible or practical.

• Action: (1) Whenever you publish an article, also publish the code you used for the analysis and implementation via an open, freely available platform (e.g., Github, OSF, papers with code). (2) Generate a DOI for the code and link to this DOI in the article. (Optional)
  Make sure that at least one or more figures from the manuscript are reproducible.
  If you think the work is too computationally intensive, one small "toy" example could be used to demostrate it.
  * Also, you can upload a config file when possible.
  We can demonstrate, leading by example, the obvious benefits of adopting these practices like more transparency, reliability and faster adoption of new methods. This will improve the future work produced on Neuroscience, even if the code doesn't fully work in 20 years. Because it will still be more precise and complete than text in a paper.
  Open-source code can be used for promotion, reference letters, and mentions. Take into account that right now, the limitation is cultural rather than technical so let's take a part in changing that. Let's free our knowledge.

If you can provide feedback on anything that could be done differently to benefit the field or get more people motivated to do anything regarding science, please don't doubt to comment. For Example, we could start by listing all the possible best practices for this.

• Eligibility criteria: All fields (but we will target neuroscience researchers initially).

• Optional anonymity: Yes.

• Threshold: 100 500 (calculated separately for each field)

• Pledge duration: 18 months. Lifetime

[CooperSmout]

Great idea! Thanks for proposing this campaign! Some initial thoughts:

• Action: publish ALL your code. ALWAYS.

I think we could make this more specific, e.g. ask people to publish code in an open (indexed?) repository (e.g., OSF, Github). Also, I wonder if will scare people off by asking them to publish ALL of their code (as great as this would be!). What do you think about instead asking people to publish all code related to any papers/preprints they publish?

• Eligibility criteria: Mainly for researchers on BCI and Neuroscience but could apply for all fields.

In previous campaigns we've included a drop-down box for people to indicate their field. This way the campaign can be open to all researchers, but pledges only activate when there is a critical mass of support in your field. But I'm also happy to target a campaign specifically at Neuroscience researchers, if you prefer. This is all experimentation after all, and we are finding out what works as we go :)

• Pledge duration: ? Not sure. Should be permanent.

Totally agree that permanent would be best, but wonder if we might get more people to sign if we only ask for a fixed period (e.g. 1 or 2 years)? I can imagine some people thinking "Sure, but what if I change labs and my new PI doesn't support open code?". Again, it's just about lowering barriers to signing, and we can of course follow up with another (bigger) campaign in a years time in which we ask for a longer/permanent duration.

[sina-mansour]

That's a great idea!

I'm thinking, a good way to achieve this goal is to create a GitHub repo for this purpose. Let me elaborate on what I mean.

I'm thinking if a template github repo is made with github pages web integration. We can ask researchers to fork that repository and add their code on top of that. This way a template for data and code sharing comes along with the open code proposal/pledge.

This way, ideally after a while, there will be quite a few forks of the repositories that have a similar format and are all linked to a separate paper.

This may require a bit of development to prepare the template (it should contain a proposed directory structure, and a landing page which loads through github pages and is linked to the published article).

[CooperSmout]

Hi and welcome to the forum @sina-mansour :) that's a cool idea! You mentioned using a similar format -- did you have something in mind? I'm aware of standards for neuroimaging data (e.g., BIDS), but not for neuroimaging code.

One thing I'm wary of is making these campaigns too complex, because various people have advised me to keep campaigns as simple as possible to reduce barriers to entry. Basically, every requirement you add gives people an opportunity to misunderstand/pause for thought/disagree, and can ultimately mean they don't take the pledge in the first place. E.g. for this idea, I can imagine some people might be happy uploading their code to OSF, but as soon as you start talking about 'forking' and 'Github pages' they will just tune out and we lose them. But I wonder if this could work as a suggestion (e.g. linking to a template as your suggest), rather than a requirement? That way people would still be free to use other methods, if they prefer, and we don't lose them to confusion.

[rmib200]

Thanks for all the feedback, I edited the details of the original a little. I guess the main goal is to encourage people who is about to publish their research to also add the code they wrote for greater chance of replicability. Report the version of the libraries, OS, docker containers, etc. Still looking for more feedback on this.

[I-Khormi]

I support this idea of publishing the code. To keep this idea simple, I think if the researchers/authors put a link in the article paper to take you to their code either in GitHub or the institutional website or personal page. It will also help new PhD students in the Neuroscience field to learn and practice to reproduce their work.

[CooperSmout]

Hi @I-Khormi, thanks for showing your support (helps us know which campaigns will have traction when we launch!)

I think if the researchers/authors put a link in the article paper to take you to their code either in GitHub or the institutional website or personal page.

Absolutely, best practices is for authors to link to their code in the paper. I wonder if this should also be part of the pledge, i.e. to provide a link in their paper?

The cool part about some repositories (e.g., OSF) is that you can also get a DOI for your data, so if people were to upload their data AND code, they could get a DOI, which they link to in their paper, and gives people a chance to cite their data/code

[rmib200]

@CooperSmout great!! And OSF asks you for anything on particular to let you get a DOI? A link for the paper would be a great idea.

[rmib200]

@I-Khormi yes! Authors can put a link of the repo on the same paper to facilitate access to the code.

[nsunami]

Great campaign! My thought is whether it would be better to frame the message positively, like "publish all research with code". Framing it as "stop doing" definitely catches attention, but it may be easier for people to put their names with a positive message (since we generally do not want to be told to stop something 😄). Just a thought!

[rmib200]

@nsunami Great idea! Yes, people will definetly will be more likely to put their names with a more positive tone. 😄 Thanks!

[rmib200]

Description updated!

[CooperSmout]

OSF asks you for anything on particular to let you get a DOI?

The project just needs to be public I believe

[CooperSmout]

better to frame the message positively, like "publish all research with code"

Great suggestion @nsunami, totally agree! Maybe we should edit the campaign proposal template to prompt positive framing in future proposals. For this campaign title, what do people think about "Open code pledge"? This would fit with the other live campaign titles, which are all quite snappy and just focus on the targetted behaviour. Then the subtitle can provide more details, e.g. "Pledge to share code underlying every article you publish, along with X of your peers"?

[rmib200]

Great, the official name will change to Open Code Pledge then. We need to make sure people feel encouraged to sign so this positive tone is key. Maybe a video could be done to encourage people to share the idea with peers and every time someone publishes a paper with code they can get a retweet from other people involved in this campaign.

[nsunami]

The title sounds great!

... Maybe a video could be done to encourage people to share the idea with peers and every time someone publishes a paper with code they can get a retweet from other people involved in this campaign.

I like the idea of incentivizing the signers in the end (like a Twitter bot to retweet open code papers of those who signed). Perhaps we cannot do this for anonymous signees. But I really like the general idea of getting a reward and celebrating the activation of the pledge.

[CooperSmout]

Great, the official name will change to Open Code Pledge then.

Done :)

Maybe a video could be done to encourage people to share the idea with peers and every time someone publishes a paper with code they can get a retweet from other people involved in this campaign.

These are both great ideas. A video would be great, and now that we have a little bit of funding we can actually pay for someone to make one. If you find any good animators please let me know!

Would also be great if people can retweet. IMO the trouble is motivating action! There's two different timelines here: (1) while collecting pledges, and (2) after pledges become active and people start publishing their papers/code. I think the latter will be easier to get tweets/retweets, because it's a celebration of their paper and we can use the mailing list to ask fellow pledgers to retweet (note @nsunami at this point anonymity isn't a problem, because all names get published when the pledges go live).

The more immediate concern is getting people to promote the campaign before pledges activate, as there's less activity and reason to get excited. Point in case, the Preregistration Pledge had an initial flurry of activity but has since stalled at ~70 pledges (in Psychology), and getting these final few pledges is proving diffficult as I've exhausted most of the communication channels I can think of (if you can think of any extra, especially in Psychology, please add them to the Communication Strategy document).

[CooperSmout]

I just attended an unconference on "Why don't we share data and code?" lead by @dylangomes at the SORTEE conference, where we discussed this proposal and a potential Open Data campaign (which has now been proposed here -- please help develop it!!). Just thought I'd summarise the main points relevant to this campaign (credit: unconference attendees listed in this document; which also contains more discussion points)

• 100 pledges seems like a reasonable target for the field of ecol/evol biol (we should get at least 13 from the unconference)
• would be good to include advice on campaign page about how to upload code
• best to give multiple options for where to upload code, as people have different preferences. Main options raised were: Dryad, Zenodo, Github (though concern over deleteability of accounts), OSF
• some participants preferred the idea of a combined data+code pledge, others preferred to keep pledges separate because not always possible to share data alongside code. I think a neat solution to this could be to keep the pledges separate, but provide a checkbox in each pledge so that people can easily pledge to both campaigns at the same time if they wish. To achieve this, we would just need to release both campaigns at the same time.
• would help to highlight open code practice if we were to write a paper outlining the reasons for/against, ways to upload, response to criticisms etc. If we do this, we could also link to this campaign to demonstrate community support for Open Data.
• useful to include links showing scarcity of data, e.g. this paper for ecol/evol biol
• blog post by @luispedro on open code as a collective action problem
• open research policy

More details in the unconference document for anyone interested (e.g. list of reasons people might not upload data/code)

[rmib200]

@CooperSmout I recently got up to date with the documents. I was very busy last weeks due to the Neuromatch. (Which by the way was great, I hope to work on Neuroscience the rest of my life (: ). I think that the reasons listed on why not to share code on the blog and the doc are important but not really an issue. They were rather expected actually. The benefits of this campaign is that it is completely democratic and will set a starting point for more change with the collaboration of many. After all, we are here to help, not to shame. I was thinking that maybe we should work with the first few people interested in joining the campaign, so in that way we make sure the code is excellent (at least to a point) and we can use it as examples on how to start publishing the code more independently with less insecurities, knowing there's a community behind. The benefits will outweigh the cons rapidly. Better programs, better libraries, more room for meta-analysis and more incentive for students to learn how to code. I don't know any animators :( but it shouldn't be so hard to find. If we have a script or a rough idea on how the video should look like I'm pretty sure we can find someone. I'm pretty happy for the funding! What should be the next steps for this campaign?

[CooperSmout]

I was thinking that maybe we should work with the first few people interested in joining the campaign, so in that way we make sure the code is excellent (at least to a point) and we can use it as examples on how to start publishing the code more independently with less insecurities

Absolutely, would be great to have good examples that others can follow in their own practice.

I think this idea could gel nicely with an idea that @malvikasharan of the Turing Way project recently suggested to me. Her idea was to try and find an 'open science champion' to represent each FOK campaign, who we can highlight as an exemplar of the practice in question. Our campaign could bring them recognition for their open science efforts (which are all too often ignored) and in return we could leverage their expertise to (1) teach others how to do it, (2) provide examples to follow (as per your suggestion @rmib200), and (3) answer questions from the community. An effective way to do this could be to organise a workshop, in which our champion teaches the participants how to do the practice, shows some good examples, and answers any questions that participants might have. I was also thinking that we could potentially coincide the workshop with the campaign launch, so that after our champion has given her presentation, we could introduce the campaign, invite feedback, finalise the campaign design and launch it right then and there. Hopefully many of the workshop participants would be willing to sign, and promote the campaign more widely, but at the very least it would be good advertising for the campaign and an opportunity to collect critical feedback (plus highlight our champion's awesome work (: )

What does everyone think of this idea? Would you join us at a workshop @rmib200 @sina-mansour @nsunami @I-Khormi?

Some considerations

• it would be great to select/identify champions from diverse backgrounds, to counteract the usual biases in academia
• maybe we should select a different champion for each research field? E.g. one champion from neuroscience, another from biology, etc., who can teach about best practices within their field? This would mean either hosting a separate workshop for each research field (which would be a lot of work)... or perhaps we could use breakout rooms to allocate people into different rooms within the one workshop?
• Lenny Teytleman (Biologist) springs to mind as a potential champion for this particular campaign, since he founded protocols.io and might be interested in promoting open code. I had contact with someone from Protocols a while back, so could follow up on this if we go with this idea

[CooperSmout]

Ps. @malvikasharan also suggested we could rename this campaign 'Citeable code', to make the benefits of sharing code (with a DOI) explicitly obvious

[CooperSmout]

Hi all, I just received this email from the OHBM organisers -- looks like we'll get to publish a short piece on this campaign (and Project FOK) in OHBM Aperture :) I'm planning to start a Latex/Overleaf document to draft the piece, so let me know if you'd like to help write it and I'll grant you access.

I'm also wondering if this could be a good opportunity to launch the campaign, e.g. by linking to the live campaign in the published article. If so, the article is due 10 Sept, so we would have 1 month to finalise the design and post the campaign to the website.

"As we mentioned during the opening and closing sessions of the event, we are joining forces with the OHBM Aperture journal to create a citable research object out of all the projects from the Brainhack. As project leaders, we wanted to ask you first to confirm that you would be happy to contribute to this paper and then to send us a short submission summarizing your project. The submission should include all the co-authors (everyone who contributed to the project) and the related affiliations, clearly indicate the title of the project, and also describe:

• the motivation for your project;
• what you have found/achieved during the hackathon;
• the implication of your results and/or future steps for the project.

Each submission can have 1 figure and up to 10 citations. The project summary should not be longer than 400 words (without counting title, citations, or authors/affiliations). If possible, we would ask you to send all your submissions by Friday the 10th of September. If you think that this time frame does not work for you, let us know. We look forward to hearing from you and making this Brainhack project paper happen!"

[dylangomes]

I apologize for the radio silence on this Cooper. If it is helpful to you, I would be happy to contribute to writing or helping others with code sharing during a workshop. If not, that is okay too. So glad to see all these ideas and work on these issues!

Dylan Gomes

On Wed, Aug 4, 2021 at 6:31 AM Cooper Smout @.***> wrote:

Hi all, I just received this email from the OHBM organisers -- looks like we'll get to publish a short piece on this campaign (and Project FOK) in OHBM Aperture :) I'm planning to start a Latex/Overleaf document to draft the piece, so let me know if you'd like to help write it and I'll grant you access.

I'm also wondering if this could be a good opportunity to launch the campaign, e.g. by linking to the live campaign in the published article. If so, the article is due 10 Sept, so we would have 1 month to finalise the design and post the campaign to the website.

"As we mentioned during the opening and closing sessions of the event, we are joining forces with the OHBM Aperture journal to create a citable research object out of all the projects from the Brainhack. As project leaders, we wanted to ask you first to confirm that you would be happy to contribute to this paper and then to send us a short submission summarizing your project. The submission should include all the co-authors (everyone who contributed to the project) and the related affiliations, clearly indicate the title of the project, and also describe:

• the motivation for your project;
• what you have found/achieved during the hackathon;
• the implication of your results and/or future steps for the project.

Each submission can have 1 figure and up to 10 citations. The project summary should not be longer than 400 words (without counting title, citations, or authors/affiliations). If possible, we would ask you to send all your submissions by Friday the 10th of September. If you think that this time frame does not work for you, let us know. We look forward to hearing from you and making this Brainhack project paper happen!"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FreeOurKnowledge/website/issues/30#issuecomment-892661095, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHYRLDO3GLVNKEJFCO77RB3T3E6LHANCNFSM462AVTIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- Cheers, Dylan Gomes

[I-Khormi]

Hi Cooper, I am happy to join your team and help with this document.

Best regards,

Ibrahim

[sina-mansour]

Hi Cooper,

Please accept my apologies for this belated email.

I think the idea of having 'open science champions' sounds really interesting, it could serve as a stage to promote open science practices. I was also thinking of another possible way to promote such practices too. I'm thinking we could create a gallery of scientific work that can resemble and showcase the open science practices being promoted (with a short note explaining the work and links to the openly shared content). This way, the article will get more publicity, and in return, someone who's thinking of implementing open science practices could have plenty of examples to look into. This gallery could be selected and updated to include all the diverse methodologies used in research (different types of studies, programming languages, data sources, etc.).

I'd also be happy to contribute to the writing of the manuscript, and/or help to organize workshops (i.e. using technologies like git and zenodo to make code openly available and citable) to promote the campaign.

Kind regards, Sina

On Thu, 5 Aug 2021 at 10:50, Ibrahim Khormi @.***> wrote:

Hi Cooper, I am happy to join your team and help with this document.

Best regards,

Ibrahim

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FreeOurKnowledge/website/issues/30#issuecomment-893077999, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD235MVIZW3FUFGHN7DOSVDT3HN6DANCNFSM462AVTIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[CooperSmout]

Awesome thanks all. I'll set up a Latex document in the next couple of weeks and link it to you when it's ready.

I think the idea of having 'open science champions' sounds really interesting, it could serve as a stage to promote open science practices. I was also thinking of another possible way to promote such practices too. I'm thinking we could create a gallery of scientific work that can resemble and showcase the open science practices being promoted (with a short note explaining the work and links to the openly shared content). This way, the article will get more publicity, and in return, someone who's thinking of implementing open science practices could have plenty of examples to look into.

I like this idea, particularly if we tie it into the campaign to celebrate our pledging community. In the preregistration pledge campaign we're planning to link to people's preregistrations once they complete them, to increase visibility and celebrate those people who complete their pledge. So we could do something similar here, linking to people's code (and relevant papers) once they share their code. Does that fit with your vision?

I'd also be happy to contribute to the writing of the manuscript, and/or help to organize workshops (i.e. using technologies like git and zenodo to make code openly available and citable) to promote the campaign.

Thanks for the offer! Will certainly be needing some help on the workshop, if we go that direction. I was also thinking we could reach out to OHBM and see if they can support a workshop, either with resources/expertise.

[CooperSmout]

By the way, I reposted the champion/workshop idea here to open it up for discussion, since it's a general idea that could relate to other campaigns as well (also, curious to test out this new 'Discussions' feature in Github).

I also posted a new idea, about publishing pledges in an indexed journal, with a view to increasing visiibility and creating citable documents reflecting people's pledges. Would love to hear if you have any thoughts on that idea, as it's something that we could potentially do for this campaign.

[rmib200]

All these ideas are incredible. @CooperSmout What should be the focus of the paper? Best practices? Future directions? Maybe a global scope of democratic science practice possibilities? I'm very excited that this is getting attention!. Regarding the champions, when shoud we start sending the invitations? can we make a database with possible names and availabilities? And when the workshop could take place? This year? Also, I never used latex, but I'm ready to learn!.

[rmib200]

Ps. @malvikasharan also suggested we could rename this campaign 'Citeable code', to make the benefits of sharing code (with a DOI) explicitly obvious

Yeah, I see the potential of making the title more explicit. But I think that just 'Citeable code' is not very clear. Maybe something like 'Citeable code 4 science' or 'Citeable code 4 neuroscience' something like that. Does anyone has any ideas maybe? please feel free to share.

[dylangomes]

One option is

Open code: reproducible and citable

You can add science if you want:

Open code in science: reproducible and citable

OR

something like

Open science: reproducible research and citable code (the alliteration here is nice)

I think the word reproducible is more important to include, in my opinion. Being citable is a nice incentive, but the goal is to have reproducible work - is it not?

Cheers, Dylan Gomes

On Mon, Aug 9, 2021 at 7:44 AM Manuel Illanes @.***> wrote:

Ps. @malvikasharan https://github.com/malvikasharan also suggested we could rename this campaign 'Citeable code', to make the benefits of sharing code (with a DOI) explicitly obvious

Yeah, I see the potential of making the title more explicit. But just 'Citeable code' is not very clear. Maybe something like 'Citeable code 4 science' or 'Citeable code 4 neuroscience' something like that. Does anyone has any ideas maybe? please feel free to share.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FreeOurKnowledge/website/issues/30#issuecomment-895283787, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHYRLDIAU3BNUYCL3XAIYFTT37STDANCNFSM462AVTIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[CooperSmout]

Hi all, sorry for the delay on this, I've been swamped with other parts of the project. But I've now drafted a first version of the Aperture submission, which you can see here (I ended up just using Google Docs because it was easier and my Zotero references were already set up). I'm hesitant to share an edit link here so please contact me via email if you would like edit rights.

I was also wondering if those of you who are keen to join the paper would consider becoming 'ambassadors' for the campaign? This is something I've been wanting to implement since the beginning of FOK but haven't had the time or volunteers to do so. The main idea here would be to increase visibility on the campaign, using small but repeatable actions (e.g. retweeting each others tweets about the campaign). I'm not anticipating a big time commitment, perhaps 15 mins/week -- the main thing is to just keep the campaign fresh in people's minds (and also our own minds, so that we remember to mention it where relevant).

Since this is another generic idea that can apply to all campaigns, I've started a new thread here to discuss the idea. Would appreciate any comments or suggestions you might have!

[CooperSmout]

All these ideas are incredible. @CooperSmout What should be the focus of the paper? Best practices? Future directions? Maybe a global scope of democratic science practice possibilities?

Let me know what you think of the scope in the draft version (I've added you because I already have your email)

Regarding the champions, when shoud we start sending the invitations? can we make a database with possible names and availabilities?

Yes a database would be great. Let's start entering any names we think of here for now.

And when the workshop could take place? This year?

Possibly later this year, e.g. November? I'm feeling very stretched across this and our new project, so timing will depend on how much support we get (e.g. see the new ambassador role) :) I already emailed the OS-SIG to ask if they can offer any support, but still waiting to hear back. I mentioned the upcoming workshop in the paper, but if we have a clearer idea by the time they publish we can update the text.

[CooperSmout]

I think the word reproducible is more important to include, in my opinion. Being citable is a nice incentive, but the goal is to have reproducible work - is it not?

Agree that reproducibility is the goal, but the challenge is getting people to actually pledge. For some reason it's very difficult to get people to actually click a button and commit. So I believe Malvika's idea was to offer a clear incentive in the title that would be self-evident. But I also don't want to misrepresent her arguments, so will ping her again and see if she had any other thoughts on the matter -- @malvikasharan

Personally, I like 'open code pledge' because it's super obvious and also fits with the names of previous campaigns, which all used the targeted behaviour followed by the word 'pledge' ('preregistration pledge', 'platinum open access pledge', etc.). But our newest campaign proposal will be breaking this format, so I'm not beholden to it.

[malvikasharan]

Hi all, (thanks @CooperSmout for pinging), my suggestion on emphasising "citability" was to focus on people who would want to adopt the practice of openness keeping added incentives of gaining citation. "Reproducibility" is a process to lead to high-quality research and "open" allows a much bigger impact than the research itself by allowing others to not only reproduce but ensure the quality, conduct a new study or ask new research questions. These are deeper concepts that a researcher would not think about when they are not incentivised for their extra work.

At the end of the day you want people to take action - so whatever works to get the idea across (open, reproducible, citation, changing culture) is what you should use.

[dylangomes]

Hi all,

A bit late to this part of the conversation (and perhaps that ship has sailed), but I wonder if it makes sense to have two options for the pledge - the 18 month "trial" option, and a "life" option. Perhaps this makes the pledge overly complicated (so just ignore this if it seems so), but re-pledging every 18 months also sounds somewhat tedious to those of us who are dedicated to publishing code (and data) with everything we ever do. Thoughts?

[CooperSmout]

I wonder if it makes sense to have two options for the pledge - the 18 month "trial" option, and a "life" option.

Hmm great question! I had imagined that a future campaign might ask for a more serious commitment, e.g. the life option that you suggest, but we could also include it in this pledge as an additional checkbox. We did something similar in the Registered Reports petition, allowing people to indicate if they're happy to be auto-included in future letters (most people were happy to be included, which is useful data to have).

I guess one concern is that we lose the opportunity to re-engage people after 18 months and get them to re-affirm their commitment (e.g., some people might leave academia, but we'd never know that with a lifelong pledge). At the same time, we could increase the scale/scope of the next campaign, using the previous community to help design/promote the next version, which puts it in the forefront of their mind and may increase engagement.

But I can also see that frustrating people who just want to commit for life and get on with it.

Honestly not sure what to do here, but the good news every option is new territory so we will learn something either way :) what do other people think? would you take the lifelong pledge if given the option?

[CooperSmout]

I just discovered this paper describing an open source pledge for neuroscientists with 250+ signatures. Copying the pledge here for comparison:

we pledge to release promptly, completely, and freely all computer code, model scripts, and parameters necessary to reproduce the analyses and simulations from any of our new publications. We will make all software applications (tools, libraries, etc.) we develop for experimental data analysis or model construction open source at time of publication, whether or not the application is the main subject of the paper. Furthermore, if and when asked to serve as peer-reviewers, we will henceforth ask authors about the availability of any code they have developed for data analysis and modelling which is essential to reproducing the results of their paper and require that this be shared publicly upon acceptance.

The main difference seems to be that we are asking people to make their code available AND citable here, whereas the other pledge is primarily about making it available (but also stipulates that peer-reviewers should request that authors make their code available). So I think it still makes sense to run our proposed campaign, for the following reasons:

• it's a more committed pledge than the previous pledge, so can be seen as a natural progression
• we're using the 'conditional pledge' format (i.e., only take action once X people sign), which is yet to be tested within academia and could be a useful mechanism for solving a range of collective action problems
• it will get more people onto the FOK platform, who we can then inform about other existing campaigns we run in the future
• we can build up to bigger and bolder campaigns in the future, e.g. targeting more specific behaviours (e.g. FAIR code) or more people (e.g. 10000 pledges)
• we can ask people to sign a parallel Open Data Pledge (TBD) when they sign this one

Some questions:

• Do we get rid of the 18 month pledge duration and just go with a lifetime pledge? (I'm inclined to say yes, given the success of this previous pledge)
• Do we let people choose when their pledge becomes active: (1) immediately (akin to the previous pledge) vs (2) only if X people have signed?
• Do we increase the critical mass target for our campaign (X), given that there are already ~250 neuroscientists who will probably be happy to sign without a threshold? Perhaps we should aim for 500 (or 1000) researchers rather than 100?
• Do we include the same statement about peer review?
• Do we borrow their name and call this campaign something like "Open source and citable code"?

[orchid00]

Hi I've been following this conversation for sometime now.

I just read this note that can impact the name on this pledge, please consider:

« open code « is an unknown neologism in the world of software.

We recommend using the terms « free software » or « open source software », which are well defined and have been used for decades. The definition of these terms is maintained by specialized NGOs (OSI, Open Source Initiative, and FSF, Free Software Foundation). They correspond to a real use in the world of software development and IP law. Some institutions of the European Union promote the aggregative term of « FLOSS », for Free / Libre and open-source software (see https://cordis.europa.eu/publication/rcn/9015_en.html), that can also be used. In the framework of Open Science, the term « Open Source Scientific Software » could therefore be used.

For reference: https://www.ouvrirlascience.fr/about-the-proposal-for-software-indicators-in-open-science-monitor-3/

my personal vote is for the name would be "open source software for science" or just "open source software"

[CooperSmout]

For reference: https://www.ouvrirlascience.fr/about-the-proposal-for-software-indicators-in-open-science-monitor-3/

Thanks for the link!

We recommend using the terms « free software » or « open source software », which are well defined and have been used for decades

This is a good point and I think where possible we should use established terms and tie into that history. I like the term "open source" (as per my above post, suggesting "open source and citable code") , but am hesitant when it comes to the term "software". Personally, I don't know any neuroscientist who uses this term to refer to their experimental/analysis code, and if I were to see that word in the title I think my first impression would be that the campaign was about specific programs/software packages/programmers, and therefore not for me and my boring old 'code'. But would be good to get more opinions on this as it may be specific to my bubble.

I think the main goal of the title is to clearly indicate what the campaign is about, rather than be 'technically correct'. So I'm inclined to go with whatever is most easily understood and commonplace. Then in the campaign description we can clarify exactly what we mean, and this would be a good place to reference the "open source software" movement.

For context, the OHBM OS-SIG website refers to 'Open Code' and I found at least a couple of official looking sites that use the term "open source code". Personally I like "Open Source and Citable Code" because it includes "Open Source", "Code" and also indicates that we are getting DOIs for our pledges (which is a differentiating feature from the existing pledge I just discovered)

[CooperSmout]

Returning to the question of scope for this campaign (and also paper):

• Eligibility criteria: Mainly for researchers on BCI and Neuroscience but could apply for all fields.

In previous campaigns we've included a drop-down box for people to indicate their field. This way the campaign can be open to all researchers, but pledges only activate when there is a critical mass of support in your field. But I'm also happy to target a campaign specifically at Neuroscience researchers, if you prefer. This is all experimentation after all, and we are finding out what works as we go :)

I'm leaning toward targeting this campaign specifically at neuroscientists, with a view to expanding to other fields once we reach the critical mass in neuro. Our main goal right now should be to reach the 'critical mass', because we still haven't had a campaign reach the target and demonstrate the FOK concept in action. We have a great opportunity to target the neuro community through this OHBM Aperture paper, our current ambassadors (most of whom are in neuroscience) and also the previous pledge to build upon (the founder of that pledge suggested it was a strength of that campaign that neuroscientists could see each others names and recognise them).

So I think it makes sense for us to really focus in on neuroscience now, with a view to expanding into other fields later. We can still include a dropbox for people to indicate their field when they pledge, which will let us scope out interest in other fields ahead of tailoring new campaigns for those communities (e.g. with a new targets for each field, based on the likely level of uptake -- this was a problem in the Preregistration pledge, where we set 100 as the target for all fields despite varying levels of interest).

So with this in mind, I'm considering changing the title to:

"Open source and citable code for neuroscientists"

because it clearly builds upon the previous pledge, while clarifying how this pledge is an 'upgrade' and also the personal incentive for making code available (citability). We need to decide this prior to submitting the paper so please let me know any feedback/objections ASAP.

[I-Khormi]

Hi Cooper,

I totally agree with your point about focusing on the Neurosciences community until we reach the primary goal. Then the neuroscientists will be the best model for other researchers in different fields. Also, the new title gives a clear vision of the campaign.

Thank you.

Ibrahim

[dylangomes]

My take on your questions: Do we get rid of the 18 month pledge duration and just go with a lifetime pledge? (I'm inclined to say yes, given the success of this previous pledge) -- Yes, I am in favor of a lifetime pledge (this was my previous point about continually pledging over and over if one is committed).

Do we let people choose when their pledge becomes active: (1) immediately (akin to the previous pledge) vs (2) only if X people have signed? -- Immediately makes the most sense to me. It will be the easiest for folks to follow through with (it is an entire extra step to follow the campaign to see how many people pledged etc. and then gauge when it starts. I say make it simple and start right away. No one is going to pledge who doesn't want to do it (at least in theory). So making it conditional on X people signing doesn't make sense to me because we still want them to publish code.

Do we increase the critical mass target for our campaign (X), given that there are already ~250 neuroscientists who will probably be happy to sign without a threshold? Perhaps we should aim for 500 (or 1000) researchers rather than 100? -- I guess it depends on your scope. You've mentioned restricting this to neuroscience so it is difficult for me to gauge how much interest there would be. I think getting 500 (or 1000) EcoEvo people to pledge might be relatively easy - there is a lot of movement in this direction. But again, not sure what the number is for neuroscience, but 500 seems totally reasonable to me.

Do we include the same statement about peer review? -- I think that it is a very easy thing to do as a peer-reviewer. The reviewer has a lot of power and requesting code (and data) is the only way to thoroughly assess one's work. So I think it is not only reasonable, but going to be so much more far reaching if the peer review statement is included. I would even go further and add responsibilities for editors, who have even more power. If one believes that code should be open source for their own work, then it isn't too much to ask that individuals also help to propagate the campaign through the peer-review process. I would be happy to help work up the language around this to see what you think?

Do we borrow their name and call this campaign something like "Open source and citable code"? -- I think this is a good, descriptive name

[orchid00]

Feedback:

500 pledges is a good number, considering this only for Neurosciences. I support the idea of having a field to ask for their research field and gauge interest from other fields. This could be the "seed" for the next pledged.

I support the call to action in two ways: 1) Pledgers can either begin sharing their code immediately (2) or wait until 500 others in their filed have taken the pledge. I would be in favour of adding back the http://opensourceforneuroscience.org/ initiative for those who wish to support it, in relation to call to action 1. Being friendly to a very similar effort has value in it. I personally don't see it as any competitive interest. By mentioning it we are saving people's time to find the other pledge and also encouraging them to sign it.

About the name: I like this name to be maintained "The open source and citable code pledge" even if the target focus is the neurosciences community (initially). I think this title will give a stronger message, than if the field is included.

Thanks Cooper, great job!

[CooperSmout]

@sina-mansour please add your affiliation to the paper, submitting later today!!

[CooperSmout]

Leaving this for when we draft the campaign page: Github's guide to DOI's

[CooperSmout]

@dylangomes this organisation seems super relevant for an open code/data campaign in the environmental sciences

[CooperSmout]

Also relevant for when we run an open data campaign: https://zenodo.org/record/3383814