Campaign proposal: Withholding unpaid review work from profitable publishers

[jonmcphetres]

Action This campaign will ask researchers to pledge to refuse to serve as unpaid scientific reviewers from profitable publishers once a threshold of pledges has been received.

Threshold Pledges will activate when 100 researchers have pledged to quit reviewing.

Anonymity Pledges will be optional- people can choose whether they want their names to be public or not.

Rationale

1. Scientific publishers rely on the unpaid labour of reviewers and editors to reap massive amounts of profit. This one is obvious.

2. Publishers have pitted us against each other.
   The publishing industry has created the perfect system that makes everyone feel like they should contribute for free. Authors need editors and reviewers. Editors need to find reviewers. Reviewers (sometimes) need to review papers “for their jobs”. All for free. If one part of the triangle quits, the others suffer. Why do we feel bad about turning down reviews when the publisher is clearly taking advantage of us all?

3. There is no way to improve our review system. There are many problems with peer review, but with the current publishing system it’s nearly impossible to fix these issues. How do you hold reviewers accountable for passing a bad paper? For late reviews? For writing a mean or hurtful review? Stop inviting them? That’s not much of a punishment when it’s just free work.

Some thoughts and Questions

The letter I have created a draft letter that can be returned to the editor when a review request is received. I welcome comments and feedback on it. Although, I think individuals will probably need to tailor it a bit for their own situation. Review response.docx

1. Which publishers/journals to boycott/support? Based on this information (https://www.wischenbart.com/upload/Global50-2018_overview_ToC.pdf) I compiled a list of the most profitable publishers who publish psychology journals. I suggest we boycott the following publishers: Elsevier (RELX Group) Springer Nature (Nature Publishing Group including Frontiers Media) Wiley Taylor & Francis (Informa PLC) Oxford University Press Wolters Kluwer APA (not in the above document) Sage (no published profit information)

1b. Where to review instead? Rather than creating a list of 'blacklisted' journals, which would be very long, it's probably easier to create a 'whitelist' of fee-free/community-friendly journals. This list is under development here. Please add any journals that you think should be on there by editing the document.

2. Do we begin with limiting pledges to psychology/behavioural science? Perhaps we can expand to other fields if there is enough interest.

3. I also think we should continue to write and submit articles like normal, but just stop reviewing. I think it's too much to ask people to quit submitting papers, as well. So I propose that we continue to submit papers to the outlets that we are no longer reviewing for. This will serve a few purposes. First, it will create a backlog of papers that need to be reviewed, keeping the demand for payment steady. Second, we technically do get paid to produce research. Third, this campaign is not about boycotting publishing as a whole (as much as I love preprints), or even about boycotting publishers, so we need to pick our battles. Plus, we all probably want to continue our work.

4. Timeline How long should the pledge stage operate? If we don’t receive enough pledges within a certain period of time, what do we do?

5. How many pledges until the boycott starts? Is 100 a reasonable number to begin the boycott?

Some extra literature and information https://nces.ed.gov/surveys/libraries/aca_data.asp https://figshare.com/articles/Journal_subscription_costs_FOIs_to_UK_universities/1186832 https://alexholcombe.wordpress.com/2015/05/21/scholarly-publisher-profit-update/ https://www.pnas.org/content/early/2014/06/11/1403006111.abstract

[CooperSmout]

Awesome work @jonmcphetres! I think this campaign will get a lot of traction. Some initial thoughts:

2. Reviewers (sometimes) need to review papers “for their jobs”.

I watched a seminar on peer review yesterday where an editor for an Elsevier journal explicitly stated that Elsevier collect data on you as a peer reviewer (number of accepted/rejected, lag time etc), and use this to evaluate you for potential editorial roles. So I would remove the word 'sometimes' from this sentence :)

Do we begin with limiting pledges to psychology/behavioural science?

Great question. We're also considering this in the Registered Reports campaign thread (#8). I should preface by saying that the eventual plan is for the platform to classify people automatically (using ORCID profiles) and display statistics for each field separately, but we need a developer to make this happen so for now we're stuck with simple metrics that just show the total number of pledges.

Pros of limiting pledges to psychology:

• we can clearly articulate which journals are 'allowed' vs not (as you have above) -- doing this for all fields would be a nightmare
• rates of dissatisfaction with peer review are highest in psych, so should get relatively high adoption
• people can self-identify as psychology researchers before pledging
• it actually solves the collective action problem, which operates at the field level (i.e. if pledges activate and you're the only psych researcher in the cohort, you're effectively acting alone!)

Cons:

• gets less people involved in the movement
• cannot 'test the waters' in other fields and find out which ones would have the most traction

So at this stage I think to limit to psychology, and we can host extra campaigns for other fields down the track.

[jonmcphetres]

@CooperSmout Thanks for the comments!

One thing:

2. Reviewers (sometimes) need to review papers “for their jobs”.

I watched a seminar on peer review yesterday where an editor for an Elsevier journal explicitly stated that Elsevier collect data on you as a peer reviewer (number of accepted/rejected, lag time etc), and use this to evaluate you for potential editorial roles. So I would remove the word 'sometimes' from this sentence :)

What I meant was that I am not aware that reviewing is necessary for one's job as a grad student/postdoc/professor (or even in non-academic jobs). I have never had any sort of evaluation or requirement that I engage with unpaid reviewing work and I don't know of others who have that requirement. Becoming an editor can benefit promotion but it is not required.

I agree, though, limiting to psych is better for the reasons you mentioned.

[CooperSmout]

I am not aware that reviewing is necessary for one's job as a grad student/postdoc/professor

Very good point, it's more of an implicit expectation than an explicit one, which should help us in getting pledges.

How long should the pledge stage operate? If we don’t receive enough pledges within a certain period of time, what do we do?

Good question. We could either take the campaign down, or consult with the community of pledgers to see if they want to extend the deadline (suggested by @samgrobson in the Registered Reports thread #8)?

Is 100 a reasonable number to begin the boycott?

Sure... I guess :) This is all new ground for the project, so really not sure. Personally I'd really like to see a short-term win for the project (i.e. reach threshold), so that we can generate some publicity and set the stage for bigger campaigns in the future (e.g. same campaign but 10x person threshold), which would suggest a small target. But then, if we go too small it might be seen as having little-to-no effect and fail to give some people the assurance they need to sign. 100 seems like a reasonable target for now (and has a nice ring to it), but let's leave it open to revision later when we have a better idea of community sentiment. On a related note, I think it would help each campaign to have some number of people committed to signing prior to it going live, e.g. 10% of the proposed target. So perhaps we could see how many people engage with the idea prior to posting, and use this to gauge what target we should set.

Some additional thoughts:

1. Should we stipulate that pledgers need to have completed a review in the past, to ensure that we're tapping into the actual reviewer pool? Otherwise our ranks may be filled with people who aren't likely to review in the future.

2. It would be good if there were some way to monitor compliance, so we can learn about success/failure of the campaign. Not sure if this is a good idea, but maybe we could ask pledgers to forward any rejection emails they send, so we can keep a tally?

3. Are you thinking this to be a permanent pledge? Or are people free to return to their previous ways after some time period, e.g. 3 years?

[CooperSmout]

Where to review instead?

PLoS One is another one to add (not-for-profit). Not sure if there's any other journals in the PLoS range that publish psych articles.

[jonmcphetres]

On a related note, I think it would help each campaign to have some number of people committed to signing prior to it going live, e.g. 10% of the proposed target. So perhaps we could see how many people engage with the idea prior to posting, and use this to gauge what target we should set.

Right- good point.

PLoS One is another one to add (not-for-profit). Not sure if there's any other journals in the PLoS range that publish psych articles.

We should definitely add PLOS to the list (I mentioned it on Twitter but forgot about it here).

Some additional thoughts:

Should we stipulate that pledgers need to have completed a review in the past, to ensure that we're tapping into the actual reviewer pool? Otherwise our ranks may be filled with people who aren't likely to review in the future.

These are good points. Maybe we could have people identify whether they have reviewed in the past somehow? This way people can still pledge even if they haven't reviewed before.

It would be good if there were some way to monitor compliance, so we can learn about success/failure of the campaign. Not sure if this is a good idea, but maybe we could ask pledgers to forward any rejection emails they send, so we can keep a tally?

I like the idea of forwarding emails, but a lot of times they are "confidential". Perhaps we can ask people to send something saying they got an invite, but honestly, I wouldn't have high hopes for a lot of emails though. It's an extra step thats easy to skip.

Are you thinking this to be a permanent pledge? Or are people free to return to their previous ways after some time period, e.g. 3 years?

I think permanent is the way to go.

[CooperSmout]

Cool, agree with all of that.

Just saw this twitter thread and it led me to this related blog post arguing against paying reviewers. I don't agree with many of the arguments, but posting here because it contains links to various tweets that could be useful for promotion later.

[CooperSmout]

And again, another tweet about charging companies for reviews. James' last post called for comments to indicate interest in receiving the template, and has 192 responses at the time of writing (will update later). This is handy, as we can use it as a rough starting point for how big the threshold should be. Just need to bear in mind that although the OP is from the field of psychology, not all of his followers will be, and also not all of those would be comfortable signing a pledge.

[CooperSmout]

@jonmcphetres I edited your OP to include a couple more journals, as per this tweet.

It was also suggested that we should choose between having a whitelist or a blacklist, and let the other be defined by default, but not try to do both (as your OP currently tries to do). I suspect it will be easier to do a whitelist, given how many for-profit journals there are out there, and I also prefer it to a blacklist because it highlights the 'good' journals rather than the bad ones.

[dylangomes]

Thanks for this pledge, I am pretty excited about it.

It would be good if there were some way to monitor compliance, so we can learn about success/failure of the campaign. Not sure if this is a good idea, but maybe we could ask pledgers to forward any rejection emails they send, so we can keep a tally?

-- I do think that it would be really interesting to try and collect some data on both rejections and acceptance of reviews. I am not sure how much we know about this process in general because it is always "private"

I like the idea of forwarding emails, but a lot of times they are "confidential". Perhaps we can ask people to send something saying they got an invite, but honestly, I wouldn't have high hopes for a lot of emails though. It's an extra step thats easy to skip.

One solution is to remove the abstract and author names in the review request OR (what I like better) is to create an end of the year survey that goes to pledgers. They would then fill in the number of requests (with accepted or rejected) and which journals they were? I would be interested in a poll like this that is sent out generally (beyond pledgers) that could stand as a baseline to assess the impact of the pledge by... Maybe this is more work than it is worth, just an idea.

PLoS One is another one to add (not-for-profit). Not sure if there's any other journals in the PLoS range that publish psych articles.

Also thought I might suggest PeerJ as a journal for the whitelist. I know I have seen some psychological literature there (although perhaps it was in PeerJs pre-print days), although I am not sure what requirements you are going by for this list. PeerJ is OA and usually a bit cheaper to publish in than PLOS ONE, so perhaps worth considering.

[CooperSmout]

PeerJ is a good one! @dylangomes I've invited you to the organisation so you can edit documents freely. If you'd like to add PeerJ to the list please go for it!

Also like your idea for an end-of-year survey. We could couple this with a paper to bring additional visibility to the project, gain collaborators etc.

[dylangomes]

As a note, I have been using the reviewer response letter that @jonmcphetres drafted as a template to respond to selected reviewer requests. If anyone else is also doing this, it might be worthwhile to compile journal responses (or lack thereof) somewhere. Or perhaps not worth the time... Feel free to reach out.

[dylangomes]

If we pick this up again, it may be worth encouraging the peer-review of pre-prints (rather than for-profit journals) via Review Commons or Peer Community In