pkp / pkp-lib

The library used by PKP's applications OJS, OMP and OPS, open source software for scholarly publishing.
https://pkp.sfu.ca
GNU General Public License v3.0
307 stars 447 forks source link

Author Reviewer Grid: the grid fails to be displayed. #9957

Open defstat opened 6 months ago

defstat commented 6 months ago

Describe the bug In the Author's workflow->review stage of a given submission, the Reviewer Grid fails to be displayed.

To Reproduce Steps to reproduce the behavior:

  1. Go to a submission with an Open Peer Review reviewer assigned to it
  2. Login as the Author of that submission
  3. Go to that submission
  4. See the error.

image

What application are you using? OJS main branch

Additional information The underling error is Fatal error: Uncaught TypeError: count(): Argument #1 ($value) must be of type Countable|array, null given and it is referring to the actions template variable of the AuthorReviewerGridCellProvider.

The problem seems to be related to this. Here, the stable branch defines the query like so

$query = $this->_getSelectQuery() .
            ' WHERE r.review_round_id = ? AND r.review_method = ? AND r.date_confirmed IS NOT NULL AND r.declined <> 1 ORDER BY review_id';

The way that is implemented in the main branch, it seems that it is missing the r.date_confirmed IS NOT NULL AND r.declined <> 1 part of the query.


PRs

@Vitaliy-1 I have crafted this suggested PR for that.

PKP-LIB: #9958

Vitaliy-1 commented 5 months ago

@defstat, while testing this I noticed another problem - unable to see any files under that grid. My steps:

  1. Go to the submission in the review stage
  2. Assign a reviewer with an open review mode
  3. Perform a review and attach a review file during 3rd step. Ensuring that it's safe in the database with a file stage SUBMISSION_FILE_REVIEW_ATTACHMENT
  4. Finish the review
  5. Log in as an author
  6. Seeing that the Reviewer's Attachments grid is empty

I can reproduce the bug described here by assigning a reviewer and declining this request. But I'm confused. I don't remember this part of the code at all. So, under that grid all files at the stage SUBMISSION_FILE_REVIEW_ATTACHMENT should be shown if the review is open, right? But author is able to read the review and see the file through that form, so it's duplication of the functionality?

jardakotesovec commented 2 months ago

@defstat Also run into that - and was not aware of this issue. So I created new one at that time https://github.com/pkp/pkp-lib/issues/10453

I will keep it opened as well as it offers alternative solution.

What I propose as fix that display correctly the review assignment also if its not submitted yet. As result author (for open review) basically sees exactly the same thing as editors, because from the code that seemed to be the original intention.

@Vitaliy-1 @Devika008 @asmecher Any opinion on this?

Context is that when the review is opened one - we provide similar reviewer listing to author as the editor can see. Just with option to Read the review and no other actions. Which means that the author see whole process of reviewers are being requested, and they responded etc.

Question is whether we want to reduce it and display maybe only the reviews that has been submitted (which is closer to @defstat PR) or keep the whole process transparent.

asmecher commented 1 month ago

@jardakotesovec, I'm hesitant to make a major change without some pretty wide user testing. We've run into complaints before from both authors and editors. Authors want more information (they want to know what's happening with their submission), but editors want authors to see less information (they are tired of authors bothering them over any perceived change in the display). So authors and editors want opposite things in this case, and the current balance (minus the bug) is a careful one.

jardakotesovec commented 1 month ago

@asmecher @Devika008 have made bit of research about this, which she will share here and we will align it with that.

My intention here was not to change the behaviour - was really trying to understand the original intention to align fix with it. But having some additional feedback from outside will help us to fix that correctly.

I think we will go likely with conservative approach to make sure that the authors can see the reviews when they are in, but not revealing much of the process itself.

Devika008 commented 3 weeks ago

Hi @jardakotesovec and @asmecher

I've connected with a few folks to understand our current open review practices and explore how we might expand them in the future. A key concern raised is that authors might misinterpret open review as unrestricted access to the review process, which could lead to issues like authors directly reaching out to reviewers. Given these potential challenges, our current approach aligns well with user needs. Here’s a brief summary of the feedback and practices shared:

1. Balance Transparency with Role Sensitivity

Consensus: Many respondents believe that while open peer review should allow authors to access reviewer feedback, certain aspects (e.g., declined review requests, unsubmitted or incomplete reviews) should remain private to protect reviewer privacy and maintain editorial control. Action: Consider limiting visibility for authors to completed reviews only. For example, reviews should only be visible after they are submitted and approved by an editor, maintaining transparency without exposing sensitive process details.

2. Provide Editor-Controlled Review Approval

Consensus: Several responses suggest that editors should have final approval over which reviews are shared with authors, even after submission. This gives editors flexibility to manage any unprofessional or potentially biased reviews before author visibility. Action: Introduce an "editor approval" feature, enabling editors to review and approve a completed review for sharing. Editors can decide if the review is constructive and meets professional standards before allowing authors access.

Currently, presenting the number of reviewers and the count of those who have completed (e.g., 1/3) on the author dashboard might not be the most effective approach. However, it may be best to stick with our current setup in the system, avoiding unnecessary adjustments.

The following next steps are simply a summary of findings from recent discussions rather than concrete next steps. With Erik’s ongoing work shaping our future direction, these perspectives serve as useful context but won’t determine our approach. Instead, they’re intended to provide a snapshot of current user sentiments around open review.

Suggested Next Steps

  1. Prioritize Development of a Configurable Transparency Framework: Start by designing the customizable framework for transparency, with the default option set to "completed reviews only," and provide flexibility for journals to adjust these settings.
  2. Implement Reviewer Consent Options: As part of the reviewer invitation flow, add a consent step that lets reviewers specify if they are comfortable with their identity being disclosed.
  3. Design Editor Approval Workflow: Create an editorial review and approval process for submitted reviews before they are visible to authors, allowing editors to filter out content that may not be appropriate for sharing.
  4. User Education and Support: Include inline support resources explaining the boundaries of open review, addressing both authors and reviewers to minimize misunderstandings and maintain professional standards in the process.