[Bug]: Report and Appeal Escalation don't seem to be working with reviewer actions - approve, deny, force disable

ioanarusiczki commented 2 months ago

What happened?

STR:

Sent a report for an extension or theme
Resolved the report from Cinder, decision is to take content down
The author is sending an appeal (meanwhile new reports are sent)
The appeal is forwarded to AMO (the appeal and new reports are present under the appeal)
Appeal escalation is resolved with Deny appeal (I also tried to force-enable)

What did you expect to happen?

Flag for human review is still present, Add-on History shows the Appeal as being denied but the extension is still displayed in the manual review queue and the report is still available in the DSA section. At deny appeal , email is not sent to the author. At force-enable, content is being enabled but email is not sent to the author.

Examples: extension with the appeal escalated and multiple new reports sent https://reviewers.addons-dev.allizom.org/en-US/reviewers/review-listed/631773#review-actions theme with the appeal escalated and multiple new reports sent https://reviewers.addons-dev.allizom.org/en-US/reviewers/review/631774 an extension, doesn't have new reports attached https://reviewers.addons-dev.allizom.org/en-US/reviewers/review/631775

Is there an existing issue for this?

[X] I have searched the existing issues

┆Issue is synchronized with this Jira Task

ioanarusiczki commented 2 months ago

I've tried the following too:

a report is escalated to AMO (extension or theme)
reviewer tries to Force-disable

Results: emails not sent to reporter/author and the report is still present in the dsa section. Flag for human review is removed in these cases.

Examples: https://reviewers.addons-dev.allizom.org/en-US/reviewers/review-unlisted/631777#review-actions https://reviewers.addons-dev.allizom.org/en-US/reviewers/review/631779

or Scenario from https://github.com/mozilla/addons/issues/14872, currently I see that the version has been resolved with Ignore/Approve but emails were not sent and multiple escalation continue to be displayed in the DSA section

eviljeff commented 1 month ago

I've just tried to reproduce and was able to. Cinder is raising a 500 error, so the job isn't being resolved with a decision.

eviljeff commented 1 month ago

For reference, AMO sentry traceback, and Cinder GCP log

eviljeff commented 1 month ago

Note: I contacted the Cinder developers and am awaiting a reply on their initial investigations

eviljeff commented 1 month ago

status update here:

I have a fix (mozilla/addons-server#22532) that works around the bug, by making the decision on the add-on, rather than the job. Which is what we were doing previously, so isn't terrible, but isn't ideal as the job is left without the correct decision, and then the actual decision is unconnected to the reports and/or appeal.
The Cinder developers have now confirmed the suspicions of the cause, but a "fix" from their side isn't on the short term horizon, because what we're doing (resolving with one decision, then trying to override it) isn't seen as standard behaviour.
Medium-term we should rework how cinder->AMO escalations work, using a new webhook that wasn't present during original planning/implementation in 2023.

ioanarusiczki commented 1 month ago

I tested again the scenarios above and worked for me.

mozilla / addons