[Task]: Reviewer Tools Pastebin Re-implementation

chrstinalin commented 2 months ago

Description

Reviewers need a way to communicate build logs with developers and rely on Pastebin to do so. With its decommission, this task is a part of the re-implementation of pastebin.mozilla.org on the Reviewer Tools side.

Acceptance Criteria

### Milestones/checkpoints
- [x] New text/file input field in the “reviewer reply” form to allow a reviewer to upload or paste build logs.
- [x] Attach the build logs as a new model linking to the activity log though a OneToOneField
- [x] Surface the logs in the reviewer tools for reviewers.

Checks

[X] If the issue is ready to work on, I have removed the "needs:info" label.
[X] If I have identified that the work is specific to a repository, I have removed "repository:addons-server" or "repository:addons-frontend"

┆Issue is synchronized with this Jira Task

diox commented 2 months ago

Note: for storage, please use id_to_path(<id>, breadth=2) in the upload_to parameter of the FileField, and a dedicated folder under shared_storage (your storage parameter should be set to SafeStorage(root_setting='MEDIA_ROOT'))

Then we'll need to communicate that storage path to SRE to adjust the nginx configuration in dev/stage/prod.

ioanarusiczki commented 1 month ago

I don't see this on -dev yet but maybe because of what @diox said above.

diox commented 1 month ago

Work is technically done but won't be enabled yet, it's behind a waffle switch, so moving to the next milestone.

diox commented 1 month ago

@ioanarusiczki : Waffle switch to enable this feature is enable-activity-log-attachments. It should work on dev now that Wei has done https://mozilla-hub.atlassian.net/browse/SVCSE-2218

ioanarusiczki commented 1 month ago

I started looking over, I've seen there is a switch. So far:

all reviewer actions have this attachment possibility including unreject or flag for human review
the download attachment throw a 404 when clicked. I tried with https://reviewers.addons-dev.allizom.org/en-US/reviewers/review-listed/633005.
I could upload small zip files, when I try something a bit larger (actually I used zipped extensions of 2 MB or Ublock origin , which is 3MB+), I get a 413 error

413

the Paste from Clipboard worked from win10, the download throws an error (same as for the zip files attached)
I did not try all the possible reviewer actions to check the emails, but that would be https://github.com/mozilla/addons/issues/15018. So far, I did notice that reviewer's reply email has this but I did not see a reference to it for approve or reject (will re-check anyway).

chrstinalin commented 1 month ago

For sanity, double-checked master branch locally --

Download attachment works as expected on both reviewer tools and DevHub without a 404.
The 413 error can be replicated when client_max_body_size is too low. Possibly relates to (#15029). With the workaround for the latter, files up to 200mb are uploaded as expected.
Only from_reviewer.txt email template was modified. I just checked and it looks like approval/rejections are in a separate email template. Reopened #15018

diox commented 1 month ago

@chrstinalin: The 404 is the addons-frontend one, so it seems we're missing the nginx config to route these kind of URLs to addons-server. Did we file the ticket for SRE to do that ?

chrstinalin commented 1 month ago

@diox https://mozilla-hub.atlassian.net/browse/SVCSE-2218 was the only one I was aware was needed

diox commented 1 month ago

There are 2:

We need the storage path (on the filesystem) for the attachments to be set as internal in nginx, allowing HttpResponseXSendFile (which does a X-Accel-Redirect) to work. That's https://mozilla-hub.atlassian.net/browse/SVCSE-2218
We need the URL for the attachements to be served by addons-server, as by default nginx will route things to addons-frontend. That's missing, since /activity/... is an entirely new kind of URL we've added for this issue. That was discussed at some point but maybe not written so we forgot to file that issue, my bad.

Could you file it? Example from a previous time we needed to do that some time ago: https://mozilla-hub.atlassian.net/browse/SVCSE-1563

ioanarusiczki commented 1 month ago

I'm going to mark this verified, I enabled the switch on stage and for any reviewer action I could:

attach a zip or txt file.
use paste from clipboard worked and it will attach a txt file, Windows has a build-in Clipboard History which I found really nice to use, macOS needs an app to do something similar so I found a solution and I could drag and drop text or emojis in the Attachment box.
could not attach a larger zip or txt on stage either (works with small files under 1 MB) but there is an issue for it https://github.com/mozilla/addons/issues/14999
I added a comment on https://github.com/mozilla/addons/issues/15018 with things noticed while testing with emails today.

mozilla / addons