search

scidsg / hushline

Hush Line connects whistleblowers with organizations and people who can help.
https://hushline.app
GNU Affero General Public License v3.0
77 stars 21 forks source link

File Upload Support #474

Open glenn-sorrentino opened 3 months ago

glenn-sorrentino commented 3 months ago

A top request is to allow file uploads. To enable this feature while protecting ourselves legally and from incurring increased costs due to cloud storage, this should only be enabled for the Personal Server.

Acceptance Criteria

  1. Users may access files settings through a dedicated tab enabled only for Personal Server.
  2. The feature is disabled if the user has not uploaded a PGP key.
  3. Toggling on the feature reveals more settings, including setting size limits and uploaded a file-specific PGP key.
  4. Size limits may be set using a numerical value and unit type chosen from a <select> menu. "100" and "GB" for example.
  5. A user must toggle on the file-specific PGP option.

Design

Frame 230

brassy-endomorph commented 1 month ago

Can you make a mockup for what submission of a file looks like? I'm assuming just a file picker with the normal message

brassy-endomorph commented 1 month ago

Also for simplicity can we have file upload sizes restricted to megabytes only for now since kilobytes is useless and gigabytes is more than we want?

glenn-sorrentino commented 1 month ago

Can you make a mockup for what submission of a file looks like? I'm assuming just a file picker with the normal message

Yeah for now just use the native HTML component. We can fine tune it once working.

glenn-sorrentino commented 1 month ago

Also for simplicity can we have file upload sizes restricted to megabytes only for now since kilobytes is useless and gigabytes is more than we want?

I suspect the most likely file sizes will be MB and GB, so let's initially include those. We're not going to be turning this on for our main Hush Line instance. At least not right away.

brassy-endomorph commented 1 month ago

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

glenn-sorrentino commented 1 month ago

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

I think we should add a toggle in Settings for the admin to decide if files should be enabled or not. If all of the env variables are present for their S3 or whatever, the toggle appears.

glenn-sorrentino commented 1 month ago

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

500 MB sounds good.

brassy-endomorph commented 1 month ago

I think we should add a toggle in Settings for the admin to decide if files should be enabled or not. If all of the env variables are present for their S3 or whatever, the toggle appears.

We really will need to do #637 after this because some of these configs are going to be very annoying to deal with otherwise.

Also this leads to another question because right now the options I have working are:

  1. S3 backend for uploading files
  2. Local FS backend for uploading files
  3. Error otherwise

What you're proposing sounds like

  1. S3 backend for uploading files
  2. Optionally no backend at all which disables things in the UI / app itself

If we go with this option, it forces dev to use S3 for dev/testing, otherwise we have to add an option 3 to that (FS backend) which changes your original statement from "Valid S3 config implies admins can enable file uploads" to "Any valid configured backend implies admins can enable file uploads." Which is probably what we want because it makes typical dev/test a little more lightweight.

brassy-endomorph commented 1 month ago

In discussions about this, it wasn't mentioned that we support rPi physical devices that run Hushline, so we 100% have to have a driver for the FS and not just S3.

IMO it should be like this:

glenn-sorrentino commented 1 month ago

Hold on this