User interface: Users can see and delete input files, but not download

[MichaelHanksSF]

Layer between Django and UI that: allows users to:

• see the files they have uploaded
• delete files they have uploaded
• download logs we have created

but prevents users from:

• download files they have uploaded

"With Django interposed in this process we need it to act as a proxy for the filesystem rather than providing it directly - instead sending us enough information to display file listings and allow some downloads from within Django (rather than from S3) but shielding the filesystem so links cannot be constructed"

So we need: "1. A new Django app inside the existing deployment to act as this proxy with a suitable API exposed" "2. A new "file listing" TS component which can display some file information and controls but which does not depend on a mounted filesystem being exposed"

[MichaelHanksSF]

• UI engineering - Tambe
• Django engineering - contractor
• How do users view the files in separate workstreams
• in one single list view
• sort files by date of upload/creation
• allow deletion through list view
• click on file to allow download for logs only

Dave to put more detail in task

[MichaelHanksSF]

@dotloadmovie to add more details please!

[dotloadmovie]

My view of this functionality is that it should consist of three API endpoints, similar to the full version elsewhere in the Django layer:

• list-files: provides a flattened list of the tree of files available to the user in their S3 storage. The code backing this will need to descend into the tree and return the results as a flat JSON array of objects. Each object will contain the data for a single file - the normal suspects of id, name, date, filetype, size, but also some properties to enable the front-end to determine whether an attempt to delete or download the file can be made. Depending on the size of the dataset, this may also need to support pagination/sorting/filtering but probably not in the initial instance.

• get-file: takes a file ID and returns the file body as a download. This essentially acts as a proxy for the S3 filesystem, preventing the user from needing to know anything about it. URLs should be formed against the Django URL, not S3. The endpoint should also prevent the access of a file which the user is not permitted to download. This will likely map to a pyfilesystem delete operation internally and be exposed as a REST GET.

• delete-file: take a file ID and performs a deletion. Again a proxy for the S3 filesystem and again should only allow the user to delete files they are permitted to. Will likely map to a pyfilesystem delete operation internally and be exposed as a REST DELETE.

This functionality is essentially middleware, sitting between the REST URL layer and the pyfilesystem mounting. It will need to contain logic to determine what files a user may delete or get (probably cast pretty broadly - ie only log files can be downloaded, all files can be deleted)

The associated front-end can then be pretty dumb. It needs only to issue the commands from the user, list files, and perform some error handling.

[MichaelHanksSF]

Thanks @dotloadmovie !

@cyramic can you please review these notes before COP today and ask Dave anything you are unsure about?

[ChoEjik]

Subtasks:

Backend

• [ ] Create API Endpoints
• [ ] Create Django app within current project
• [ ] Implement robust testing framework