Restrict Access to files with sensitive Data

javier-molina commented 6 years ago

Summary

Users of biocache-hub with special CAS_SDS roles can generate additional sensitive data as part of the download process. This additional data will be contained in the zip file.

In order to prevent the leaking of sensitive data the DOI Service will prevent access to the zip file if a user does not have the roles that would allow him to generate the same data himself.

As a data producer with one ore more roles to access sensitive data I want to prevent any other user from accessing the file from the DOI service if they don't have at least the roles I have to access sensitive data.

Details

The behaviour of the access to the zip file from the DOI Service can be explained through the following scenarios that assume the data producer generated a zip file via the download functionality and there is a data consumer willing to access that zip file from the doi Service.

Scenario 1: The data producer does not have any roles to access sensitive data.

The data consumer can access the zip file without restriction. No authentication is even required.

Scenario 2: The data producer and the consumer are one and the same. The data consumer can access the zip file without restriction.

Scenario 3: The data producer has one or more roles to access sensitive data, the consumer has a subset of those roles.

The data consumer won't be able to access the zip file, if he clicks on the zip file link he will be presented with a legend on the DOI page asking him to to go to the ALA page to request permission. The system should indicate the consumer what roles he needs.

Scenario 4: The data producer has one or more roles to access sensitive data, the consumer has the same or a superset of roles that the producer has.

The consumer will have access to the zip file.

Implementation considerations

When Biocache Service produces a zip file that contains sensitive data, the roles that grant that access must be passed to the DOI Service so this in turn can later validate the roles need it for a consumer to access the file.
The DOI Service will treat the existing userid field and the field hosting the roles as hidden data that won't be exposed via GUI or REST API. (Will admins have access to these fields?)
The rules listed in this ticket need to be retrofitted as much as possible to the current biocache zip download functionality as it seems that at the moment there is no authentication for a download link sent via e-mail.

javier-molina commented 6 years ago

@nickdos Hopefully I covered all that we discussed, feel free to edit the ticket if something does not reflect what is needed.

nickdos commented 6 years ago

Thanks @javier-molina, that looks good and I think you covered everything.

Do you think that the downloads-plugin code that exposes the link to the ZIP file needs to check for roles as well? I'm thinking that because that file is hosted on the DOI server, all the logic can reside there.

@M-Nicholls: any additional comments or considerations?

M-Nicholls commented 6 years ago

Thanks @javier-molina and @nickdos That's a really good description of the concerns and the solution.

No other issues I can think of.

javier-molina commented 6 years ago

@nickdos I think the download plugin will have to implement the same logic in the UI as the DOI Service particularly around scenario 3 in order to display error messages in a nice way rather than showing a dry message coming directly from the rest api.

nickdos commented 6 years ago

@M-Nicholls What is the status of the state-based SDS roles? Are we using them now in production or is that change still waiting on code changes in biocache-service?

nickdos commented 6 years ago

@javier-molina - the ZIP is just a link and is served from the DOI server, so wouldn't the DOI server show a HTML page with a useful error message if the user didn't have the required roles? If the user does have the correct roles, then the file would download to the user's machine. Maybe I'm missing something though?

javier-molina commented 6 years ago

@nickdos your approach will work fine, at the moment the download plugin serves the file using the api call directly (https://doi-test.ala.org.au/api//doi/10.5072%2F63%2FTEST_DOI_5a669621cd15a/download) while the doi service itself uses a UI controller (https://doi-test.ala.org.au/doi/db7243c3-b3df-43df-98b9-aa846a49dcd9/download). The download plugin just need to use the UI version of the download action.

nickdos commented 6 years ago

OK, I missed that detail about downloads-plugin calling API and not the file URL. Makes sense then to jack-up API to return useful message/s.

javier-molina commented 6 years ago

@M-Nicholls, @nickdos this is ready in test.

Miles would you mind testing the scenarios and see if they work as expected?

javier-molina commented 6 years ago

After a chat with Nick it was decided that for existing downloads (prior do doi service roll out) the access to them will be removed (Apache/nginx virtual directory) to minimize the risk of exposing any existing zip file with sensitive data.

ansell commented 4 years ago

Downloads that are more than 2 years old have been purged from the biocache-service downloads server.

AtlasOfLivingAustralia / doi-service