Create infrastructure for requesting data

NSAPH-Software / NSAPHutils

R package providing tools for working with big data

GNU General Public License v3.0

2 stars 2 forks source link

Create infrastructure for requesting data #2

Open mbsabath opened 5 years ago

mbsabath commented 5 years ago

Necessary Functions:

Take user input of required information for data request
Log the request in a central location
Notify Data Science team of the submitted request

More advanced functions:

Automatically Check if the user requesting data has permissions to access the data
Provide the data to the user upon receipt of an approved request

mbsabath commented 5 years ago

A potential way to provide the user input would be to create a YAML file template with the fields of the data request form. A function would then read in the file and add the information along with the requester's username in to a csv file stored in a central location. The notification could occur either via an automated email or an automated creation of a github issue.

Challenges with this include:

Checking the request file input
permissions around the central file (i.e. allowing individuals to write lines to it, while ensuring it doesn't get moved)
Notification infrastructure, may require setting up an email server if that is the method of notification chosen

mbsabath commented 5 years ago

@izahn thoughts?

mbsabath commented 5 years ago

Looks like the Github API supports automated issue creation. https://developer.github.com/v3/issues/#create-an-issue

izahn commented 5 years ago

The more I think about it the less convinced I am that this is a good idea. Probably something simpler, like a Qualtrics form will be better for now.

What I would ultimately like is to have a catalog of all the available variables, broken out by level (e.g., all the zipcode/year variables, all the person level variables, all the person/year variables. The list_available_data function will query this catalog and build_data_set or similar will merge the specified variables from their various sources and return the merged data. The immediate problem with this vision is that access to health data is restricted, so there needs to be a mechanism for requesting it, but I'm not sure we're ready to build that into NSAPHutils. Thoughts @mbsabath ?