Add data flow architecture diagram

[harrisj]

Is this PDF showing how it works compared to a traditional survey enough?

[afeld]

For ATO purposes, I think the important thing is showing the system architecture, e.g. what are the technical components (databases, external APIs), authentication/authorization layers, etc.) Does that make sense?

[harrisj]

Yes, I am going to do that as well, thank you for the reminder!

[harrisj]

I should note that I'm trying to do a version of this one that was done for Micropurchase https://github.com/18F/micropurchase/issues/93

[DavidEBest]

If you are going to have HTTP Basic Auth in front of the system, that should be represented. Maybe some endpoint-specific info would be helpful (what happens to the data when I submit?, where do I get directed after I submit?)

Also, what is the "admin flow" (how do I create a survey?)

[harrisj]

User Flow

To take a survey, a user must first enter some basic HTTP Authentication credentials (these are shared across all users and would not identify a single user). The credentials are just there to limit access to 18F employees and/or potential hires. The user would also need to know the exact URL of the survey they are invited to take (there is no root-level directory of surveys).

Once the user submits a survey, the complete response is used only to increment counters for various fields and a full record of the survey is not retained. The user is redirected to a screen thanking them for their participation.

Survey Creation

Surveys are implemented as YAML configuration files within the config/surveys directory of the application (here is a sample survey included in the repo). Surveys do not need to be – and probably should not be – checked into the repo.

1. To make a new survey live, the app (with survey file in its config/surveys) must be deployed to production. This limits the ability to create/edit surveys on the system only to the lead developer or anybody else with deploy access to the specific space. If the survey is named SURVEY_NAME.yml, the new survey form is accessible at /surveys/SURVEY_NAME
2. To mark a live survey as inactive – meaning that it no longer accepts responses – the developer has to edit a field in the survey's YAML configuration to be active: false and redeploy the survey.
3. To delete the survey form entirely, the developer can delete the survey's YAML file and redeploy. This will not remove the counts recorded for the survey from the database.

JSON Data

To allow admin users to retrieve the count, the system also supports a JSON endpoint for each survey at /surveys/SURVEY_NAME.json. This is secured with a different HTTP Authentication password than the one used by survey takers that should only be used by personnel and scripts authorized to download the JSON.

While it is the goal to eventually make the JSON for each survey public, this should not be done by copying the JSON to a static file (perhaps on S3) rather than serving it directly from the application

[DavidEBest]

Does each survey create its own set of tables?

[harrisj]

No. There is only one table with four columns:

1. The survey id
2. The question key name
3. A specific response key
4. The count for that triple

So if I had a survey of ice cream flavors it might look like this.

survey	key	value	count
ice-cream	flavor	chocolate	23
ice-cream	flavor	vanilla	8

[harrisj]

Okay, I have updated the system flow architecture diagram to be less confusing about the DB tables

[DavidEBest]

Much clearer!

[harrisj]

This is now part of https://github.com/18F/cg-compliance/pull/33