data sources monthly digest email

[josh-chamberlain]

Context

We have mailgun set up for the password reset workflow. We should use that to make a monthly email to anyone who chooses to subscribe. The logic should be to use all data sources from the API that are edited within the last month, but in the future we will have parameters—for example, people can subscribe to updates for a specific search of data sources.

Requirements

• requires https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/419
• [ ] get an API key from PDAP
• [ ] create a script which hits our API and sends a Mailgun email with the below contents
• [ ] test the script with your own Mailgun API key (free)
  • when you submit, include the test results
• [ ] PDAP staff will set up the automation with Jenkins, our pipeline
• [ ] The script should use environment variables: SECRET_KEY for the API key and MAILGUN_KEY for Mailgun.

Email Contents

New data sources: number of rows Updates to existing sources: number of rows Number of agencies affected by new/updated sources: number of agencies Explore the data at https://pdap.io/data

Email Recipients

operations@pdap.io

Email Frequency

Monthly, on the first at 12pm (noon) ET.

[aaryauchamkeri]

Hello Josh, I am new to the project and saw this under good first issue. Can I contribute to this task?

[josh-chamberlain]

Hi there @aaryauchamkeri ! Yes, that would be much appreciated. I tweaked the issue to contain more detail and I'm going to cc @maxachis to answer a couple questions:

• I'm thinking the v2 API is ready for this, because the needs are basic; it might save some effort later. Would you agree?
  • based on the answer to this, could you please create an account so our fine volunteer has an API key? I don't think we have self-serve signup live anywhere yet.
• Can this live in the main app repo? If so, where should it go? I assume since it's so small that's simplest.

[maxachis]

@josh-chamberlain @aaryauchamkeri

Yes, I believe this can get started now! There are a few components to this as I envision it:

• [ ] Tweaking the data-sources endpoint to be able to get the requisite information on desired filters (we decided to accomplish this by paginating the endpoint + allowing users to sort by newest: https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/419)
• [ ] Developing an automated script which calls that endpoint and then performs the requisite mailgun wizardry (this I believe is where @aaryauchamkeri can come in most readily)
• [ ] Putting together the Jenkins components along with a list of subscribers.

@aaryauchamkeri and I can chit-chat in discord about setting them up with an API key (I can be found as PDAP Staff with a fun little goose as my profile picture). I can also happily chat with them about the finer details of the task.

Can this live in the main app repo? If so, where should it go? I assume since it's so small that's simplest.

We can place this within the main app repo now for ease of development, although I think over time we'd want to migrate it to a repo devoted to automated components, just to separate concerns. We can also skip the first part and just go straight to putting it in a separate repo!