Muckrock Data Portal Scraper

[EvilDrPurple]

Fixes

• Muckrock task of #216

Description

• Adds Muckrock data portal scraper to our repertoire
• Functions for scraping individual files, all files from a foi request, and all files from an agency
• Template scrapers provided for each use case as examples

Docs

• See README.md, function documentation

[EvilDrPurple]

• Can you explain why someone would run muckrock_scraper instead of just using one of the templates with their chosen URL?

  • would recommend "FOIA" instead of "foi", pointing here at first mention

  • I can't think of a use case for scraping an individual file, if the user has a URL for the file's direct download. Maybe cut this one?

  • I think the use cases are good, but I might reformulate the readme so that it is a little more cut and dry. Assume the person running this barely knows how to use the terminal and needs it spelled out for them. Something like this:

I want to scrape all files from a FOIA request

1. Open `template/single_file_template.py` in a text editor.

2. Change the `url` variable to the URL of a FOIA request in MuckRock, and save your changes. Be sure it ends in #files!

3. In your terminal, `cd` into `template/`

4. Run `python3 single_file_template.py`

@josh-chamberlain We discussed this a little in private but I figured I'd lay it out here in case there's any confusion.

• The templates are not meant to be run where they are and as they are, rather copied and modified for use in other scrapers. They are to help people understand how to use the module
• The individual file one I figured could be used if someone has a list of files from various FOIA requests and different agencies, but don't necessarily want to download every file for an agency or FOIA request. They can call the function multiple times with the different files each time. I did consider a similar approach to the opendata_scraper_2, where the user could pass a list of lists that would download every file. I ultimately didn't to make things more straightforward and figured people could make a for loop with lists of their own if they want to call the function multiple times in a row.
• I updated the README to your specifications, though the instructions are a little different. Let me know what you think!

[josh-chamberlain]

@EvilDrPurple fantastic, thanks! passing to @mbodeantor