Add read-only REST API for site/scan data; enable CORS for GET/HEAD

[eloquence]

This uses the Django REST framework, which adds a nice browsable API as well.

API can be found at /api/v1, /api redirects to it.

[eloquence]

That all makes sense, thanks! Will poke.

[eloquence]

Okay, I think I've got the latest_scan bit working (https://github.com/freedomofpress/securethenews/pull/109/commits/08afcd4e8f14f15df7f5411fe62c15082d0a9a29), and have updated the README. Let me know if that implementation looks sane to you. I'll poke at fleshing out the site view -- my hunch is that we should keep the latest_scan field there as well, and have an all_scans field that links to a /scans view as you suggest.

[eloquence]

Per-site scan lists should now work, ordered, filterable & paginated.

All files have also gone through autopep8, and I've moved module comments to the top of each file.

Let me know if that looks good. If so, do you want me to prepare a fresh single-commit PR, or are you OK cherry-picking the relevant commits?

[garrettr]

If so, do you want me to prepare a fresh single-commit PR, or are you OK cherry-picking the relevant commits?

Don't worry about that, I'll handle that. Will probably do a squash merge. Reviewing your latest changes now.

[garrettr]

@thisisparker Since your Twitter bot is the motivating use case for the Secure the News API, care to comment on the API implemented by this pull request? Is there anything that needs to be added or changed for it to be used by the Secure the News Twitter bot?

[eloquence]

@thisisparker : FYI, if the bot currently relies on parsing the JS blob, using the API e.g. /sites/?limit=1000 should give you equivalent info. What's missing is the score-derived grade. On the other hand, it also includes the timestamp of the latest scan, the timestamp the site was added, and a link to all previous scans, which the JS blob doesn't give you. Here's an example site result from the API:

     {
            "name": "Mic",
            "slug": "mic",
            "domain": "mic.com",
            "added": "2017-01-14T01:49:45.228000Z",
            "latest_scan": {
                "timestamp": "2017-01-14T02:07:07.838000Z",
                "live": true,
                "valid_https": true,
                "downgrades_https": null,
                "defaults_to_https": true,
                "hsts": false,
                "hsts_max_age": null,
                "hsts_entire_domain": null,
                "hsts_preload_ready": false,
                "hsts_preloaded": false,
                "score": 70
            },
            "all_scans": "http://localhost:8000/api/v1/sites/mic.com/scans/"
        },

[conorsch]

@eloquence This is great work, thank you for taking the time!

What's missing is the score-derived grade.

Hmm, that sounds pretty critical for @thisisparker's use case, but I'll let him chime in to confirm.

[eloquence]

It can be pretty easily calculated, but let me poke - should be easy to include so people don't have to do that themselves. (edit: done)

[thisisparker]

Sorry for the delay, I was traveling, but this all looks good from my end. Of course I'll have to rewrite a bit to use the API, but it looks like that will be just a few lines different and I'll end up with a much more robust app. Thanks for working on this, @eloquence!

[eloquence]

Thanks for taking a look, Parker, and thanks for running the bot! Happy to help with any issues that may arise in migrating.

[garrettr]

Syncing up on this after a few days off; I think it's ready to merge, but doing a final review right now 😁

[garrettr]

This looks great, @eloquence! Thanks so much for your contribution, and for going back and forth with me on a few rounds of review. I'm going to merge this now, and will comment again on this PR when we've deployed these changes into production.

[garrettr]

@eloquence Changes are live on the production site! https://securethe.news/api/v1/ 🍾

Thanks again for this excellent contribution! Would you mind if we wrote a quick blog post announcing the change, and credited you by your Github handle?

[eloquence]

Sweet, thanks for all the help shepherding this to production. :) +1 to a blog post, will share the news once it's up.

[thisisparker]

Finally pushing an update to the bot to make it use this API. Pretty seamless, good work everybody! :+1: