Recommend HTTPS for APIs

[konklone]

Hi! I'm from down south, at 18F. We recently published our own API standards, which we also forked from the White House.

I'm really impressed with the rigor and energy of your work. As you finalize your standards, please consider using this opportunity to include HTTPS support among them.

We require HTTPS in ours, for several reasons:

• Security. The contents of the request are encrypted across the Internet.
• Authenticity. A stronger guarantee that a client is communicating with the real API.
• Privacy. Enhanced privacy for apps and users using the API. HTTP headers and query string parameters (among other things) will be encrypted.
• Compatibility. Broader client-side compatibility. For CORS requests to the API to work on HTTPS websites -- to not be blocked as mixed content -- those requests must be over HTTPS.

When I was at Sunlight, I added HTTPS support for one of their APIs for similar reasons, though that was an especially acute case because the API accepted user lat/longs.

As the web moves towards encryption, and TLS becomes more performant, it's a curve worth getting ahead of.

[LaurentGoderre]

@konklone just to clarify, do you mean to also support HTTPS or ONLY support HTTPS?

[konklone]

To only support HTTPS. It's just easier, both architecturally and procedurally. People tend to just use the simplest thing available -- when I added HTTPS support to an API I used to manage, few people switched. I had to track protocol usage in the API analytics and then write an email to people using the insecure endpoint to bend the curve on that.

[LaurentGoderre]

Wouldn't that cause some issue if you were using the API with AJAX from JavaScript on a page that is not using HTTPS? It would give you a lot of unsecure content warning

[konklone]

Nope, just the opposite -- you can make AJAX (CORS) requests to an HTTPS API from an HTTP website. You will get an insecure content warning if you make a request to an HTTP API from an HTTPS website -- which is a great reason to use HTTPS with your API.

[ydagenais]

I think my issue with requiring HTTPS is that it would be adding a barrier to entry. It's already challenging to get public APIs created in a government environment. Requiring read-only APIs to be encrypted adds another level of friction in deploying the API.

GC.ca's current draft doesn't yet address authentication; that would be optional of course but would definitely require SSL. However, in gc.ca's case I would say the vast majority of public APIs are probably going to be read-only.

@konklone I agree that eventually, it will all be encrypted and that the draft probably needs to include something on the topic of encryption.

[konklone]

However, in gc.ca's case I would say the vast majority of public APIs are probably going to be read-only.

@konklone I agree that eventually, it will all be encrypted and that the draft probably needs to include something on the topic of encryption.

Yeah, even just recommending it as desirable, even for read only APIs, would be a step forward. I'm happy to be helpful if I can be.

One thing to bear in mind when writing that section is that even with read-only APIs, people pass information to them. SSL encrypts the query string and hides those parameters. Sometimes those parameters are very personal, like lat/long, but many parameters can be potential tracking signals, like what keywords a user is using to conduct research.

APIs have a special responsibility to protect user privacy, because they often function behind-the-scenes. Users filling out petitions online that use a third-party API may have no idea there's a third party involved at all, or what their expectations of privacy are with them. SSL is an important way of addressing that responsibility.

[chrismajewski]

I'm very late on this but what are we talking about, some default states, some conditions when you should and shouldn't either? How does that look, has anyone already written or considered this?

I've scheduled time for Oct 15th to put a section in, I'll do some reading to draft something we can hammer into a reasonable shape.

As it stands now I'd consider recommending making both available to developers allowing them to design to their preference/needs. From there you can offer a matrix or checklist to make informed choices, likely in one direction. I don't see a condition where https should not be.

EG: If your API outputs or accepts any of the following the API must be HTTPS only:

• Contact information ( name, email, phone number )
• Private information ( age, sex, weight, name )
• Financial information ( tax credit, credit card, balance )

These conditions may already be available in a standard or some established security policy.

[konklone]

I can understand the desire to prioritize. However, if your team is in the position of being able to deploy HTTPS for some of your APIs, I think you should just recommend it for all of your APIs. When it's operationalized -- the same way that managing DNS and domain name renewal is -- it's not so bad.

I say this in part because it's very difficult to predict how seemingly insensitive data can be used and correlated. I was making this argument to the rbenv project, which distributes public Ruby builds, and verifies them off-band through a checksum packaged with the downloader. Even when verification is in place, and the contents are not individually sensitive, taken together just about any data can draw a picture on the network that wasn't intended by any one user or provider to be drawn.

I'm starting to collect some of 18F's operational practices and technical standards at the 18f/ssl-standards repo -- it's just beginning, but I hope to develop simple, reproducible organizational processes that make SSL less intimidating and more straightforward for our team.

[chrismajewski]

There is no guarantee of a binding standard, if there were a reasonable technical case works.

Convincing will be required. I can make the case for sensitive data but a blanket case would require something more approachable. Lower the barrier to entry, risk analysis, future proofing, etc...

I'll be eager to see what you come up with. I see where you are, not sure how we get people there yet.

[LaurentGoderre]

It may be a good idea to push this for all new API. It would be much better than letting new APPI proliferate and then ask everyone to change to HTTPS when a problem arises

[samperd]

+1 for https

[konklone]

Bumping to mention a couple things happening down here in the US.

First, the US government has proposed an HTTPS-only standard for all publicly accessible federal websites and web services, including APIs. We've begun a page on migrating APIs, since they need special attention.

As far as arguments for blanket encryption go, I like the text we ended up on in the proposal:

The majority of Federal websites use HTTP as the primary protocol to communicate over the public internet. Unencrypted HTTP connections create a privacy vulnerability and expose potentially sensitive information about users of unencrypted Federal websites and services. Data sent over HTTP is susceptible to interception, manipulation, and impersonation. This data can include browser identity, website content, search terms, and other user-submitted information.

All browsing activity should be considered private and sensitive.

An HTTPS-Only standard will eliminate inconsistent, subjective decision-making regarding which content or browsing activity is sensitive in nature, and create a stronger privacy standard government-wide.

And we have a "Why Everything?" page that goes into more detail.

And second, api.data.gov, an API "umbrella" and proxy service that many other federal APIs use, has officially begun its HTTPS transition. It's doing it in stages -- first by ensuring all new APIs only use HTTPS, and then creating a "transition mode" for existing APIs that ensures all new keys are only authorized to use HTTPS URLs. There's of course parallel work to update documentation, talk with individual agencies using the service, etc.

I don't know if the Canadian government is considering anything similar around pushing for HTTPS in the .gc.ca space. But I hope the above is helpful to anyone here looking to build momentum in their own work.

[ydagenais]

Thanks for the update @konklone. I have to admit that HTTPS-only is growing on me. As I mentioned in my original comment, I'm all about lowering barriers to get more departments to engage. GC.ca still has a mixture of solutions hosting departmental servers/resources. e.g. SSL certificates limits creative solutions to maximize resources such as using name-based virtual hosts (sharing an IP for multiple web sites).

Despite technical/admin challenges, I'm starting to agree that perhaps pushing for HTTPS-only would avoid the headaches of doing what you guys are doing; having to get existing APIs moved to HTTPS and all its users/consumers to also update their side. The umbrella/proxy obviously makes that easier to manage/transition. But you're only as secure as your weakest link. So APIs or connections in the back-end feeding the proxy service should also be using SSL. And that brings us back to just pushing for all to deploy APIs using HTTPS from the start... :wink:

Again, thanks for sharing!

[konklone]

e.g. SSL certificates limits creative solutions to maximize resources such as using name-based virtual hosts (sharing an IP for multiple web sites).

I know exactly what you mean. Lack of support for Server Name Indication in legacy clients will be the death of me. It won't be a barrier forever -- and fortunately, Python 2 finally added SNI support in 2.7.9 -- but yes, more to go.

[chrismajewski]

It's getting harder to not accept this outright.

For me it's the nature of a standards, describing use of what is right over what's easy.

What's right, as the authoritative source of so much data, is protecting that delivery as best we can. This should be HTTPS with few and prescribed exemptions as required.

[konklone]

As a quick update, the US shipped its policy last week:

https://www.whitehouse.gov/blog/2015/06/08/https-everywhere-government https://18f.gsa.gov/2015/06/08/the-us-government-is-moving-to-https-everywhere/

And shortly before that, my team at 18F worked with the team at the .gov registry to produce a lightweight public dashboard to measure progress (with SSL Labs grade data and everything):

https://pulse.cio.gov/https/domains/

The policy ended up specifying no exemptions for publicly accessible federal domains/sites/APIs (intranets were encouraged, but not required). It's undoubtedly a big lift, but I'm heartened by the momentum in the US and around the world on getting it done.