crypto/tls: add PSK support

[gopherbot]

by tiebingzhang:

RFC 4279 (http://tools.ietf.org/html/rfc4279#page-10) added PSK to TLS.
OpenSSL and GnuTLS already have support for it.

The RFC defines three additional key exchange algorithms:
PSK
RSA-PSK
DHE-PSK

It would be nice to add at least PSK and DHE-PSK to GO's crypto/tls package. The work
seems to be reasonable size.

According to Wikipedia
(http://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Key_Exchange_Algorithms_.28Alternative_key-exchanges.29),
RSA-PSK has not been implemented by any of the listed implementations, so it is maybe
okay to push that one off for later.

[rsc]

Comment 1:

For now at least, very low priority.

Labels changed: added priority-someday, removed priority-triage.

Status changed to Accepted.

[rsc]

Comment 2:

Labels changed: added go1.3maybe.

[rsc]

Comment 3:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 4:

Labels changed: added repo-main.

[tchap]

Might be a good idea to revisit this. I am interested in IoT and some devices simply cannot do regular TLS. Having a solid implementation of TLS-PSK would enable people to use Go to implement a server communicating with a swarm of IoT devices. I am not so much into crypto to be able to implement this myself. Basically just saying that IoT is on hype today and the last comment in this thread is more than 3 years old. For someone knowing crypto and crypto/tls it seems that this would not be too much work...

[tchap]

Sorry, the last change in this thread is 2015, but anyway :-)

[rsc]

/cc @agl for advice

[agl]

My feeling is that this is fairly obscure and that we (by which I mean "I") should focus on TLS 1.3 support in crypto/tls, at least in the 1.9 cycle.

[mordyovits]

@agl @rsc @tchap I have implemented the PSK and DHE_PSK ciphers (server and client). Is there interest in merging code for this? It required removing assumptions about there always being a server cert, so it's not totally trivial.

[bradfitz]

Go 1.9 is frozen, but I'll let @agl decide for Go 1.10.

[mordyovits]

You can find my fork with PSK support here: https://github.com/mordyovits/golang-crypto-tls

[EdSchouten]

TLS-PSK would be quite a nifty tool for securing Paxos/Raft-based systems without the need for complex certificate management.

[tommie]

I have a hopefully mergeable version in https://github.com/tommie/go/tree/tls-psk. It's a bit behind master by now, but seems fine.

I need to go over it one more time to double-check sanity there. There are some assumptions about certificate fields being set in the current code, and my patch needs to wrap those in if-statements, making it a bit scary. I also only cared about relatively modern PSK cihper suites.

If there is interest, I can push forward with that. I just haven't had the time lately. OTOH golang/go allows GitHub pull requests now, so that's nice. :)

[jvmatl]

It would seem we are a lot closer to external PSK support with TLS 1.3, but as far as I can tell, the existing TLS stack only supports PSKs in the context of session resumption. I'm not a TLS expert, but I think the only things missing now would be:

(a) an interface to getKey/getIdentity on client and server side (I like what Raffael Sena did here https://github.com/raff/tls-psk -- this is my current "solution") and (b) support for the (pskModePlain) Key Exchange mode on the client and server.

I know that PSKs are not 'cool' -- but the whole world is not a web browser! There are places where PSKs still make a lot more sense than certificate-handling: @EdSchouten, above has one use case for etcd. - In my case, I have a network of many, many highly constrained IoT devices (constrained in both code space and battery power) that need to talk to a Golang backend; I can't afford the code space or CPU time for asymmetric crypto or elliptic curves, but pre-provisioning AES keys at the factory is easy.

So currently my options are to use a forked crypto/tls (suboptimal, because of the tight coupling between crypto/tls and net/http, and with each version of Go the fork gets more out of date,) or switch to a CGo wrapper around WolfSSL or mbedTLS or {shudder} OpenSSL, with all the fun and performance benefits of CGo...

I had put off making the switch because I had thought if I could hold out until TLS13 in Go the PSK support I needed would be built-in... but alas no.

At this point, there are at least three people (Raff, above, @tommie , and @mordyovits ) who thought having external PSK functionality was useful enough that they went ahead and forked crypto/tls themselves to implement it. And I have to assume the number of people who are qualified to do that are pretty small, so the fact that three did suggests to me that there are many others who would do it if they felt comfortable doing so. Please consider taking the last step and making this use case supported by Go out-of-the-box...

[FiloSottile]

It is indeed pretty easy to add PSK support in TLS 1.3. (Or at least everything but the actually hard part of picking an API for it.) The problem is that with static RSA cipher suites hopefully on their way out, we are close to getting to a sane place where all crypto/tls connections have forward secrecy.

Would ECDH+PSK actually satisfy anyone's use cases? Because pure PSK doesn't have forward secrecy and I am extremely reluctant to add that.

Next it's a matter of how many people need this. Judging from the popularity of the forks, it doesn't look like a very widespread need, and crypto/tls is in a constant fight against the wide scope of TLS to manage complexity.

Finally, now that modules are here it's actually pretty easy to replace crypto/tls by using the vendor folder, without having to fork net/http or replace the whole GOROOT. I should probably write that up.

[jvmatl]

Well, I can't speak for everybody, but in my case, forward secrecy is not important to me because we have a mechanism to rotate the PSKs outside the context of TLS. (Also, because in an IoT sensor network, the ability to go back and decode old sensor data is of limited value.) I can certainly understand why we might not want to enable TLS modes that don't provide forward secrecy by default, but enough people thought there was a valid use case for plain to include it in the TLS1.3 spec despite the lack of forward secrecy, so I can't be the only one who would use it if it were available.

I realize I'm in the minority, but I work with tiny embedded devices for which every kilobyte of code space is precious, and every single byte sent over a network costs me battery life. So, rather than have a 'dummyproof' TLS stack with PFS, I would much rather have one that lets me make the security tradeoffs that make sense for my problem space.

All that said, if this change never makes it into the std library, I would love a writeup explaining how to handle vendoring crypto/tls. And if any kind soul wants to provide some unofficial code with the 'easy' changes to handle plain external PSKs, I wouldn't turn my nose up at that, either. ;)

[tommie]

I agree with John. My specific usecase was using the ESP8266 embedded computing modules, and trying to avoid rolling my own crypto. The benefit of ESP8266 is an extremely cheap WiFi enabled module, but it has limited computational power (and no hardware support for cryptography). It's normally running at 40 MHz, but the vendor's SSL library boosts it to 80 MHz during RSA computations... 1,536 bits is barely doable, and at 2,048 bits the watchdog timer usually resets the device.

And so, I didn't feel the RSA support was worth using. AES is fine, and as @jvmatl mentioned, pre-provisioning per-device symmetric keys is often fine for embedded devices.

I think it's important to have something that allows even the smallest of devices to run encryption, and ECDH/RSA/EC is too slow to be really practical. I completely agree it should be harder to enable PSK than forward-secrecy mechanisms, there is certainly a risk that making it difficult to use TLS at all for some devices leads us to a "roll-your-own" situation that benefits no-one in the end.

Finally, I'd like to mention that Mosh is another great example of a situation where PSK works. They have a pre-authenticated channel over SSH, but need to switch to UDP. So they simply send a symmetric key. (They're of course not using TLS, due to UDP.)

[rmspeers]

I had been watching this thread for a while but noticed today there was some discussion happening on it. I work consulting on cryptography (and other items) with a number of companies in the embedded IoT space, and it is very common that we have to figure out a transport encryption solution that works with their embedded constraints. It looks like @jvmatl above has mentioned some of those cases -- limited code size and battery life. In some cases, our code size is more constrained than the chip as for security purposes (among others) we want to be able to bank memory to be able to safely update in the field -- meaning our code size is cut in half. Regardless of how it happens -- code size and battery usage are real-world constraints and it would be great to use TLS rather than roll-their-own. The movement to use a supported TLS implementation in my opinion would add security benefit to people implementing transport encryption -- and allow them to go directly to their Go backends (if they use Go, which we have seen movement on a few of our customers to want to use) rather than having a proxy to decrypt between the IP connection and the backend.

[tommie]

@rmspeers For battery-constrained devices where you wouldn't want to wake up just to send a TCP keep-alive (or similar), wouldn't a more datagram-friendly layer like DTLS be preferred?

My ESP8266 use-case was just a hobby project, so I didn't have any hard constraints to work with. I wanted PSK in TLS because the ESP8266 standard library implements that, so I could have faster turn-around. Does this apply to other devices, or would it make more sense to add PSK in e.g. crypto/dtls specifically for these?

[jvmatl]

@FiloSottile - you said

It is indeed pretty easy to add PSK support in TLS 1.3. (Or at least everything but the actually hard part of picking an API for it.)

-- What do you think about what Raff did as a starting point for discussion? (https://github.com/raff/tls-psk/blob/tls13/psk.go#L44)

If we take that, and add a boolean to explicitly allow pskModePlain, wouldn't that address everybody's concerns? The server could (and should!) still prefer to use pskModeDHE if offered by the client, and so nobody would ever accidentally get a session without forward secrecy;

The only time a TLS1.3 session with a Go server would lack forward secrecy would be if:

• The client and server have a shared PSK (can't happen by accident)
• The client only offers only pskModePlain in the clientHello.pskModes
• The server enables AllowPskModePlain in the tls config

package tls
type Config struct {
    ...
    // Configure the use of Pre-Shared external keys for use between
    // endpoints. This is typically only used in closed environments
    // where client and server can agree on keys out-of-band from TLS
    // and use of public/private keys is impractical. 
    PreSharedKeys PSKConfig
    ...
}

type PSKConfig struct {
    // Controls support for a PSK key exchange mode that is easier to
    // compute on very low-powered devices, but which does NOT offer
    // "Forward Secrecy". Enable only if you understand the security tradeoff.
    // (See `psk_ke` in RFC 8446, Section 4.2.9.)
    AllowPskModePlain bool

    // client-only - returns the client identity
    GetIdentity func() string

    // for server - returns the key associated to a client identity
    // for client - returns the key for this client
    GetKey func(identity string) ([]byte, error)
}

If the change to add external, plain PSK to TLS1.3 really is an easy one, and it's impossible for users to accidentally shoot themselves in the foot, and there are real-world use-cases for the feature where potentially millions of IoT devices get better transport security, isn't this a no-brainer?

[jvmatl]

@tommie - in some contexts, yeah, DTLS with PSK would be preferred, but DTLS support in applications/frameworks is not nearly as widely available as TLS.

For example, the Eclipse Paho project (arguably the go-to project for people looking for open-source mqtt clients) has clients for MQTT-SN (which is explicitly designed for low-power sensor networks!) and they don't even support DTLS yet, let alone DTLS with PSK. (See https://github.com/eclipse/paho.mqtt-sn.embedded-c/issues/90)

But if Go's official crypto were to add baked-in support for external PSKs in TLS1.3 support, you can bet that every single Golang-based MQTT, CoAP, and LWM2M project would find they had a bunch of users wanting to make use of that new capability.

[raff]

I have just updated my "tls-ext" package to latest Go crypto/tls (from Go 1.13.4). See https://github.com/raff/tls-ext/tree/tls13

This is the minimum required for my PSK implementation (i.e. I have only exported methods/values needed to build my tls-psk package) so it's easy to compare with the original code.

Note that this version doesn't actually support TLS 1.3 for PSK (the TLS 1.3 path is still trying to pick a certificate, that doesn't exist). I wasn't sure of what cipher-suites to add and I don't have any tests or implementation to compare to. So for now it's really just an "update" to my original implementation.

[robmccoll]

With these suggestions, adding PSK seems like a good choice.

The argument that it doesn't achieve PFS seems like perfect is the enemy of good. Encouraging forking the standard implementation and rolling your own PSK extension sounds more likely to lead to vulnerable code in the wild than having a PSK implementation in the standard library. Having it in the standard library leads to better testing, compatibility, and visibility over an external implementation. It would have to be explicitly enabled, so the security of default implementations isn't diminished.

[jvmatl]

Finally, now that modules are here it's actually pretty easy to replace crypto/tls by using the vendor folder, without having to fork net/http or replace the whole GOROOT. I should probably write that up

@FiloSottile -- can you elaborate on this? It seems like discussion on this topic halted over the holidays...

[jvmatl]

Now that I have gone further down my current path, the real cost of not having this in the standard library is growing geometrically; the downstream cost is much higher than just having to use an alternate TLS package in my own homegrown code. So just in case it's not clear, here is where I am at.

• I cannot change the embedded platforms: there are literally millions of IoT endpoints already manufactured over the course of years, and even if some of them are upgradeable over-the-air (and many are not) -- they don't have the code space or computational power to use asymmetric crypto effectively.
• If I want to develop a service that these devcies can connect to securely, I can (and am) using @raff's fork of the Go 1.13 crypto/tls. This is pretty straightforward
• I wanted to run a third-party MQTT broker and have it listen for TLS-PSK connections. In order to do this, I am forced to fork the MQTT broker package to make the modifications required to let it use the new tls package, rather than simply modifying some entries in the broker config file's tls.Config. Once I have done that, I have added the indefinite maintenance burden of keeping my changes in sync with upstream (so I can continue to receive new features/bugfixes)
• Next, I wanted to be able to run load-testing and integration tests with simulated clients written in Go. But in order to do that, I had to fork the Eclipse Paho mqtt client package o use the new tls. Again, I now have the maintenance burden of keeping that fork in sync with upstream, all because I couldn't add psk-tls functionality in a config file.

From where I'm sitting, this ever-growing spiral of forked code is unmanageable - so at some point I will probably bite the bullet and fork the builtin library itself, because at least that maintenance cost is a constant, but given the complexity of TLS implementation, I am less confident that I will always get this right.

None of this would be necessary if the TLS1.3 stack were extended to include support for PSK connections - something we have already established is technically easy to do, at least according to @FiloSottile.

() before somebody points out that getting tls-psk in TLS1.3 does not help me with my embedded clients, that's sort of true, but since I can write homegrown psk listeners relatively easily, I can deploy proxies that listen with the forked tls-psk and forward connections to my internal servers -- and over time, as the old devices retire, I can phase this monstrosity out of service. But if tls-psk never* makes it into the standard library, the treadmill never ends...

[gdamore]

Just for the record, I came here looking for something like this. PSK's might make my life easier. I have an already established secure channel (out of band) where I can exchange the PSKs securely, and discard them after I've used them. I don't need PFS in this context because I already have it by virtue of the fact I'm' using a secure channel (which does have PFS properties) to exchange these PSKs (which for my use would make them more like single use session keys.)

Notably earlier today I also came looking for DTLS support (for another but related reason).

It almost seems like the maintainers of this package kind of don't care about having it fully featured.

The failure to also have a FIPS 140-2 cert is also limiting.

I'm feeling like there is a business case in here somewhere.... probably an opportunity for someone to charge real $$ to provide a solution to these gaps.

[FiloSottile]

Since in TLS 1.3 PSKs are merged with resumption, what would help here is an actual proposal of how an API that allows applications to inject PSKs through the same mechanism used to store session tickets/IDs would look like.

A good API would probably solve in one shot this, #25351, #25228 and #19199. (And #46718 and #57753.)

It almost seems like the maintainers of this package kind of don't care about having it fully featured.

For the record, the maintainers of this package care about not having it fully featured. crypto/tls implements a useful subset of the TLS protocol, it's how it stayed secure and maintainable for 10 years.

[FiloSottile]

Looks like https://tools.ietf.org/html/draft-ietf-tls-external-psk-importer-05 is going to get published, so we might want to just jump ahead to its Importer API, and provide a way to import an external PSK into the session store through that mechanism.

Note that we are not going to support ECDH-less PSK modes, as they don't provide per-connection PFS.

[Sean-Der]

@gdamore Go does have pion/dtls and it supports Certifitates and Pre-shared Keys. If it is missing any features that stop you from using it would love to hear!

Sorry to de-rail this ticket. I was looking around crypto/tls for an unrelated thing and saw this.

Also if anyone is interested I would totally be up for giving up ownership to the Go team! https://github.com/golang/go/issues/13525#issuecomment-510230541 the code base is drastically different though. I wrote it to be more verbose/readable, things like each handshake message being their own struct

[gdamore]

Thanks @Sean-Der -- I will look later.. I don't have an immediate need for DTLS, but its probably something I need to think about soon.

[gopherbot]

Change https://go.dev/cl/415034 mentions this issue: crypto/tls: add cipher suites TLS_ECDHE_PSK

[salrashid123]

Anyone cite a plain PSK client-server sample?

i'm using the rc candidate version

$ go version
go version go1.21rc3 linux/amd64

and trying to create a simple TLS client server with just PSK alone and i'm unsure how to construct the tls.Config

the best i could get is

    sstate := &tls.SessionState{
        Extra: [][]byte{[]byte("client1")},
        //secret: []byte(combinedKey),
    }

    tlsConfig := &tls.Config{
        GetConfigForClient: func(ci *tls.ClientHelloInfo) (*tls.Config, error) {
            return &tls.Config{
                MinVersion:   tls.VersionTLS13,
                WrapSession: func(cs tls.ConnectionState, ss *tls.SessionState) ([]byte, error) {
                    // todo encrypt
                    return sstate.Bytes()
                },
                UnwrapSession: func(identity []byte, cs tls.ConnectionState) (*tls.SessionState, error) {
                    b, err := sstate.Bytes()
                    // todo decrypt
                    if err != nil {
                        return nil, err
                    }
                    return tls.ParseSessionState(b)
                },
            }, nil
        },
    }

but don't know how to specify the 'secret' (i think you're supposed to encode the PSK secret into the sessionstate which gets wrapped unwrapped)

for ref, with nodejs and PSK, you can "just set" the secret via tls pskCallback

[raff]

If you look at the code there is a mention of an "extra master secret" (it ends up being a 0 for non-secret and 1 for secret). I didn't really follow the latest implementation but you can try to set the secret at Extra[1] (i.e. Extra[0] would be your "client1" thing, and Extra[1] would be the secret). When you unwrap you can retrieve from there.

Again, I didn't try, just a wild guess :)

[salrashid123]

@raff unfotunately, that didn't work.

@FiloSottile is it possible to directly inject a PSK or is the implementaion for go1.21 using PSK internally for other usecases and not surfaced to users? (i suspect thats the case)

[let4be]

Hey guys, any update on this? Would be interested to see an example on how to use PSK with crypto/tls(even if it's forked), stuff I found is a bit outdated and I'm not sure what state is it in...

[salrashid123]

i looked at this a bit more and i don't think this feature allows for user-supplied PSK like stunnel, openssl (SSL_set_psk_server_callback), node or python.

I think its used for Session resumption with a pre-shared key where tls with certificates was done initially.

(i also noticed this bit seems to show all tls in go needs certs here which isn't needed for user-specified psk

@FiloSottile if the above is confirmed, that'll clarify what we can use this capability in go for.

thanks

[hujun-open]

another use case for PSK TLS support is that it is quantum safe, compares to normal TLS using diffi-hellman and PKI; I know NIST/IETF are working on standardizing new PQC alg. for key exchange and pki, but it will take while, even longer for widely adoption, so it would be nice to have PSK TLS support before that

[gdamore]

Over 10 years later... and still no action. There are numerous use cases where an out of band key synchronization is far easier. For example, imagine a remote device setup (like a drone) where you can pair using USB keys or even using a charging cable. This is simpler, easier to get right in implementation, and can be done on much lower end hardware than full up key exchange would allow. It's also as pointed out, Quantum Safe.

Please also recognize while the device might have a minimal C version (again because embedded), the peer might have a lot more resources and be written in Go. But the constraint here is not the Go side, it's the tiny embedded side.

I completely understand the desire for an easily maintained, minimalist, library especially where security is concerned. But the failure of the maintainers of this package to consider PSK use cases represents an extraordinarily myopic view -- TLS is for more than just web browsers connecting to servers!

Numerous parties have offered to contribute solutions, and have in fact provided code to do this. I could do it as well. But without someone from the maintainership taking it on, it's dead in the water. What will it take to convince the maintainers that this is a critical capability gap?

[gdamore]

I've been asked to solve this problem for Mangos (my NNG/nanomsg compatible API.) I would really like to not have to abandon the standard library. Again, real use cases. In the meantime I will tell my users to use the C version of NNG and avoid Go, and point them at this issue.

[fabiang]

Zabbix Agent 2 for the famous Zabbix Monitoring Server is using CGO and some Code from OpenSSL to allow using TLS-PSK: https://github.com/zabbix/zabbix/blob/39c05453954c1182076677439392f27b8ae0b055/src/go/pkg/tls/tls.go

Maybe not their wisest decision to use Go for their new Agent. But I guess we've jet another use case.