Support Vault ssh certificate endpoints

[eholzbach]

Is your feature request related to a problem? Please describe. For example, I have a Vault ssh secret store mounted at ssh/. Users expect to submit their public key to ssh/sign/name and be returned a certificate signed by the certificate authority. The user can authenticate to over ssh using their private key and ca signed ssh certificate. I want to use Boundary as the the authentication mechanism and transit to hosts. We can configure the Vault ssh endpoint as a Boundary Credential Library, but there does not appear to be a way to pass the user's public key through to Boundary to be signed by Vault and returned.

Describe the solution you'd like I'd like a solution were the user can pass a parameter of the public key to have signed, which would be passed through to the Vault ssh mount. For example, a user would run boundary connect ssh -target-name=somehost -public_key=~/.ssh/id_ed25519.pub and be returned a signed certificate.

Describe alternatives you've considered Continue to expose Vault and bastions to end users instead of using Boundary.

[covetocove]

@eholzbach this is an excellent suggestion! We are actively investigating this improvement.

I will leave this request open to gauge interest from the community with their upvotes.

[bourribab]

+1

[lorellalou]

+1

[xingluw]

Hi everyone, I am a product manager on the Boundary team that is working on the SSH Certificate feature and I am looking for some users to provide feedback. If you are interested, please fill out this form: https://forms.gle/YmVZwyTSGX5FqLNK7

[primmus]

+1

[mahirkemal]

+1

[Delta-VII]

+1

[m-tanner-dev0]

@xingluw how's the progress on this feature?

[xingluw]

The feature has been approved for the roadmap and will be coming in a future open source release!

[victorhooi]

I'm definitely interested in this feature as well - will be amazing to see this rolled out =).

Any idea of how far away it is - in terms of either ETA, or how many releases away? (i.e. next release vs say, 2-3 releases down the track?)

[xingluw]

It won't be in the next major release but it will be in one of the upcoming releases

[primmus]

@xingluw how's the progress on this feature?

[xingluw]

Hi @primmus, we had to delay this item on the roadmap for other features but it is still on the way!

[victorhooi]

I checked, and this feature doesn't seem to have made the last three major releases (0.9, 0.10, 0.11) after the last comment.

Do you happen to know if this work is scheduled for the next release?

Or is it still unplanned as of yet?

Thanks!

[xingluw]

Hi @victorhooi, this feature is planned and I will have an update on its status early next year

[xingluw]

I am happy to announce that SSH signed certificates is now available in HCP Boundary 0.12! Learn more here: https://www.hashicorp.com/blog/boundary-0-12-introduces-multi-hop-sessions-and-ssh-certificate-injection

[victorhooi]

Coolies - just checking though, from the announcement, it seems this is a paid-only feature, in the cloud-hosted version of Boundary, right?

As in, it won't be available in the open-source version?

Is that a technical limitation, or more just a pricing/product placement type of thing?

[xingluw]

Today, credential injection is currently only available on HCP Boundary. User research suggested introducing the management of ssh certificates was much more straightforward as an injected workflow.

However, credentials used by Boundary can either be brokered to end-users or injected into sessions on worker nodes through protocol decoding such that the credentials are never exposed to users. Credential brokering is fully open source and can be done securely with dynamic credentials. Even when static credentials are brokered, users can only access credentials if they're authenticated and authorized by Boundary.

For OSS users still desiring Boundary credential management with dynamic creds, users can consider using Boundary with Vault's OTP secrets engine if they desire credential brokering with dynamic ssh secrets. Furthermore, we will still consider introducing a brokered version of SSH certificates in future releases and will continue to monitor this post for interest from the community.

[justenwalker]

@xingluw:

Furthermore, we will still consider introducing a brokered version of SSH certificates in future releases and will continue to monitor this post for interest from the community.

Count me interested. How will you be measuring interest in this from the community? Today there are 41 👍 on the issue presumably under the expectation this was going to be an OSS feature enhancement; it came as a surprise that this was instead implemented as a paid feature. OTP is not as good of a substitute as it requires a vault-ssh-helper on the host with a connection to the Vault server; whereas SSH certificates only require trusting a SSH CA which can be done entirely within the sshd_config instead of involving PAM + a connection from the host to the vault server.

[AdamBouhmad]

In thinking through the brokering of SSH Certificates -- would the preferred workflow here be that Boundary does the keygen, hits the sign endpoint on vault, and brokers the keypair + ssh cert to the user, or that there’s some workflow that allows the user to input their public key for signing when looking to connect to a target, boundary passes that public key onto vault & hits the sign endpoint, and the user is brokered a signed cert back? Is there perhaps another option I’m missing here?

[JannikZed]

We are also already using Vault with signed SSH certs and integrated a first boundary prototype. Being stuck now, as these two things don't work with each other is a big showstopper :/

[AdamBouhmad]

@JannikZed Any thoughts on the workflows outlined here?

https://github.com/hashicorp/boundary/issues/1768#issuecomment-1433903603

In thinking through the brokering of SSH Certificates -- would the preferred workflow here be that Boundary does the keygen, hits the sign endpoint on vault, and brokers the keypair + ssh cert to the user, or that there’s some workflow that allows the user to input their public key for signing when looking to connect to a target, boundary passes that public key onto vault & hits the sign endpoint, and the user is brokered a signed cert back? Is there perhaps another option I’m missing here?

[JannikZed]

@AdamBouhmad the perfect workflow for us would be, that the enduser does not need to handle the certificate at all. So that boundary asks vault so sign the pubkey and injects that into the session. This would make the life of our engineers way more easy, as we don't need no scripts on their machines, but the only thing they need is the boundary CLI. When brokering back the keypair, you need to handle that as client user and the question is - why? Boundary could do all of that for us.

[lbpage]

@xingluw Im not sure how OTP is an equivalent feature for OSS installations. OTP's drawbacks are mentioned in the vault documentation and advise that the OTP implementation could be insecure. How can we implement a completely secure system on prem without a good work around to ssh CA? Our company uses Google IDP for our internal users which works well with OSS Vault and Boundary but not the HCP products which prevents us from migrating. Ideally we would prefer to pay a license fee for the OSS version that included ssh CA feature.

[xingluw]

Hi @lbpage, Google IDP will work the same with Boundary if it is OSS or HCP. Are there other requirements preventing you from adopting HCP Boundary? HCP and OSS Boundary also work with all editions of Vault (OSS/HCP/On-prem).

[victorhooi]

Just to clarify here - as a home user, using the open-source version of Hashicorp Boundary, I wanted to use SSH Certificates for my homelab setup, as it's meant to be far superior from a security point of view to keyfiles (which need to be distributed, rotated, etc.), and this would be a good learning experience for me:

https://goteleport.com/how-it-works/certificate-based-authentication-ssh-kubernetes/

E.g. article on the recent Gihtub SSH key leak, and why they didn't use SSH certificates

https://mjg59.dreamwidth.org/65874.html

However, it seems like support for SSH Certificates is gated behind the paid cloud-hosted version of Boundary currently.

I don't think OTP is quite the same as SSH Certificates.

Is there some way to use the open-source version of Vault, and use SSH Certificates, similar to how say, Teleport do it? Or is this some functionality that Vault will introduce at some point? (I kinda thought that was the whole point of this ticket, but it's very possible I'm missing the technical nuance here, as I'm still learning all the terminology etc.).

[gardar]

While it would be nice to be able to use the credential injection in the OSS version it is possible to achieve this in a single command like so:

boundary connect -exec vault -target-id ttcp_1234567890 -- \
ssh -mount-point=ssh-client-signer -role=administrator-role -mode=ca \
administrator@{{boundary.ip}} -p {{boundary.port}}

Not the prettiest or most user friendly command in the world, but perhaps it's possible to make some nice wrapper for this.

[driskell]

While it would be nice to be able to use the credential injection in the OSS version it is possible to achieve this in a single command like so:

boundary connect -exec vault -target-id ttcp_1234567890 -- \
ssh -mount-point=ssh-client-signer -role=administrator-role -mode=ca \
administrator@{{boundary.ip}} -p {{boundary.port}}

Not the prettiest or most user friendly command in the world, but perhaps it's possible to make some nice wrapper for this.

The only downside is this still exposes the certificate to the client so it could be used/abused/misused with arbitrary SSH connections whereas with the credential injection the certificate would only be available to the Boundary Worker and only usable for that specific connection.

If credential injection isn’t a need then absolutely there are plenty of ways to broker the credentials seamlessly with wrappers

[gardar]

The only downside is this still exposes the certificate to the client so it could be used/abused/misused with arbitrary SSH connections whereas with the credential injection the certificate would only be available to the Boundary Worker and only usable for that specific connection.

If credential injection isn’t a need then absolutely there are plenty of ways to broker the credentials seamlessly with wrappers

Yes absolutely, I just wanted to write down that command in case someone found it useful for their home lab, etc. but hopefully this feature will be brought to the open source edition sooner rather than later. Or more importantly if the support for external plugins is ever implemented then there's potential to create something which does the lookups on the workers and out of user sight.