Support multiple storage accounts

[pksorensen]

To support high availability and scalability it would make sense to support volumes across multiple storage accounts.

One option would be to do -o account=accountname -o share=shareName when creating the volume.

As of configuration for all account, secrets - it would be really nice if all secrets was coming from azure keyvault.

Thinking that we could give the volumedriver the clientid and clientsecret of an azure ad application identity and then whenever it needs a key for a accountName for the first time it would get a token from azure ad and then fetch it from https://myvault.vaults.azure.com/secrets/avd-accountname/

Personally I prefer to prefix secrets with the name of there purpose, in this case avd for azurevolumedriver - but it could simply be /accountname/ also.

[pksorensen]

availability and scalability implies that the azure ad team dont bring down the hole login system like the other day :)

[ahmetb]

Thanks for bringing this issue up. We will consider Key Vault integration it for future releases. I should probably get familiar with Key Vault and see how it fits.

[pksorensen]

I have done a lot with keyvault/azure AD, so i might give this a try. I know nothing about GO, so maybe I can put together a proof of concept as a base and we can discuss from there if it brings any value :)

[kwansing]

Is this planned for a future release? I see this feature as being mandatory to support a proper level of data segregation in a shared platform infrastructure.

[ahmetb]

@kwansing We very much like to implement this as well, it's in our radar. The original issue reported states 2 requests:

1. ability to use multiple storage accounts
2. ability to retrieve secrets from Azure Key Vault.

I think we can focus on (1) first as Key Vault does not yet support delivery of plain-text secrets (storage account name/key) in this case.

I think we need to have a design proposal for (1). Key requirements I can think of:

1. Can this be done without breaking existing users? We currently have config files like this and I'm not sure how it can be extended to have multiple (without writing our own config file parser). If it turns out we can't do this without breaking the existing schema, we can release a new major version (which would be a v1.0).

2. How does users specify which account to use? Do we get a new driver option that contains the storage account name, e.g. docker volume create -d azure -o account=ahmetk3j4kl6hj or do we make this somewhat easier, perhaps by giving the credentials some aliases? I see an opportunity for a more elegant design here, which would support change of account names without deleting/recreating the volumes.

[pksorensen]

@ahmetalpbalkan About keyvault, I currently is using it to store storage account connection strings with no problem.

There is a few options to take on defining how the connection info is stored.

For Azure VMs and their use of certificates in keyvault - they have choosen to serialize a json object.

$jsonObject = @”
{
“data”: “$filecontentencoded”,
“dataType” :”pfx”,
“password”: “$certPassword”
}
“@

In the solutions I been doing we choose to store the secret as the secret value and the storage account name as tag "name". We found this to be nice, but ofcause requires abit more from the client as it needs to parse the tags also and not just the value.

I also see that you could include everything in the value as a connection string.

Correct me if I am wrong, but I dont see why connection info cannot be stored in keyvault. Just need to agree on the format the credentials are stored in it.

[ahmetb]

@pksorensen Key Vault is capable of storing text credentials, however you can't specify them on VM model's "secrets" section to have them securely delivered to the instances. I said "delivery" in my statement earlier. I am guessing you call KV API to retrieve the secrets inside the machines. However, for cert type of secrets, you can have them delivered to the machine automatically.

Regardless, this issue contains 2 issues. 1) multiple accounts 2) KV support. We should probably separate them.

[pksorensen]

I actually dont know how certifcates are delivered within the azure infrastructure when using a cert stored as text in keyault. Your right, if it can be done the proper way thats nice.

Seperating is good. I was focused on the user experience of setting it up, where I think a nice experience for this and also delivering the secrets in a secure manner -(but i am not security expert). Let me try to explain what I would have done.

When the volumn driver is being set up, just like now, i would provide it with a principalID and PrincipalKey to an azure AD principal that has read access to a keyvault secrets as well as this keyvault namespace.

Then when running the create volume i would use a secretName property specifying the secret that has storrage credentials. docker volume create -d azure -o secretName=ahmetk3j4kl6hj

This gives given my knowledge a secure and very easy user experience, where the above command would get a access token from AD, ask the keyvault for the secret ahmetk3j4kl6hj and use this information to mount the drive. Then one just have to add the connection string to keyvault secret before creating the volume.

Again, just iterating here to provide feedback. Use it as you see fit :)

[ahmetb]

@pksorensen yeah we have a way of delivering cert secrets to VMs without sending service principal ID/key to the VM. That comes in very handy, however we don't have the same method working for plain-text secrets. The method you proposed to query KeyVault certainly has advantages and disadvantages. Also, we probably don't want to lock users in to KeyVault as for a lot of people, just dropping one (or multiple) storage credentials work just fine today and making KeyVault required will increase the learning curve. But I agree that it is a very nice feature to have.

[pksorensen]

@ahmetalpbalkan I agree with your view points. The reason that I am very supportive of the key vault solution is that when configured adding more storage to a solution is as easy to adding the connection strings to keyvault and no reconfiguration/deployments are needed to the cluster confiuration/machines or where its running simply being able to do create volume. Rolling secrets is also possible without reconfiguration.

But yes keep it simple for learning is a huge win too, so separating into two tasks is good, but in my mind they are equally important and I think the learning curve would be reasonable for keyvault. Happy to help moving it forward.

[gjonespf]

Very keen to see this, even just from the point of view of visibility of data usage. It's currently quite hard to sort out data usage for specific containers using Azure File Storage without splitting container use into multiple storage accounts. Keen to see the Key Vault idea also, but agree it's a separate issue.

[elgertam]

Is this feature still on anyone's radar?

[msftgits]

This driver is no longer supported and will not be maintained moving forward. We recommend users use CloudStor for Docker native solutions.

Thanks for your contribution, but to reduce confusion, we are closing issues, pull requests, and marking the repo as 'Archive' for the time being.