kubeprod install aks doesn't work with service principal

[acm-073]

kubeprod install aks by default uses Azure credentials stored by az CLI. If az CLI is logged in with a 'normal' user, this works well. However, if az cli is logged in with a service principal (as in az login --service-principal -u http://azure-cli-2016-08-05-14-31-15 -p VerySecret --tenant contoso.onmicrosoft.com), then kubeprod fails complaining that no acceptable token can be found:

% ./kubeprod install aks
INFO  Installing platform aks
ERROR Error: No acceptable token found.  Perhaps you need to run `az login` first?

az account show however shows that a service principal user is logged in.

It would be great if kubeprod install aks would also support service principal users. Our whole infrastructure automation is running with SP users. Just for the installation of BKPR we need to keep a 'normal' non-personal user around, which is at least a non-standard approach (if not bad practice) for Azure infrastructure automation.

Thanks Albrecht

[javsalgar]

Hi,

Checking the kubeprod code, I would like to know the kind of tokens you get in this case and how the kubeprod installer recognizes them. Could you try again with a higher log level so the installer shows more verbosity?

More context here: https://github.com/bitnami/kube-prod-runtime/blob/master/kubeprod/pkg/aks/aks.go#L42

[acm-073]

Hi,

here is the output with more verbose log level:

INFO  Installing platform aks
DEBUG Reading existing cluster settings from "<redacted>/kubeprod-autogen.json"
DEBUG About to create Azure clients
DEBUG Trying to initialise Azure SDK from azure-cli credentials
DEBUG skipping azure-cli token: No refresh token
ERROR Error: No acceptable token found.  Perhaps you need to run `az login` first?

When az cli is logged in with a Servcie Principal user, the ~/.azure/accessTokens.json file looks like this:

[{"servicePrincipalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "servicePrincipalTenant": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "accessToken": "<base64-string>"}]

[javsalgar]

Thanks for the information. Indeed, what this code is doing is create a service principal https://github.com/bitnami/kube-prod-runtime/blob/master/kubeprod/pkg/aks/aks.go#L76

So maybe it would make sense to allow using an already existing service principal instead of creating one. I believe that would cover your use case, am I right?

[acm-073]

Actually, that's not exactly our use case:

Kubeprod should use a low-privilege service principal that has access only to the required Azure resources, like the DNS zone it manages. This service principal's client secret is also stored as secret in the Kubernetes cluster. The service principal that we use for Infrastructure Automation is a high-privileged account has access to the whole Azure subscription to manage all resources. Thus, I would not suggest to use this service principal instead of the one that kubeprod creates. I'd rather create the new service principals as before, just not with a "regular" user but with an existing SP user.

BTW, kubeprod creates 2 service principals: "Kubeprod cluster management" and "Kubeprod External DNS support". The ExtDNS support user has access to the resource group where the corresponding DNS zone is located. However, I could not figure out what the "cluster management" SP is used for. Can you shed some light on this?

[javsalgar]

Hi,

Thanks for the input. I checked the azure library and it seems that there's a function that allows creating a service principal with another service principal token

// NewServicePrincipalTokenWithSecret create a ServicePrincipalToken using the supplied ServicePrincipalSecret implementation.
func NewServicePrincipalTokenWithSecret(oauthConfig OAuthConfig, id string, resource string, secret ServicePrincipalSecret, callbacks ...TokenRefreshCallback) (*ServicePrincipalToken, error) {

https://github.com/bitnami/kube-prod-runtime/blob/26ea6b7a294a3061acdcc144c4a977dd4d000f09/kubeprod/vendor/github.com/Azure/go-autorest/autorest/adal/token.go#L469

I think it could be worth exploring how difficult this would be to implement

[dedri-github]

It turns out that it's possible to authenticate BKPR installation to AKS without using Azure CLI credentials but with an authorization file. Unfortunately this is not documented but supported in the BKPR code.

First, generate an authorization file:

az login --service-principal --username <username> --password <password>
az account show --sdk-auth >/tmp/authorization.json

and provide it with the AZURE_AUTH_LOCATION environment variable to the install command:

AZURE_AUTH_LOCATION="/tmp/authorization.json" kubeprod install aks --email <email> --dns-zone <dns_zone> --dns-resource-group <resource_group>

@javsalgar , is it possible to add it to the AKS install quick start or to kubeprod install aks --help?