search

claranet / terraform-azurerm-storage-sftp

Terraform module for Azure Storage SFTP
Apache License 2.0
2 stars 1 forks source link

[FEAT] Store secrets in KeyVault #2

Closed victorZKov closed 10 months ago

victorZKov commented 1 year ago

Community Note

Description

I am running this module from Azure DevOps. To keep the private values secret, I am adding a module to store the generated values in KeyVault, as it's not clear how to download the values in a secure way. My module is like: 

resource "azurerm_key_vault_secret" "sftp_password" {

  for_each = local.sftp_users

  name = "${each.key}-sftp-password"

  value        = local.sftp_users_output[each.key].password
  key_vault_id = data.terraform_remote_state.law.outputs.mgmt_key_vault_app_id
}

resource "azurerm_key_vault_secret" "sftp_private" {

  for_each = local.sftp_users

  name = "${each.key}-sftp-private-key"

  value        = local.sftp_users_output[each.key].auto_generated_private_key
  key_vault_id = data.terraform_remote_state.law.outputs.mgmt_key_vault_app_id
}

resource "azurerm_key_vault_secret" "sftp_public_key" {

  for_each = local.sftp_users

  name = "${each.key}-sftp-public-key"

  value        = local.sftp_users_output[each.key].auto_generated_public_key
  key_vault_id = data.terraform_remote_state.law.outputs.mgmt_key_vault_app_id
}

Of course, it must be my problem, but I can't find a way to make this work. Terraform is not creating a different entry for each secret, when adding a second sftp user to the list, it's replacing the first one, so it's a mess. Perhaps you can add some feature to do this.

Kind regards, Victor

New or Affected Resource(s)/Data Source(s)

azurerm_storage_account

Potential Terraform Configuration

No response

References

No response

Shr3ps commented 1 year ago

Hello @victorZKov , can you add some more code to well understand your issue? How do you populate local.sftp_users? How do you call this module in your code? What about using the module outputs to create your secrets?

victorZKov commented 1 year ago

Hi, thanks for that fast answer!! I have a file called sftp.tfvars with this content: ##############

Containers

##############

containers = [{
  name : "victorz"
  },
]

##############

SFTP Users

##############

sftp_users = [
  {
    name = "victorz"

    permissions_scopes = [
      {
        target_container = "victorz"
        name             = "victorz-folder"
        permissions      = ["All"]
      }
    ]
    ssh_password_enabled = true
    ssh_key_enabled      = true
    ssh_authorized_keys = [{
      key         = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDto9SR/v6l8sCCL35/BsgDSrO90aJ0GmxfPA5mbUyRqXOPHEkw8nK7ok4Q27FQ2EIqhvRFIHXEtFrjWkk5MQ3Zou/D/H5gz93uKFjGqflP6eN8d+BBfwdVY0+SpJFw2yq7mCrqdfuhxjyvNJgjjdWDudQpTwljUhBqucEMoRs5iYDqnruKSOsWxwmPmp+O5PJ6j6noIn4sS/SzMCUFxdj8JZK+4sK0RmArIlNh6rOInFj4EKptY1Xe+ZbbMGKw2qFrbb8o+Ls1XIFBWkVkd3U3ug+1zMT5mfm7u60qGeU0D7yGl11MUTcquQUAJ7QFvG6gXA47hvmDMZiIDma8v3qYbwRtyD5M5iAvXXMQiID7gHbZJQOx8poqtLB4PCIH5f3ujtuZ1VOwPKUTzFGuJQHOVu0XaWuuaR+TfLZTuuyz9dfBenrX3Rm55Toa0ULOGFDc7wZZBSsUvqEi4oUZz5W5Ps3mjaCjtGzYLbyC+QfJPl/pXy+CumMedS5UsO3Y96c= victor_admin@DESKTOP-V1PDQ7E",
      description = "victorz public key"
    }]
  },
]

And I am executing using terraform apply -var-file=sftp.tfvars in a release pipeline in Azure DevOps.

The idea is to have that file available for change for the administrators of the system, that way, they can simply change that file and create or remove users from the sftp system.

The fact that I want to put the generated secrets in KeyVault is to be able to access them anytime after execution. Not sure how to use the module outputs from Azure DevOps.

Shr3ps commented 1 year ago

Hum, sorry but I don't get how you're using our module. Did you check the example: https://github.com/claranet/terraform-azurerm-storage-sftp/tree/master/examples/main ?

Using the module outputs would be something like this (sample code to complete):


module "storage_sftp" {
  source  = "claranet/storage-sftp/azurerm"

  ... # all parameters to set

}

resource "azurerm_key_vault_secret" "sftp_password" {

  for_each = module.storage_sftp.storage_sftp_users

  name = each.value.name
  value  = each.value.password

 key_vault_id = data.terraform_remote_state.law.outputs.mgmt_key_vault_app_id
}
victorZKov commented 1 year ago

The module: `

module "storage_account" {
  source  = "claranet/storage-account/azurerm"
  version = "~> 7.8.0"

  location       = var.location
  location_short = var.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = azurerm_resource_group.sftprg.name

  use_caf_naming = var.use_caf_naming
  name_prefix    = var.name_prefix
  name_suffix    = var.name_suffix

  storage_account_custom_name     = var.custom_storage_account_name
  custom_diagnostic_settings_name = var.custom_diagnostic_settings_name

  access_tier              = var.access_tier
  account_kind             = var.is_premium ? "BlockBlobStorage" : "StorageV2"
  account_tier             = var.is_premium ? "Premium" : "Standard"
  account_replication_type = var.account_replication_type

  https_traffic_only_enabled  = var.https_traffic_only_enabled
  public_nested_items_allowed = var.public_nested_items_allowed
  shared_access_key_enabled   = var.shared_access_key_enabled
  min_tls_version             = var.min_tls_version

  static_website_config = var.static_website_config

  sftp_enabled  = true
  nfsv3_enabled = var.nfsv3_enabled

  containers = var.containers

  storage_blob_data_protection = var.storage_blob_data_protection
  storage_blob_cors_rule       = var.storage_blob_cors_rule

  advanced_threat_protection_enabled = var.advanced_threat_protection_enabled

  network_rules_enabled   = var.network_rules_enabled
  default_firewall_action = var.default_firewall_action
  subnet_ids              = var.subnet_ids
  allowed_cidrs           = var.allowed_cidrs
  network_bypass          = var.network_bypass

  identity_type = var.identity_type
  identity_ids  = var.identity_ids

  logs_destinations_ids   = []
  logs_categories         = var.logs_categories
  logs_metrics_categories = var.logs_metrics_categories

  default_tags_enabled = var.default_tags_enabled

  extra_tags = var.extra_tags
}

resource "azurerm_role_assignment" "assign_admin" {
  scope                = module.storage_account.storage_account_id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = var.principal_id
}

The users: 

resource "tls_private_key" "sftp_users_keys" {
  for_each = local.sftp_users_with_ssh_key_enabled

  algorithm = "RSA"
  rsa_bits  = 4096
}

resource "azurerm_storage_account_local_user" "sftp_users" {

  depends_on = [module.storage_account]

  for_each = local.sftp_users

  name = each.key

  storage_account_id = module.storage_account.storage_account_id

  ssh_key_enabled      = each.value.ssh_key_enabled
  ssh_password_enabled = each.value.ssh_password_enabled

  home_directory = coalesce(each.value.home_directory, each.value.permissions_scopes[0].target_container)

  dynamic "permission_scope" {
    for_each = each.value.permissions_scopes
    content {
      service       = "blob"
      resource_name = permission_scope.value.target_container
      permissions {
        create = contains(permission_scope.value.permissions, "All") || contains(permission_scope.value.permissions, "Create")
        delete = contains(permission_scope.value.permissions, "All") || contains(permission_scope.value.permissions, "Delete")
        list   = contains(permission_scope.value.permissions, "All") || contains(permission_scope.value.permissions, "List")
        read   = contains(permission_scope.value.permissions, "All") || contains(permission_scope.value.permissions, "Read")
        write  = contains(permission_scope.value.permissions, "All") || contains(permission_scope.value.permissions, "Write")
      }
    }
  }

  dynamic "ssh_authorized_key" {
    for_each = each.value.ssh_key_enabled ? ["auto"] : []
    content {
      key         = tls_private_key.sftp_users_keys[each.key].public_key_openssh
      description = "Automatically generated by Terraform"
    }
  }

  dynamic "ssh_authorized_key" {
    for_each = each.value.ssh_key_enabled ? each.value.ssh_authorized_keys : []
    content {
      key         = ssh_authorized_key.value.key
      description = ssh_authorized_key.value.description
    }
  }

  lifecycle {
    precondition {
      condition = alltrue([
        for scope in each.value.permissions_scopes : contains(keys(module.storage_account.storage_blob_containers), scope.target_container)
      ])
      error_message = format("At least one target container does not exist (or is being deleted) for user %s.", each.key)
    }
    precondition {
      condition = alltrue(flatten([
        for scope in each.value.permissions_scopes : [
          for permission in scope.permissions : contains(local.sftp_users_permissions, permission)
        ]
      ]))
      error_message = format("One or more permissions are wrong for user %s. Allowed values in the list are: %s.", each.key, join(", ", [
        for permission in local.sftp_users_permissions : "'${permission}'"
      ]))
    }

postcondition {
      condition     = contains(self.permission_scope[*].resource_name, split("/", self.home_directory)[0])
      error_message = format("The home directory of user %s does not refer to any container in its permissions scopes.", self.name)
    }
  }
}

`

victorZKov commented 1 year ago

And I think I am using the outputs in my secrets.tf

resource "azurerm_key_vault_secret" "sftp_password" {

  for_each = local.sftp_users

  name = each.key

  value        = local.sftp_users_output[each.key].password
  key_vault_id = "/subscriptions/xxxx/resourceGroups/ai-tests/providers/Microsoft.KeyVault/vaults/kov-ai-kv"
}

resource "azurerm_key_vault_secret" "sftp_private" {

  for_each = local.sftp_users

  name = "${each.key}-sftp-private-key"

  value        = local.sftp_users_output[each.key].auto_generated_private_key
  key_vault_id = "/subscriptions/xxxx/resourceGroups/ai-tests/providers/Microsoft.KeyVault/vaults/kov-ai-kv"
}

resource "azurerm_key_vault_secret" "sftp_public_key" {

  for_each = local.sftp_users

  name = "${each.key}-sftp-public-key"

  value        = local.sftp_users_output[each.key].auto_generated_public_key
  key_vault_id = "/subscriptions/xxxx/resourceGroups/ai-tests/providers/Microsoft.KeyVault/vaults/kov-ai-kv"
}

But it's not working as expected, when I add a second user, it's rewriting the resource for the first user in the state.

BzSpi commented 1 year ago

Hi @victorZKov are you using the sftp module from the registry or copying the code ?

victorZKov commented 1 year ago

I have exactly what you can see above

BzSpi commented 1 year ago

So it seems that you have duplicated the module code. That's not how it's supposed to be used.

Please take a look at the example https://github.com/claranet/terraform-azurerm-storage-sftp/blob/master/examples/main/modules.tf

Terraform has also great documentation https://developer.hashicorp.com/terraform/tutorials/modules

The example @Shr3ps gave you should do what you expect or something close https://github.com/claranet/terraform-azurerm-storage-sftp/issues/2#issuecomment-1757564059