[GKO] Raw values for encrypting properties from GKO

ivank commented 5 months ago

🌈 Feature

-feature

rainbow-feature

As a Software developer deploying and maintaining a Gravitee gateway

I want to Be able specify encrypted passwords through GKO

So that I can define them with Infra as Code tools (terraform)

🌄 Additional information

-additional-information

sunrise_over_mountains-additional-information

Currently you can specify encrypted properties, but this feature seems to have been designed for the export / import functionality, and not GKO. It demands the values be base64 encoded encrypted binary blobs that were produced by Gravitee Gateway itself when using the console.

All of our secrets are stored in GCP secret manager via terraform as such, our IaC (terraform) already knows the raw secrets, but there doesn't seem to be a documented way to directly put them as "encrypted" variables.

We don't really want the secrets visible in the Console if we can avoid it, and the encrypted properties seem to be exactly the feature built for this, but to use it we need to follow those steps, which kinda defeats the purpose of having infra as code.

Create another API by hand
Set the property there manually through the console
Export the API
Read the generated file and put that value in our terraform env variables.

✅ Acceptance criteria

-acceptance-criteria

white_check_mark-acceptance-criteria

[ ] Given ApiDefinition defined via GKO When I set env variables as encrypted , with raw values Then I want to set properly.

Alternatively:

[ ] Document the encryption process so that base64encode(encrypt(raw value)) can be performed in terraform as well.

Or even better:

[ ] Allow GKO to load properties as encrypted directly from Kubernetes secrets

I'm aware that GKO can resolve k8s secrets with the [[ secret ... ]] syntax, but that would mean the secret would be on display in the console, which is something we're trying to avoid.

☑️ Developer sub-tasks

️-developer-sub-tasks

ballot_box_with_check-developer-sub-tasks

[ ] Jest test for acceptance criteria

kamiiiel commented 5 months ago

GKO should support the same thing too. The property value can be base64(encrypted_data). It doesn't matter how the value is created, using a script, exported from Console ...

ivank commented 4 months ago

But how do we arrive at the encrypted_data? Which encryption algorithm is used, with which secrets? We need to be able to go from unencrypted data to a working property.

ivank commented 4 months ago

We've been told that instead of setting the properties on the API, we can create shared properties and reference them in the API, that would probably be acceptable as a workaround - is there a documentation somewhere how to do this?

a-cordier commented 4 months ago

Hi @ivank unfortunately at this point there is no way to share encrypted properties between APIs

As a workaround , you can try and use something similar to the following script to encrypt your data in your pipeline. Right now, the default cypher for encrypting the values we use is AES-256-ECB

## RAW_SECRET is the value you got from your secret provider using Teraform
## ENCRYPT_KEY is located in the `api.properties` value of your gravitee.yml config file

# convert encryption key to an hexadecimal format
HEX_KEY=$(echo -n "${ENCRYPT_KEY}" | xxd -p -c0)
# encrypt your secret before setting it in your API definition
echo -n "${RAW_SECRET}" | openssl enc -nosalt -aes-256-ecb -K ${HEX_KEY} | base64

A more long term solution for you might be to issue a feature request for a GCP secret provider plugin (as of now, we only support vault and kubernetes secrets)

Hope this helps

ivank commented 4 months ago

This sounds like a solution - but could utilize kubernetes secrets for something like that?

Right now we can but the values still end up exposed in the configuration for anyone with read access to see. We were hoping there was an EL way to reference kubernetes secrets to be loaded directly.

kamiiiel commented 4 months ago

You already have the syntax to reference a secret inside the API definition and you mentioned it inside your comments as well. [[ secret secrte-name/key ]]

ivank commented 4 months ago

Yes but that results in them being visible in the console as plain text - which is what we're trying to avoid. If there was a way to keep them being shown as coming from k8s secrets, rather than resolving them on import, that would have been awesome!

a-cordier commented 4 months ago

Hello @ivank we have an open discussion in the team regarding your proposal, but right now I think encrypting properties beforehand in your IAC pipeline is the short term solution.

Just to make sure

Am I right to assume that what you want is the location of the secret displayed in the UI ? e.g. sourcing a value baz from a Kubernetes secret foo in the NS bar would result in secrets://kubernetes/foo/bar/baz displayed in the console ?

ivank commented 3 months ago

Am I right to assume that what you want is the location of the secret displayed in the UI ? e.g. sourcing a value baz from a Kubernetes secret foo in the NS bar would result in secrets://kubernetes/foo/bar/baz displayed in the console ?

I mean we don't need to, it could be nice. What we need is to not show the plaintext value in the console for values coming from k8s secrets. e.g. like you do for normal encrypted values

jmcx commented 2 months ago

Hi @ivank any chance we can jump on a call to discuss how this could fit in with our plans around integrations with secret managers? And to try to converge on a short term workaround if needed.

jmcx commented 1 month ago

Closing for now until we can re-connect to discuss our plans to integrate with secret managers.

ivank commented 1 month ago

Just wanted to report that we are using the suggested workaround ant it works for us quite well.

#!/bin/sh

# Encrypt
# https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external#processing-json-in-shell-scripts

set -e

usage() {
  echo "Usage: json | $0"
  echo
  echo "Example:"
  echo "  'echo '{\"encryption_secret\":\"my-secret\", \"value\":\"my-value\"}' | $0"
  exit 1
}

eval "$(jq -r '@sh "ENCRYPTION_SECRET=\(.encryption_secret) VALUE=\(.value)"')"

# Check if ENCRYPTION_SECRET and VALUE are set
if [ -z "$ENCRYPTION_SECRET" ] || [ -z "$VALUE" ]; then
  echo -e "Both 'encryption_secret' and 'value' must be provided."
  usage
fi

# Generate the encrypted password
ENCRYPTED_VALUE=$(echo -n "$VALUE" | openssl enc -nosalt -aes-256-ecb -K $(echo -n "$ENCRYPTION_SECRET" | xxd -p -c80) | base64)

# Output the result as JSON
cat <<JSON
{"encrypted_value":"$ENCRYPTED_VALUE"}
JSON

And then in terraform we can:

data "external" "am_basic_auth" {
  program = ["sh", "encrypt_api_property.sh"]
  query = {
    encryption_secret = var.apim_encryption_secret
    value             = "my secret value"
  }
}

gravitee-io / issues