Conjur supports Dynamic Secrets

[jvanderhoof]

Time estimate: 5 weeks ETA for completion of this epic: TBD Confidence level of ETA accuracy: Low

This feature issue is a work in progress

Overview

Dynamic secrets are credentials created on-demand by Conjur to connect to a third-party service. This is accomplished using pluggable Credential Factories. A Credential Factory is a Conjur resource that is annotated configure:

• A credential factory provider to use:

  credential-factory/provider: aws

  credential-factory/provider: postgresql

• Credentials for the external service to use when creating new dynamic credentials:

  credential-factory/aws/region: us-east-1
  credential-factory/aws/access-key-id: "23478207027842073230762374023"
  # reference to a Conjur variable
  credential-factory/aws/secret-access-key: aws/dev-account/secret-access-key 

  credential-factory/postgresql/user: postgres
  # reference to a Conjur variable
  credential-factory/postgresql/password: psql/dev-account/root-password

• Service-specific parameters to apply to dynamic credentials when created (e.g. IAM policy, TTL):

  # Common parameters
  credential-factory/ttl: PT1H # One hour TTL

  # Provider specific parameters
  credential-factory/aws/iam-policy: |
      {    
        "Version" : "2012-10-17",   
        "Statement" : [
          {
            "Action" : [           
              "sts:GetCallerIdentity"
            ],           
            "Effect" : "Allow",           
            "Resource" : "*"       
          }
        ]

  credential-factory/postgresql/database: postgres
  credential-factory/postgresql/template: |
    CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' 
    VALID UNTIL '{{expiration}}'; 
    GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";

Once created, these dynamic secrets are stored in Conjur, and Conjur manages their lifecycle. The lifecycle for a dynamic secret may include:

• Creation Dynamic secrets are created using the Conjur API

  COMMENT: It's convenient if this can simply use the /secrets endpoint as described in the initial write-up. For managing the lifecycle, we'll need some kind of lifetime identifier. This could be in the form of a conjur variable, or a convention on the factory path (e.g. appending the AWS token id, or PostgreSQL username) to enable renewal or revocation).

• Refresh/Renewal It may be desirable to allow a short-lived credential to be renewed, perhaps to a maximum_ttl.

• Revocation Dynamic secrets may be revoked, either through the Conjur API or through properties on the dynamic secret (e.g. expiration).

  COMMENT: This could probably be handled by rotators as they exist now. There may be a desire to enable "hard" expiration, where the credential is removed, rather than just having its value rotated.

Use cases

AWS Access Token

Description

Dynamic secrets for AWS make it simple to issue short-lived, limited scope credentials on-demand for users or services to access cloud resources.

Prerequisites

A credential factory for AWS would need to be initially created with:

1. Privileged access credentials that can be used to issue new STS tokens
2. IAM policy to attach to the newly created tokens

Example policy for creating an AWS credential factory:

- !webservice aws-credentials
  annotations:
    credential-factory/provider: aws
    credential-factory/aws/region: us-east-1
    credential-factory/aws/access-key-id: "23478207027842073230762374023"
    credential-factory/aws/secret-access-key: aws/dev-account/secret-access-key
    credential-factory/aws/iam-policy: |
        {    
          "Version" : "2012-10-17",   
          "Statement" : [
            {
              "Action" : [           
                "sts:GetCallerIdentity"
              ],           
              "Effect" : "Allow",           
              "Resource" : "*"       
            }
          ]

- !layer

- !permit
  resources: !webservice aws-credentials
  privileges: [ read, execute ]
  role: !layer

Workflow

1. The requester authenticates to Conjur using their user or host credentials

2. The requestor then asks the AWS credential factory for a new AWS access token

   GET https://conjur.myorg.net/secrets/cucumber/webservice/aws

3. The Conjur credential factory authenticates to AWS using the configured credentials and uses the AWS API to create a new access token with the configured policy.

4. AWS returns the token to the credential factory, which will then store it in Conjur

5. The credential factory will return the token to the requester

   {
       "id": "cucumber:variable:cucumber/webservice/aws/ASIAI6...F7KPBKA",
       "token": "{ \"access_key_id\": \"ASIAI6...F7KPBKA\", \"secret_access_key\": \"EhyQYceI...arRtkZfYugB9dDM6\",  \"session_token\": \"FQoDYXdzEH4aD...7bz7Nw+NL0mKNr8ztYF\",  \"expiration\": \"2018-04-15 22:04:26 UTC\",  \"federated_user_id\": \"<acct-id>:myapp\", \"federated_user_arn\": \"arn:aws:sts::<acct-id>:federated-user\/myapp\"}"
   }

   COMMENT: Alternatively, the id could be provided as a response header

6. The requester uses the token to connect to AWS (e.g. to create an EC2 instance)

7. Conjur will revoke the token when either the TTL elapses, or when requested through the Conjur API

Postgres Database Access

Description

Dynamic secrets for PostgreSQL make it possible to issue short-lived, limited scope credentials on-demand for users or services to access a PostgreSQL database.

Prerequisites

A credential factory for PostgreSQL would need to be initially created with:

1. Privileged access credentials that can be used to create new database users
2. PostgreSQL in which to create dynamic users
3. (Optional) CREATE template for customizing the user creation.

Example policy for creating a PostgreSQL credential factory:

Implementation Note: Named create templates (e.g. read, read_and_write) could be provided to make it easier to specify alternative behavior for security personas, as well as make them more reusable.

- !webservice psql-credentials
  annotations:
    credential-factory/provider: postgresql
    credential-factory/postgresql/user: postgres
    credential-factory/postgresql/password: psql/dev-account/root-password
    credential-factory/postgresql/database: app-db-dev
    credential-factory/postgresql/template: |
      CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' 
      VALID UNTIL '{{expiration}}'; 
      GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";

- !layer

- !permit
  resources: !webservice psql-credentials
  privileges: [ read, execute ]
  role: !layer

Workflow

1. The requester authenticates to Conjur using their user or host credentials

2. The requestor asks the PostgreSQL credential factory for a new database user

   GET https://conjur.myorg.net/secrets/cucumber/webservice/psql

3. Conjur generates a new random username and password combination and connects to the database using the configured credentials to create a new user.

4. Conjur returns the new credentials to the requester

   {
       "id": "cucumber:variable:cucumber/webservice/psql/C3hg25bCS3",
       "user": "C3hg25bCS3",
       "password": "a85-wPLyFs6zvBMZ5ZS^"
   }

5. The requester uses the username/password to connect to the database.

6. Either when the TTL expires or the requester uses the Conjur API, Conjur connects to the database and drops the dynamic user.

---

Pending Questions/Topics

Routing HTTP Verb

Credential factories are intended to be used very similarly to variables in fetching the secret value. Because of this, the secret generation and returning is currently specified as using the GET route for secrets. Because this is generating and creating a new secret, it may be better to use the POST route to generate the secret, and then use the GET to fetch its value.

---

Resources:

• https://github.com/cyberark/conjur/compare/feature/credential-factory

[dustinmm80]

imo it's a bit strange that you have to worry about case with provider.

"credential-factory/provider: PostgreSQL"

Is there a standard list of names we can create or just copy from (like from dockerhub or something)?

To avoid ambiguity, can this be an enum or something, where the options are very well defined?

[micahlee]

@dustinmm80, good point. My assumption when writing this is that the provider lookup would be case insensitive, so postgresql and PostgreSQL would resolve to the same provider. I need to check to see how that works with other plugins though.

[kgilpin]

We should decide if annotation names will use dashes or underscores.

[kgilpin]

We could provide a web service endpoint which lists the names of the available credential factory names.

[jonahx]

I had only had two comments. The key one is that this:

    credential-factory/create_template: |
      CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' 
      VALID UNTIL '{{expiration}}'; 
      GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";

feels odd to me. I would expect this to belong in the code, not in the policy. So there would be 1 or more standard default templates that you'd reference by name, with the ability to plug in your own in the same way you can plug in rotators or authenticators.

Even though this might be slightly less convenient, it seems far less prone to error, and doesn't mix concerns.

GET https://conjur.myorg.net/secrets/cucumber/webservice/aws

This should be a POST, right? It's creating a new secret...

[jonahx]

Oh I forgot to mention, the other, even more important, reason to have named templates in the codebase is so they can be reused across policies.

[micahlee]

Thanks for the comments!

I checked the rotators to confirm that the expectation there is for annotations to be all lower-case with dashes, so I've updated the examples accordingly.

[jonahx]

After discussing with Kevin, he convinced me that a GET could be appropriate here, and should be used. From that discussion:

if it’s that, i’m fine with it, that fits in with semantics of http. i’d just want that documented so that other devs don’t have the same confusion i had when seeing the GET. Just a note like:

NB: The entire process of creating credentials on the fly is to be considered an implementation detail, and the “resource” that the client is retrieving represents “the currently valid, secure dynamic secret”

[dividedmind]

Generally, I wonder whether this should belong in Conjur core; we seem to roll everything into it these days and I'm not 100% happy with that: it breaks separation of concerns and presents a potential security problem. I also share concerns regarding putting policy and SQL templates in annotations; we seem to try to do everything in policies and I'm not sure if that's a good idea in the long run.

Perhaps this should be a separate service instead, at least conceptually; this way eg. setting up templates could offer a richer interface (say, you could POST conjur.example.org/account/credential-factories with the settings and then, say POST conjur.example.org/account/credential-factories/:id/iam-policy to set up the policy). The only thing that's missing to support this generically is to allow setting up some hook to say that a variable is backed by another service, and an interface for that. Note this could be useful not only for credential factories, but also things like CYBR Vault proxying for specific variables.

Some more specific comments:

credential-factory/aws/region: us-east-1
credential-factory/aws/access-key-id: "23478207027842073230762374023"
# reference to a Conjur variable
credential-factory/aws/secret-access-key: aws/dev-account/secret-access-key 

I find this weird: you'll have to change the annotation on credential factory when you rotate the access key.

The credential factory will return the token to the requester

   "id": "cucumber:variable:cucumber/webservice/aws/ASIAI6...F7KPBKA",
   "token": "{ \"access_key_id\": \"ASIAI6...F7KPBKA\", \"secret_access_key\": \"EhyQYceI...arRtkZfYugB9dDM6\",  \"session_token\": \"FQoDYXdzEH4aD...7bz7Nw+NL0mKNr8ztYF\",  \"expiration\": \"2018-04-15 22:04:26 UTC\",  \"federated_user_id\": \"<acct-id>:myapp\", \"federated_user_arn\": \"arn:aws:sts::<acct-id>:federated-user\/myapp\"}"
}

COMMENT: Alternatively, the id could be provided as a response header

This seems wrong from the HTTP perspective. If conceptually it's creating a new resource at a different location, it should instead return 302 Found redirecting to the target variable (regardless of whether it was just created or it had been already present), eg.

GET /secrets/cucumber/webservice/aws HTTP/1.1
Host: conjur.example.org

HTTP/1.1 302 Found
Location: /secrets/cucumber/variable/cucumber/webservice/aws/ASIAI6...F7KPBKA

(BTW, we should use standard RFC 6761 reserved domains such as example.org.)

Routing HTTP Verb

It's GET. Period. :)

[kgilpin]

We don’t want the access key id in an annotation value. It should be stored in a referenced variable. The way that the AWS credential rotator works is a fine example.

On Fri, Aug 3, 2018 at 4:47 PM Rafał Rzepecki notifications@github.com wrote:

Generally, I wonder whether this should belong in Conjur core; we seem to roll everything into it these days and I'm not 100% happy with that: it breaks separation of concerns and presents a potential security problem. I also share concerns regarding putting policy and SQL templates in annotations; we seem to try to do everything in policies and I'm not sure if that's a good idea in the long run.

Perhaps this should be a separate service instead, at least conceptually; this way eg. setting up templates could offer a richer interface (say, you could POST conjur.example.org/account/credential-factories with the settings and then, say POST conjur.example.org/account/credential-factories/:id/iam-policy to set up the policy). The only thing that's missing to support this generically is to allow setting up some hook to say that a variable is backed by another service, and an interface for that. Note this could be useful not only for credential factories, but also things like CYBR Vault proxying for specific variables.

Some more specific comments:

credential-factory/aws/region: us-east-1 credential-factory/aws/access-key-id: "23478207027842073230762374023"

reference to a Conjur variable

credential-factory/aws/secret-access-key: aws/dev-account/secret-access-key

I find this weird: you'll have to change the annotation on credential factory when you rotate the access key.

The credential factory will return the token to the requester

"id": "cucumber:variable:cucumber/webservice/aws/ASIAI6...F7KPBKA", "token": "{ \"access_key_id\": \"ASIAI6...F7KPBKA\", \"secret_access_key\": \"EhyQYceI...arRtkZfYugB9dDM6\", \"session_token\": \"FQoDYXdzEH4aD...7bz7Nw+NL0mKNr8ztYF\", \"expiration\": \"2018-04-15 22:04:26 UTC\", \"federated_user_id\": \":myapp\", \"federated_user_arn\": \"arn:aws:sts:::federated-user\/myapp\"}" }

COMMENT: Alternatively, the id could be provided as a response header

This seems wrong from the HTTP perspective. If conceptually it's creating a new resource at a different location, it should instead return 302 Found redirecting to the target variable (regardless of whether it was just created or it had been already present), eg.

GET /secrets/cucumber/webservice/aws HTTP/1.1Host: conjur.example.org

HTTP/1.1 302 FoundLocation: /secrets/cucumber/variable/cucumber/webservice/aws/ASIAI6...F7KPBKA

(BTW, we should use standard RFC 6761 reserved domains such as example.org .)

Routing HTTP Verb

It's GET. Period. :)

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/cyberark/conjur/issues/659#issuecomment-410372130, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFRe2TZJ601ppLFmchfFGXm3vPnXGc4ks5uNLb0gaJpZM4Vqpp8 .

[kgilpin]

The reasons that drive the use of policies to configure new features are that policy is a well defined, access controlled, audited, secure, documented way to get metadata into Conjur.

What drives the architecture of Conjur as a mono-repo is that its easy to contribute, easy to deploy, easy to manage, and easy to secure. And also, there’s no drive from users to deploy only parts of the code and not others.

It’s basically a field decision which features to enable for a given deployment.

On Fri, Aug 3, 2018 at 5:36 PM Kevin Gilpin kevin.gilpin@alum.mit.edu wrote:

We don’t want the access key id in an annotation value. It should be stored in a referenced variable. The way that the AWS credential rotator works is a fine example.

On Fri, Aug 3, 2018 at 4:47 PM Rafał Rzepecki notifications@github.com wrote:

Generally, I wonder whether this should belong in Conjur core; we seem to roll everything into it these days and I'm not 100% happy with that: it breaks separation of concerns and presents a potential security problem. I also share concerns regarding putting policy and SQL templates in annotations; we seem to try to do everything in policies and I'm not sure if that's a good idea in the long run.

Perhaps this should be a separate service instead, at least conceptually; this way eg. setting up templates could offer a richer interface (say, you could POST conjur.example.org/account/credential-factories with the settings and then, say POST conjur.example.org/account/credential-factories/:id/iam-policy to set up the policy). The only thing that's missing to support this generically is to allow setting up some hook to say that a variable is backed by another service, and an interface for that. Note this could be useful not only for credential factories, but also things like CYBR Vault proxying for specific variables.

Some more specific comments:

credential-factory/aws/region: us-east-1 credential-factory/aws/access-key-id: "23478207027842073230762374023"

reference to a Conjur variable

credential-factory/aws/secret-access-key: aws/dev-account/secret-access-key

I find this weird: you'll have to change the annotation on credential factory when you rotate the access key.

The credential factory will return the token to the requester

"id": "cucumber:variable:cucumber/webservice/aws/ASIAI6...F7KPBKA", "token": "{ \"access_key_id\": \"ASIAI6...F7KPBKA\", \"secret_access_key\": \"EhyQYceI...arRtkZfYugB9dDM6\", \"session_token\": \"FQoDYXdzEH4aD...7bz7Nw+NL0mKNr8ztYF\", \"expiration\": \"2018-04-15 22:04:26 UTC\", \"federated_user_id\": \":myapp\", \"federated_user_arn\": \"arn:aws:sts:::federated-user\/myapp\"}" }

COMMENT: Alternatively, the id could be provided as a response header

This seems wrong from the HTTP perspective. If conceptually it's creating a new resource at a different location, it should instead return 302 Found redirecting to the target variable (regardless of whether it was just created or it had been already present), eg.

GET /secrets/cucumber/webservice/aws HTTP/1.1Host: conjur.example.org

HTTP/1.1 302 FoundLocation: /secrets/cucumber/variable/cucumber/webservice/aws/ASIAI6...F7KPBKA

(BTW, we should use standard RFC 6761 reserved domains such as example.org.)

Routing HTTP Verb

It's GET. Period. :)

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/cyberark/conjur/issues/659#issuecomment-410372130, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFRe2TZJ601ppLFmchfFGXm3vPnXGc4ks5uNLb0gaJpZM4Vqpp8 .

[pbramesh1979]

Hi,

I have a use case of connecting from AWS Lambda to an API which is hosted on an on premise server. I would like to use a rotating credentials. I assume that my Lambda function can call some API to get the credentials from Conjur and call the on prem API using that and that API can contact Conjur to verify the credentials.

Could you please let me know how this can be implemented in Conjur.

I am able to create the Conjur instance using the yaml template provided on the Conjur.org (https://docs.conjur.org/Latest/en/Content/OSS/Installation/CloudFormationTemplate.htm)