Creating a new host using hf token can cause an exception, depending on the name

[ryanprior]

Creating a host called brand-new-host using an hf token causes an exception, while naming it brand_new_host works fine.

Steps to reproduce

1. login to Conjur as admin
2. load this policy as root:

   - !policy
    id: database
    body:
      - !layer users
      - !host-factory
        id: users
        layers: [ !layer users ]

3. create a host factory token: conjur hostfactory tokens create database/users
4. use the token to create a host: conjur hostfactory hosts create $token brand-new-host

Expected result: new host created, id and API key returned Actual result: 500 error

Workaround

If you instead create the host with id brand_new_host, you get the expected result.

Side note

With the request to create a new host, the Conjur CLI is sending some unfamiliar data. Examples:

__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x000000018bf6b8%3E]&id=brand-new-host

__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x00000002bd76b0%3E]&id=brand_new_host

All I'd expect to see is the last part, id=whatever. Why is there a ParentKey with some kind of pointer address or something in there?

Session log

$ conjur hostfactory tokens create database/users
[
  {
    "token": "[redacted]",
    "expiration": "2018-09-21T20:53:24+00:00",
    "cidr": [

    ]
  }
]
$ conjur hostfactory hosts create [redacted] "brand-new-host"
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x000000018bf6b8%3E]&id=brand-new-host", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"1eydjya19ht5xd1tyyf5k8dm7me3gq3g5420nbxhdk84nyr1j5t1r\"", "Content-Length"=>"89", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"
# => 500 InternalServerError | text/html 0 bytes

error: 500 Internal Server Error

$ conjur hostfactory hosts create [redacted] "brand_new_host"
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x00000002bd76b0%3E]&id=brand_new_host", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"1eydjya19ht5xd1tyyf5k8dm7me3gq3g5420nbxhdk84nyr1j5t1r\"", "Content-Length"=>"89", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"
# => 201 Created | application/json 261 bytes
{
  "created_at": "2018-09-21T19:53:56.525+00:00",
  "id": "ryan.prior@cyberark.com:host:brand_new_host",
  "owner": "ryan.prior@cyberark.com:host_factory:database/users",
  "permissions": [

  ],
  "annotations": [

  ],
  "api_key": "[redacted]"
}

[micahlee]

Hey @ryanprior, I'm having trouble reproducing these results. I have a couple of follow up questions that I'm hoping can help refine this for me:

• How are you executing the conjur CLI? Is this native on a host machine, running in docker, etc.?
• Which CLI version are you running via conjur --version?
• What are you using to print out the RestClient data showing the __parent__... data?

Thanks! Micah

[ryanprior]

Annotated full session

Here's the policy I'll load:

mydas-heros:policy ryan$ cat issue-744-policy.yml
- !policy
  id: database
  body:
    - !layer users
    - !host-factory
      id: users
      layers: [ !layer users ]

I start my CLI container and get logged into my account:

mydas-heros:policy ryan$ docker run --rm -it -v $PWD:/policy cyberark/conjur-cli:5
root@ddb5b9b3bd6f:/# conjur --version
conjur version 6.1.0
root@ddb5b9b3bd6f:/# conjur init -u https://eval.conjur.org -a ryan.prior@cyberark.com

SHA1 Fingerprint=0D:84:48:42:78:A4:E5:87:59:4B:8C:BD:81:A4:0C:89:74:C3:AA:A6

Please verify this certificate on the appliance using command:
              openssl x509 -fingerprint -noout -in ~conjur/etc/ssl/conjur.pem

Trust this certificate (yes/no): yes
Wrote certificate to /root/conjur-ryan.prior@cyberark.com.pem
Wrote configuration to /root/.conjurrc
root@ddb5b9b3bd6f:/# conjur authn login -u admin
Please enter admin's password (it will not be echoed): 
Logged in

Now I load my policy:

root@ddb5b9b3bd6f:/# conjur policy load --replace root /policy/issue-744-policy.yml 
Loaded policy 'root'
{
  "created_roles": {
  },
  "version": 28
}
root@ddb5b9b3bd6f:/# conjur hostfactory tokens create database/users
[
  {
    "token": "[redacted]",
    "expiration": "2018-09-26T23:56:05+00:00",
    "cidr": [

    ]
  }
]

This environment variable enables logging in the restclient gem:

root@ddb5b9b3bd6f:/# export RESTCLIENT_LOG=stderr

Now I go to reproduce the behavior I reported:

root@ddb5b9b3bd6f:/# conjur hostfactory hosts create [redacted] "brand-new-host"                                                                                                                
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x
000000016222e8%3E]&id=brand-new-host", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"[redacted]\"", "Content-Length"=>"89", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"                           
# => 500 InternalServerError | text/html 0 bytes

error: 500 Internal Server Error
root@ddb5b9b3bd6f:/# conjur hostfactory hosts create [redacted] "brand_new_host"                                                                                                                 
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x0000000187b2b0%3E]&id=brand_new_host", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"[redacted]\"", "Content-Length"=>"89", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"                           
# => 500 InternalServerError | text/html 0 bytes
error: 500 Internal Server Error

Wait a minute! brand_new_host was supposed to work! What's going on?

root@ddb5b9b3bd6f:/# conjur hostfactory hosts create [redacted] "test"
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x00000002a23508%3E]&id=test", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"[redacted]\"", "Content-Length"=>"79", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"                                     
# => 500 InternalServerError | text/html 0 bytes

error: 500 Internal Server Error
root@ddb5b9b3bd6f:/# conjur hostfactory hosts create [redacted] test123
RestClient.post "https://eval.conjur.org/host_factories/hosts", "__parent__[%23%3CGLI%3A%3ACommand%3A%3AParentKey%3A0x0000000240a4f0%3E]&id=test123", "Accept"=>"*/*", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Token token=\"[redacted]\"", "Content-Length"=>"82", "Content-Type"=>"application/x-www-form-urlencoded", "User-Agent"=>"rest-client/2.0.2 (linux-gnu x86_64) ruby/2.4.1p111"                                  
# => 201 Created | application/json 251 bytes
{
  "created_at": "2018-09-26T23:03:13.808+00:00",
  "id": "ryan.prior@cyberark.com:host:test123",
  "owner": "ryan.prior@cyberark.com:host_factory:database/users",
  "permissions": [

  ],
  "annotations": [

  ],
  "api_key": "[redacted]"
}

Okay then, to recap

These host names got a 500 error

• brand-new-host
• brand_new_host (this time!)
• test

This host name got a 201 created

• test123
• brand_new_host (last time!)

Analysis in light of new behavior

I suspect that this has something to do with creating a new host with the same name as an existing host. I had previously created a host called brand-new-host but didn't think to mention it in the original report. But now that seems like crucial information. The second time around, brand_new_host also failed; it had been first created when I reported the initial bug. Then test failed, and while I don't specifically recall creating test as a host, it seems like something I'd have plausibly created. But then we get to test123 which works fine, and it has the distinction of having never been created before.

If you use the hostfactory to create a host that already exists, the intended behavior is to rotate its API key and return the fresh identity to the hostfactory user. I suspect that instead, we're turning up an exception.

[micahlee]

Thanks Ryan! That's a help clarification. I'll check this again and begin fix shortly!

[micahlee]

@ryanprior, ok I was able to get a reproducible automated test for this. The basic repro steps are:

1. Load policy for a host factory "A"
2. Create a token for host factory A
3. Create a host "my-host" with host factory A
4. Load a new policy REPLACING the existing policy for a host factory "B"
5. Create a token for host factory B
6. Create a host "my-host" with host factory B

The issue is that the policy replace deletes the Resource objects, but not the Roles from the initial policy load. So rather than a 403 forbidden because host factory B doesn't have ownership of "my-host", there is a 500 response because Conjur can't find the resource associated with the existing role "my-host".

It looks like the correct behavior in this scenario would be the 403 forbidden response. Do you agree?