Closed infoMatt closed 4 years ago
wojtek0806
commented
5 years ago Have you followed this when filing an issue, from the above it does not seem so:
https://clouddocs.f5.com/products/orchestration/ansible/devel/usage/filing-issues.html
Once you test with the latest role and complete the issue template with the required information I will reopen it
infoMatt
commented
5 years ago I've updated the first post, to include the debug info and the tests with latest galaxy module. No difference.
wojtek0806
commented
5 years ago @infoMatt we do not use SDK, SDK will be removed in 2.8. I do not see a reproduction playbook that you are trying to use to update key/cert. Please provide full YAML that is used to reproduce the above error.
infoMatt
commented
5 years ago @wojtek0806 I don't have anything in the playbook beyond what is actually stated in the examples... it's something like this:
- name: Update certificate public
bigip_ssl_certificate:
name: testAnsible
server: ltmtest1
provider:
user: ******
password: *******
state: present
content: "{{ lookup('file', '/var/cert/testAnsible/testAnsible.crt') }}"
validate_certs: no
delegate_to: localhost
# IT WILL FAIL AT THIS POINT with 01070317:3: profile /Common/test-ansible\\'s key and certificate do not match
- name: Update certificate private
bigip_ssl_key:
name: testAnsible
server: ltmtest1
provider:
user: ******
password: ******
state: present
content: "{{ lookup('file', '/var/cert/testAnsible/testAnsible.key') }}"
validate_certs: no
delegate_to: localhostThe point is, however, that if the certificate and key pair is used in a SSL profile, the update procedure must be transactional, otherwise there's a moment between the two tasks where the private key won't match the public one (and vice-versa), and thus it will fail with the error stated in the original message (01070317:3: profile /Common/test-ansible\\'s key and certificate do not match, where /Common/test-ansible is a client ssl profile that uses the testAnsible.crt and testAnsible.key).
The scope of the example (and my request) is to specify in one of the two modules (bigip_ssl_certificate and/or bigip_ssl_key) both the public and private part, and in this case:
OR, and it might be a better idea after all, add a transaction management module (begin/commit/rollback), and implement a transaction parameter in all the modules.
If I've understood correctly, transactions can be opened/committed or aborted with the REST API, and to append a command to a transaction the API call needs to have the X-F5-REST-Coordination-Id header, that can be obtainted from a previous (hypothetical) bigip_transaction module execution results...
https://f5-automation-labs-von.readthedocs.io/en/latest/module1/lab7.html
In this last example, an hypothetical playbook will be something like:
- name: Begin transactional update
bigip_transaction:
state: create
server: ltmtest1
provider:
user: ******
password: *******
validate_certs: no
delegate_to: localhost
register: f5_transaction
- name: Update certificate public
bigip_ssl_certificate:
name: testAnsible
server: ltmtest1
provider:
user: ******
password: *******
state: present
content: "{{ lookup('file', '/var/cert/testAnsible/testAnsible.crt') }}"
validate_certs: no
transaction_id: "{{ f5_transaction.transId }}" ### ADDED
delegate_to: localhost
- name: Update certificate private
bigip_ssl_key:
name: testAnsible
server: ltmtest1
provider:
user: ******
password: ******
state: present
content: "{{ lookup('file', '/var/cert/testAnsible/testAnsible.key') }}"
validate_certs: no
transaction_id: "{{ f5_transaction.transId }}" ### ADDED
delegate_to: localhost
- name: Commit transactional update
bigip_transaction:
state: commit
server: ltmtest1
provider:
user: ******
password: *******
validate_certs: no
transaction_id: "{{ f5_transaction.transId }}"
delegate_to: localhost
register: f5_transactiontransaction context manager is already implemented and used in few modules, but we will not be doing that for all the modules, as I do not plan to use transactions where it is not necessary. If it makes sense to do this here I will implement it. And certainly won't be as above, transactions will happen without the user's knowledge.
infoMatt
commented
5 years ago Yes, please... in this case there's definitely the need for a transactional update, otherwhise certificates can't be upgraded in place... It would be necessary to upload every cert revision with a different name, and change all the client ssl profiles that use the old certificate... it would be a nightmare!!
infoMatt
commented
5 years ago transactions will happen without the user's knowledge.
In most case I'll agree with you, but transactions can be used also in TMSH, and in some circumstances it might be advisable to "complete all or fail as it wouldn't have happened"... say a complex definition of a virtual, with balancing pool and nodes... if there's an error on a node step, it doesn't make sense to leave other definitions behind... (Maybe I'm thinking as a programmer here, I might admit that...)
wojtek0806
commented
5 years ago ansible core modules do not support roll-back mechanism, what they do support is doing a diff which allows the user to see what will be changed, which is going to be implemented at some point in F5 modules. As far as transactions are concerned, I have no intention of doing transactions for modules in the near future.
wojtek0806
commented
5 years ago FMFA-565
amolari
commented
4 years ago Any chance to get that RFE implemented soon?
pcloup
commented
4 years ago Hi @infoMatt , First, the issue you mentioned is captured in the following entry in our system (FMFA-237), and not the one mentioned by @wojtek0806 in April ##(FMFA-234).
i have played with your scenario on my side, and tried multiple options. I tested the transaction with ANSIBLE URI module, as as you noted, the "transaction header" is not accessible in F5 modules. Here after the playbook i created for that ( this is not a production proven playbook, and there is alot of things that need to be changed to make it more production ready ;-)), like some hardcoded values and names, no provider configured, no collections, ... ;-))
tasks:
- name: Authenticate to BIG-IP with creds to get token
uri:
url: "https://{{ my_bigip }}/mgmt/shared/authn/login"
method: POST
body:
username: "{{ my_admin }}"
password: "{{ my_password }}"
loginProviderName: "tmos"
body_format: json
return_content: yes
validate_certs: no
register: resultat
- name: store the auth tokens
set_fact:
Token: "{{ resultat.json.token.token }}"
- uri:
url: https://{{ my_bigip }}/mgmt/tm/transaction
headers:
"X-F5-Auth-Token": "{{ Token }}"
return_content: yes
body_format: json
body: "{}"
method: POST
validate_certs: no
register: resultat
- name: store the transaction ID
set_fact:
transId: "{{ resultat.json.transId }}"
- name: Upload the cert Pub pem file
uri:
url: https://{{ my_bigip }}/mgmt/shared/file-transfer/uploads/pipomolocrtv2.crt
headers:
"X-F5-Auth-Token": "{{ Token }}"
"Content-Range": "0-1913/1914"
return_content: yes
body_format: raw
body: "{{ lookup('file','./certv1.pem') }}"
method: POST
validate_certs: no
register: resultat
- name: Upload the cert key pem file
uri:
url: https://{{ my_bigip }}/mgmt/shared/file-transfer/uploads/pipomolokeyv2.key
headers:
"X-F5-Auth-Token": "{{ Token }}"
"Content-Range": "0-1702/1703"
return_content: yes
body_format: raw
body: "{{ lookup('file','./privkeyv1.pem') }}"
method: POST
validate_certs: no
register: resultat
- name: updated the key in the SSL Cert object
uri:
url: https://{{ my_bigip }}/mgmt/tm/sys/crypto/key
headers:
"X-F5-Auth-Token": "{{ Token }}"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: json
body: {"command": "install","name":"/Common/pipomolo","from-local-file":"/var/config/rest/downloads/pipomolokeyv2.key"}
method: POST
validate_certs: no
register: resultat
- name: updated the Cert Pub in the SSL Cert object
uri:
url: https://{{ my_bigip }}/mgmt/tm/sys/crypto/cert
headers:
"X-F5-Auth-Token": "{{ Token }}"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: json
body: {"command": "install","name":"/Common/pipomolo","from-local-file":"/var/config/rest/downloads/pipomolocrtv2.crt"}
method: POST
validate_certs: no
register: resultat
- name: Commit the Transaction
uri:
url: https://{{ my_bigip }}/mgmt/tm/transaction/{{transId}}
headers:
"X-F5-Auth-Token": "{{ Token }}"
return_content: yes
body_format: json
body: {"state":"VALIDATING"}
method: PATCH
validate_certs: no
register: resultat
If the certificate is not linked to a PROFILE (CLIENTSSL or SERVERSSL), this will work. Even more, If you remove the Transaction, it will work too.
The issue in reality is not an ANSIBLE module issue here. The issue is that you can't update an existing cert+key pair in a cert object, when this one is in use ini a profile. The only way to do it is to use ARCHIVE or PKCS12 format, which contains BOTH CERT and KEY. And this accepts the "in place" update/replacement of those 2 files at once.
If you try with the TMUI (F5 GUI) or TMSH, you will face the exact same issue and challenge.
So what i did is a playbook that is doing the following:
- name: Authenticate to BIG-IP with creds to get token
uri:
url: "https://{{ my_bigip }}/mgmt/shared/authn/login"
method: POST
body:
username: "{{ my_admin }}"
password: "{{ my_password }}"
loginProviderName: "tmos"
body_format: json
return_content: yes
validate_certs: no
register: resultat
- name: store the auth tokens
set_fact:
Token: "{{ resultat.json.token.token }}"
- name: Generate PKCS#12 file
openssl_pkcs12:
action: export
path: ./pipomolo.p12
friendly_name: pipomolo
privatekey_path: ./privkeyv1.pem
certificate_path: ./certv1.pem
- name:copy the p12 file from local to BIG-IP (/var/config/rest/downloads/)
# this part has been removed intentionally as the
# lookup file plugin does not accept binary format for files
# there is a way to SCP the file to the destination as a workaround
# but i had not the time to play with the workaround here.
# and it is not the topic
- name: install the PKCS12 in BIGIP SSL Certs
uri:
url: https://{{ my_bigip }}/mgmt/tm/sys/crypto/pkcs12
headers:
"X-F5-Auth-Token": "{{ Token }}"
return_content: yes
body_format: json
body: {"command": "install","name":"/Common/pipomolo","from-local-file":"/var/config/rest/downloads/pipomolo.p12"}
method: POST
validate_certs: no
register: resultat
If you try to do the same task with the TMUI (F5 GUI) or TMSH, that works too.
HTH
infoMatt
commented
4 years ago Hi @pcloup, the last part of your message covers our needs and doubts that we had the last year, as I haven't noticed a task that could be used to install a PKCS12 or the pair (cert+key) with a deferred check of the coherence of the pair itself, and this is why I was thinking of using the transaction.
The project I was working on unfortunately was put aside for various reasons, but I might still try to check the feasibility of this solution and give a feedback in the next couple of weeks.
Thanks!
amolari
commented
4 years ago @pcloup I've tested your solution. It works. One can upload the file with the uri module in such way:
- name: Upload PKCS12 file to BIGIP
uri:
url: https://{{ ansible_host }}:{{ bigip_port }}/mgmt/shared/file-transfer/uploads/pipomolo.p12
user: "{{ bigip_admin_username }}"
password: "{{ bigip_admin_password }}"
headers:
Content-Type: application/octet-stream
Accept: application/json
Content-Range: "0-{{ my_pkcs12_file.stat.size - 1 }}/{{ my_pkcs12_file.stat.size }}"
force_basic_auth: yes
method: POST
src: ./pipomolo.p12
validate_certs: no
return_content: no
delegate_to: localhostNote: Installing the certificate with the API call https://{{ ansible_host }}:{{ bigip_port }}/mgmt/tm/sys/crypto/pkcs12
won't add the suffix .key and .crt to the objects. When creating the client SSL profile, one should use the option true_names: yes
Remains the fact that this is not an idempotent solution
amolari
commented
4 years ago @pcloup , you wrote
The issue in reality is not an ANSIBLE module issue here. The issue is that you can't update an existing cert+key pair in a cert object, when this one is in use ini a profile.
The SDK documentation above mentionned here let on think, it would be possible (Ansible module "combined" for cert/key/chain): (https://f5-sdk.readthedocs.io/en/latest/userguide/transactions.html)
pcloup
commented
4 years ago Hi @amolari, I am double checking internally if what is mentioned in the F5-SDK documentation you mentioned is still valid, or if it is due to different REST API entry points (or even some old SOAP XML embedded code somewhere). Hope to update this thread shortly.
amolari
commented
4 years ago @pcloud, it seems that a few F5 Ansible modules (bigip_provision, bigip_pool_member, bigip_gtm_pool_member, bigip_pool) use transactions => ansible_collections.f5networks.f5_modules.plugins.module_utils.icontrol import TransactionContextManager
pcloup
commented
4 years ago sorry @amolari, my point was not around transaction being used or not in F5 ansible modules. My point was related to the issue you face when you try to update the CERT and KEY when they are linked to SSL profiles. And using Transaction, on my side, did not make it as mentioned, and even if you try to do it with TMSH or the GUI, you face the same issue, except if you use archive or PKCS12. So my investigations actually is around the F5 SDK documentation you mentioned, and if my testings were wrong by targeting potentially the wrong REST entry points (/tm/sys/crypto) or if there has been a change on behaviour.
stay tuned
focrensh
commented
4 years ago Tracking with FMFA-565
pcloup
commented
4 years ago Hi All, i know that this issue has been captured and is actually moving forward with development team, but in //, i wanted just to confirm that my previous tests were a bit messy (i used the wrong private key with the wrong public cert). So my findings are that with the following playbook, if you change the private key file and public key file accordingly, together with the TRANSACTION. Here after the playbook i am using (and you change the cert and key file name in the palybook if you want to check, of be more creative than me and make the name a variable of your playbook to make it portable ;-)).
---
- name: This playbook update Pub/Key of an existing cert linked to a SSL Profile
hosts: pclolab
connection: local
gather_facts: no
vars:
my_admin: "{{ admin }}"
my_password: "{{ password }}"
my_bigip: "{{ bigip }}"
tasks:
- name: Authenticate to BIG-IP with creds to get token
uri:
url: "https://{{ my_bigip }}/mgmt/shared/authn/login"
method: POST
body:
username: "{{ my_admin }}"
password: "{{ my_password }}"
loginProviderName: "tmos"
body_format: json
return_content: yes
validate_certs: no
register: resultat
- name: store the auth tokens
set_fact:
Token: "{{ resultat.json.token.token }}"
- uri:
url: https://{{ my_bigip }}/mgmt/tm/transaction
headers:
"X-F5-Auth-Token": "{{ Token }}"
return_content: yes
body_format: json
body: "{}"
method: POST
validate_certs: no
register: resultat
- name: store the transaction ID
set_fact:
transId: "{{ resultat.json.transId }}"
- name: Upload the Pub SSL pem file
uri:
url: https://{{ my_bigip }}/mgmt/shared/file-transfer/uploads/pipomolocrt.crt
headers:
"X-F5-Auth-Token": "{{ Token }}"
"Content-Range": "0-1913/1914"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: raw
body: "{{ lookup('file','./certv1.pem') }}"
method: POST
validate_certs: no
register: resultat
- name: Upload the key SSL pem file
uri:
url: https://{{ my_bigip }}/mgmt/shared/file-transfer/uploads/pipomolokey.key
headers:
"X-F5-Auth-Token": "{{ Token }}"
"Content-Range": "0-1702/1703"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: raw
body: "{{ lookup('file','./privkeyv1.pem') }}"
method: POST
validate_certs: no
register: resultat
- name: update the key in the SSL Cert object
uri:
url: https://{{ my_bigip }}/mgmt/tm/sys/crypto/key
headers:
"X-F5-Auth-Token": "{{ Token }}"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: json
body: {"command": "install","name":"/Common/pipomolo","from-local-file":"/var/config/rest/downloads/pipomolokey.key"}
method: POST
validate_certs: no
register: resultat
- name: update the Cert Pub in the SSL Cert object
uri:
url: https://{{ my_bigip }}/mgmt/tm/sys/crypto/cert
headers:
"X-F5-Auth-Token": "{{ Token }}"
"X-F5-REST-Coordination-Id": "{{ transId }}"
return_content: yes
body_format: json
body: {"command": "install","name":"/Common/pipomolo","from-local-file":"/var/config/rest/downloads/pipomolocrt.crt"}
method: POST
validate_certs: no
register: resultat
- name: Commit the Transaction
uri:
url: https://{{ my_bigip }}/mgmt/tm/transaction/{{transId}}
headers:
"X-F5-Auth-Token": "{{ Token }}"
return_content: yes
body_format: json
body: {"state":"VALIDATING"}
method: PATCH
validate_certs: no
register: resultat
ISSUE TYPE
COMPONENT NAME
bigip_ssl_certificate bigip_ssl_key
ANSIBLE VERSION
PYTHON VERSION
BIGIP VERSION
CONFIGURATION
Error appearing using both the RH version of ansible, and with the versions v2019.2.1 and master (as of today) of the f5ansible galaxy module.
OS / ENVIRONMENT
Red Hat Enterprise Linux 7.6
SUMMARY
There's no way, using the current implementation of the modules bigipssl{certificate, key}, to update in place an existing certificate with a renewed one (with different keys). Updating it's public or private key will indeed cause an error message like "profile /Common/\'s key and certificate do not match".
Looking in the SDK documentation there's a specific example that matches this use case, and it involves the use of Transactions: https://f5-sdk.readthedocs.io/en/latest/userguide/transactions.html
Ideally, the bigip_ssl_certificate module should have another parameter, say for example "key_content", that, when used, will cause a transactional upload of both files (public and private), following the example in the docs.
STEPS TO REPRODUCE
EXPECTED RESULTS
Transactional update of both public and private keys of certificate.
With the tmsh utility, this can be accomplished uploading a .p12 file (that contains both the public and the private key at the same time):
ACTUAL RESULTS
The certificate upload fails with the error "key and certificate do not match", because it will update just the public key, not the private one (or the opposite).
(hostname and certificate name redacted).
Thanks!