Update tasks to use cert_info plugin

[danopt]

Fixes #161. Update certificate handling tasks and docs

[widhalmt]

Linter fails because of #159 . We already have a PR #160 and only wait for one last approve from someone who requested changes.

[danopt]

Tasks have been updated to use cert_info plugin in:

• Elasticsearch role
• Kibana role
• Logstash role

I verified the renewing mechanism in the Elasticsearch role and Kibana role. I'm still checking why I can't get the mechanism to work in the Logstash role, but the plugin and the task itself are working fine.

I will double check one more time. It'll be ready for review on Friday.

[widhalmt]

Please note, that we have to use different formats for certificates with the different tools. Logstash for example forces us to use PKCS8 certificates. And there's a bug that doesn't allow to use passwords with all the certificates. So some are encrypted and some aren't. :-(

[danopt]

The task which checks the not_valid_after date in the Logstash role uses a .p12 certificate. The module can check that date.

I think I found a minor bug, because the logstash role won't copy the .p12 to the logstash_certs_dir but maybe someone can verify that for me who knows more about the creation process of the certificates of this role.

I copied the certificate from /opt/es-ca/ to {{ logstash_certs_dir }}/{{ ansible_hostname }}-ls.p12. After that the renewing mechanism was triggered. So the task works fine for the logstash role, too.

[afeefghannam89]

@DanOPT Thank you very much for this notice. You are right. I have fixed it.

The task which checks the not_valid_after date in the Logstash role uses a .p12 certificate. The module can check that date.

I think I found a minor bug, because the logstash role won't copy the .p12 to the logstash_certs_dir but maybe someone can verify that for me who knows more about the creation process of the certificates of this role.

I copied the certificate from /opt/es-ca/ to {{ logstash_certs_dir }}/{{ ansible_hostname }}-ls.p12. After that the renewing mechanism was triggered. So the task works fine for the logstash role, too.

[lcndsmr]

Tested on existing nps elastic - failed on beats role

[widhalmt]

My central systems seem to lack a python cryptography module. Maybe they are too old (CentOS 7). Newer systems where I tested, failed - all of them are hosts with only beats.

[widhalmt]

I just checked. The required python-cryptography package is not available in a version >=2.5. So CentOS 7 won't be easy to support. Since CentOS 7 is still supported until June 2024 I don't want to drop the support all together. What do you think of building an if that doesn't check for certificate age on CentOS 7? What that be easily added? Or could we lower the bar for the dependency to the library?

[afeefghannam89]

@dnssmr beat role is missing or failed on beat role?

Can you please post the Error, thanks

[afeefghannam89]

I just checked. The required python-cryptography package is not available in a version >=2.5. So CentOS 7 won't be easy to support. Since CentOS 7 is still supported until June 2024 I don't want to drop the support all together. What do you think of building an if that doesn't check for certificate age on CentOS 7? What that be easily added? Or could we lower the bar for the dependency to the library?

This package should be installed using pip not through package manager. I would like to write this as requirement in the README, What do you think? Can you please install the cryptography and test it?

[afeefghannam89]

Tested on existing nps-elaszic (centos 8), works like a charm!

Was the beat problem on this cluster?

[widhalmt]

I just checked. The required python-cryptography package is not available in a version >=2.5. So CentOS 7 won't be easy to support. Since CentOS 7 is still supported until June 2024 I don't want to drop the support all together. What do you think of building an if that doesn't check for certificate age on CentOS 7? What that be easily added? Or could we lower the bar for the dependency to the library?

This package should be installed using pip not through package manager. I would like to write this as requirement in the README, What do you think? Can you please install the cryptography and test it?

Yes, I'll install it and test. Anyway, would really love it to work with with packages alone. But that's not a reason to force users to. We can just write it into the dependencies, you're right

[lcndsmr]

@dnssmr beat role is missing or failed on beat role?

Can you please post the Error, thanks

Sorry, i meant the changes are missing in the beat role. Error Message:

TASK [beats : Set the beats certificate expiration date in days] ***************
fatal: [nps-elastic-0]: FAILED! => {"msg": "the field 'args' has an invalid value ({'beats_cert_expiration_days': \"{{ ((beats_cert_expiration_date.stdout | to_datetime('%b %d %H:%M:%S %Y %Z')) - (ansible_date_time.date | to_datetime('%Y-%m-%d'))).days }}\"}), and could not be converted to an dict.The error was: time data 'May 21 10:06:37 2026 GMT' does not match format '%b %d %H:%M:%S %Y %Z'\n\nThe error appears to be in '/home/denise/repositories/shared/nps_elastic_stack/roles/beats/tasks/beats-security.yml': line 29, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set the beats certificate expiration date in days\n  ^ here\n"}

Tested on existing nps-elaszic (centos 8), works like a charm!

Was the beat problem on this cluster?

To be honest, i cant remember if i used the beats role here.

[widhalmt]

I did use the beats role:

fatal: [ceph08]: FAILED! => {"msg": "the field 'args' has an invalid value ({'beats_cert_expiration_days': \"{{ ((beats_cert_expiration_date.stdout | to_datetime('%b %d %H:%M:%S %Y %Z')) - (ansible_date_time.date | to_datetime('%Y-%m-%d'))).days }}\"}), and could not be converted to an dict.The error was: time data 'May  3 11:28:02 2026 GMT' does not match format '%b %d %H:%M:%S %Y %Z'\n\nThe error appears to be in '/home/widhalmt/.ansible/collections/ansible_collections/netways/elasticstack/roles/beats/tasks/beats-security.yml': line 29, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set the beats certificate expiration date in days\n  ^ here\n"}

[afeefghannam89]

@dnssmr Thanks for your feedback. You can approve