Feature/use uri not curl 123

[lcndsmr]

Close #123 Close #266 Close #265 Close #11

[lcndsmr]

i dont understand why the test fails, it cant create the logstash user because the password isn't hashable or something? It works locally, can someone look into it?

Error Message on elasticsearch 7:

"reason": "Provided password hash uses [NOOP] but the configured hashing algorithm is [BCRYPT]", "root_cause": [{"reason": "Provided password hash uses [NOOP] but the configured hashing algorithm is [BCRYPT]",

Error Message on elasticsearch 8:

"reason": "The provided password hash is not a hash or it could not be resolved to a supported hash algorithm. The supported password hash algorithms are [bcrypt, bcrypt4, bcrypt5, bcrypt6, bcrypt7, bcrypt8, bcrypt9, bcrypt10, bcrypt11, bcrypt12, bcrypt13, bcrypt14, pbkdf2, pbkdf2_1000, pbkdf2_10000, pbkdf2_50000, pbkdf2_100000, pbkdf2_500000, pbkdf2_1000000, pbkdf2_stretch, pbkdf2_stretch_1000, pbkdf2_stretch_10000, pbkdf2_stretch_50000, pbkdf2_stretch_100000, pbkdf2_stretch_500000, pbkdf2_stretch_1000000]"

Can someone look into it? @widhalmt @afeefghannam89

[afeefghannam89]

@dnssmr Did I understand correctly, this problem appears only in the pipelines/containers but NOT on VMs or servers?

Can you please guide me to the Pipeline or to the place of this issue?

[afeefghannam89]

@dnssmr I found it thanks.

[afeefghannam89]

So I give up here :( I do not understand why setting logstash writer user here has a problem with hashing algorithm! I did really change the algorithm to bcrypt, what is the default by Elastic. But here Elastic do not accept bcrypt 12 I did not find a way to NOT use version 12.

Locally using THE SAME CONTAINER I do not face this problem.

@dnssmr should I undo my changes?

[widhalmt]

How about https://github.com/NETWAYS/ansible-collection-elasticstack/issues/132 ? If it's "only" a problem with the user management, we can keep curl for this single task and later replace it with a module. Changing all other curl commands to uri is a big benefit by it's own.

[afeefghannam89]

It is only a problem with GitHub. User management do really work locally without problem on both environments containers and VMs using both distributions Ubuntu and rocky. This problem reminds me with the problem with wait_for tasks.

When we skip adding the logstash writer user, I do not think that the task/s checking for written data in verify.yml will success, do they?

[lcndsmr]

@afeefghannam89 @widhalmt i will review the changes later or tomorrow and then test again using command module and curl for this task.

If that works, we can keep it for now, and i will change the molecule tests (there are some curls in command tasks there).

I just don't understand what is wrong here, because it works locally! And the EXACT SAME THING with the role instead of the user works without problem.

[afeefghannam89]

@widhalmt Sorry I understood you wrong. I did not test keeping command module on this task! As I implemented the user encryption in an old branch, the pipeilines did not have problems. Maybe your solution will work.

[widhalmt]

Why it sometimes works and sometimes it doesn't is beyond me, too. I'd suspect different libraries to be installed. But usually Elastics monolithic packages don't use external libraries. And all Ansible related libraries shouldn't have any effects.

To be honest, I really want to know but I don't see any sense in investing enough time to find out where there's a possible workaround.

[widhalmt]

I guess I found a related issue: https://github.com/elastic/elasticsearch/issues/51132 So maybe the process of hashing must be done with another tool / library? It seems, even Elastic isn't completely sure if the hash is compatible or not.

[widhalmt]

I was bold enough to merge this branch with main thus incorporating the latest changes @afeefghannam89 made to the testing configuration. I guess now we'll get a way better view at what works and what not.

[danopt]

@dnssmr Maybe I am completely wrong. I tried to use the password_hash function locally with the 'bcrypt' algorithm. Interestingly enough the function doesn't fail, but gives me a corrupted password hash:

TASK [Display Hashed Password] ***********************************************************************************************************************************************
ok: [localhost] => {
    "hashed_password": "*0"
}

After I installed passlib with pip, the function works:

TASK [Display Hashed Password] ***********************************************************************************************************************************************
ok: [localhost] => {
    "hashed_password": "$2b$12$u1Zp45pE3eXArHdR9MIcYOfSPtzLSowaiKHHbhu972TBSoy9KqHie"
}

Edit: Same issue with 'blowfish' algorithm.

So I'm not sure if passlib is installed on the Ansible controller (runner). Maybe we should debug the content of the logstash_writer_user file to get the password hash?

If this is the reason, we can install passlib on the runner, since templating is done by the controller and not the remote node itself.

[afeefghannam89]

@DanOPT passlib must be installed when using ‘blowfish’ (BCrypt). This information is available under requirements-test.txt but not in the documentation.

[danopt]

@DanOPT passlib must be installed when using ‘blowfish’ (BCrypt). This information is available under requirements-test.txt but not in the documentation.

Ok, I missed that it is already in the requirements-test.txt.