kibana not provisioned, 404 error ?

[mathieuchateau]

What is not working? Provisioning works for everything except kibana. VM is created with nic etc, but no service. I have warning during terraform provisioning

What OS are you using? Mac OS X

Full Terraform / Ansible output?

azurerm_virtual_machine.es_kibana (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'"]
azurerm_virtual_machine.es_kibana (local-exec): Using /Users/mathieuchateau/Local/Adaz-main/ansible/ansible.cfg as config file
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with auto plugin: a batched request failed
azurerm_virtual_machine.es_kibana (local-exec): with status code 404, url
azurerm_virtual_machine.es_kibana (local-exec): /subscriptions/XXXXXXXX/resourceGroups/ad-hunting-
azurerm_virtual_machine.es_kibana (local-exec): lab/providers/Microsoft.Compute/virtualMachines
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with yaml plugin: Plugin configuration YAML
azurerm_virtual_machine.es_kibana (local-exec): file, not YAML inventory
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with ini plugin: Invalid host pattern
azurerm_virtual_machine.es_kibana (local-exec): 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to
azurerm_virtual_machine.es_kibana (local-exec): provide a port.
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: Unable to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml as an inventory source
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: No inventory was parsed, only implicit localhost is available
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: provided hosts list is empty, only localhost is available. Note that
azurerm_virtual_machine.es_kibana (local-exec): the implicit localhost does not match 'all'
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: Could not match supplied host pattern, ignoring: elasticsearch

[christophetd]

Thanks for reporting! Can you confirm that you did all the following?

• Created a virtual environment prior to running the readme commands

python3 -m venv ansible/venv 
source ansible/venv/bin/activate
pip install -r ansible/requirements.txt
deactivate

• You have a SSH key in ~/.ssh/id_rsa.pub

• You tried a second time to make sure it wasn't a random issue with Azure

• You have Terraform >= 0.12

Thanks!

[mathieuchateau]

Yes I did that. All VM (DC & wks) are created, subnet, IP, even kibana VM. I am using Terraform 1.3.5

I tried 3 times.

[christophetd]

Thanks! Let me have a look.

[christophetd]

Seems to "work on my machine".

The error message from your side is:

Failed to parse /Users/mathieuchateau/Local/Adaz-main/ansible/inventory_azure_rm.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory

Can you try applying this patch and let me know if it works best?

diff --git a/ansible/inventory_azure_rm.yml b/ansible/inventory_azure_rm.yml
index 6569684..2203899 100644
--- a/ansible/inventory_azure_rm.yml
+++ b/ansible/inventory_azure_rm.yml
@@ -1,4 +1,4 @@
-plugin: azure_rm
+plugin: azure.azcollection.azure_rm
 auth_source: cli
 # Note: The Ansible dynamic inventory plugin does not allow this to be specified dynamically at runtime
 # (see https://github.com/ansible/ansible/issues/69949) for a feature proposal
@@ -10,4 +10,4 @@ conditional_groups:
   elasticsearch: "tags.kind == 'elasticsearch'"
 keyed_groups:
 - prefix: tag
-  key: tags
\ No newline at end of file
+  key: tags

[mathieuchateau]

I removed previous reply to confirm, but still have same issue. Replaced azure_rm by azure.azcollection.azure_rm

my ansible version:

ansible [core 2.12.10]
  config file = None
  configured module search path = ['/Users/mathieuchateau/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/mathieuchateau/Local/Adaz-main/ansible/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/mathieuchateau/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/mathieuchateau/Local/Adaz-main/ansible/venv/bin/ansible
  python version = 3.10.6 (main, Aug 30 2022, 04:58:14) [Clang 13.1.6 (clang-1316.0.21.2.5)]
  jinja version = 3.1.2
  libyaml = True

[christophetd]

I have almost the same (which makes sense given it's in a venv):

ansible [core 2.12.10]
  config file = None
  configured module search path = ['/Users/christophetd/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/christophetd/workspace/Adaz/ansible/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/christophetd/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/christophetd/workspace/Adaz/ansible/venv/bin/ansible
  python version = 3.10.8 (main, Oct 13 2022, 10:18:28) [Clang 13.0.0 (clang-1300.0.29.30)]
  jinja version = 3.1.2
  libyaml = True

After having run the Terraform, does any of these commands work?

# First go to ansible/
$ cd ansible

$ ansible-playbook elasticsearch-kibana.yml
$ ansible-playbook -i inventory_azure_rm.yml elasticsearch-kibana.yml

If you can try and post the output. Thanks!

[mathieuchateau]

@christophetd thanks for helping :)

here are the command results after applying terraform:

[mathieuchateau]

Also in fact DC does not have AD role installed. I can log in with password but just a bare windows

[mathieuchateau]

Another try : I cloned your project and started it without changing a single line..then it works. Will try to find what's wrong. I only changed to France Central and login name/password. Digging..

[mathieuchateau]

@christophetd : ok so I guess I found out the root cause. When changing ad-hunting-lab in vars.tf, then you must manually update inventory_azure_rm.yml with the new value also.

[christophetd]

Amazing, can you provide your sample config so I can fix the docs? Thanks!

[mathieuchateau]

I did a pull request to add comment to avoid this problem, thanks again for your help

[christophetd]

Thanks but the default version of the inventory file has no region in it, can you show an example with a specified region? I'll add it to the PR

[mathieuchateau]

@christophetd region is not any issue in fact. I thought it may be, but I could change to "France Central" in vars.tf without touching anything else about it, it works as expected

[christophetd]

So from what you say we should close https://github.com/christophetd/Adaz/pull/48 without merging?

[mathieuchateau]

Warning I added is needed: # Warning: if you change this, also change it in ansible/inventory_azure_rm.yml It's not linked to region (like France Central) but to resource group name which is hardcoded in ansible file too

Merge is needed to add this warning comment