search

ansible-middleware / amq

A collection to manage AMQ brokers
Apache License 2.0
16 stars 12 forks source link

Template error when defining a broker_connection #126

Closed herbertkb closed 4 months ago

herbertkb commented 4 months ago
SUMMARY

I try to define a simple bridge between brokerA and brokerB, but get a templating error

ISSUE TYPE
ANSIBLE VERSION
ansible [core 2.16.6]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/kherbert/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.12/site-packages/ansible
  ansible collection location = /home/kherbert/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] (/usr/bin/python3)
  jinja version = 3.1.3
  libyaml = True
COLLECTION VERSION
2.0.1
STEPS TO REPRODUCE

On a fresh CentOS9 VM, where admin is a member of wheel group, and bridge is a host group with just that VM's IP address. 

ansible-playbook -i inventory test-vm-connection.yml --ask-pass --ask-become-pass -vvvv
---
- name: Playbook for ActiveMQ Artemis
  remote_user: admin
  hosts: bridge
  vars:
    activemq_ports_offset_enabled: true

  tasks:
    - name: Install brokerA
      include_role:
        name: activemq
      vars:
        activemq_instance_name: brokerA
        activemq_ports_offset: 0
        activemq_port: 61616
        activemq_jmx_exporter_port: 18080
        activemq_http_port: 8161
        activemq_addresses:
          - name: a.test
            anycast:
              - name: a.test
          - name: b.test
            anycast:
              - name: b.test
        activemq_broker_connections:
          - uri: vm://0
            name: brokerA-2-brokerB
            operations:
              - type: sender
                parameters:
                  address-match: 'b.#'
              - type: receiver
                parameters:
                  address-match: 'a.#'

    - name: Install brokerB
      include_role:
        name: activemq
      vars:
        activemq_instance_name: brokerB
        activemq_ports_offset: 1
        activemq_port: 61617
        activemq_jmx_exporter_port: 18081
        activemq_http_port: 8162
        activemq_addresses:
          - name: a.test
            anycast:
              - name: a.test
          - name: b.test
            anycast:
              - name: b.test
EXPECTED RESULTS

A brokered-connection bridge configured between brokerA and brokerB.

ACTUAL RESULTS
TASK [activemq : Create amqp broker connections configuration string] **************************************************************************************************************************************
task path: /home/kherbert/work/ansible-middleware/amq/roles/activemq/tasks/validate_config.yml:118
File lookup using /home/kherbert/work/ansible-middleware/amq/roles/activemq/templates/broker_connections.broker.xml.j2 as file
exception during Jinja2 execution: Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 975, in do_template
    t = myenv.from_string(data)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 1105, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
    self.handle_exception(source=source_hint)
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<unknown>", line 2, in template
jinja2.exceptions.TemplateSyntaxError: expected token 'end of statement block', got '='

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 873, in _lookup
    ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/plugins/lookup/template.py", line 160, in run
    res = templar.template(template_data, preserve_trailing_newlines=True,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 764, in template
    result = self.do_template(
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 977, in do_template
    raise AnsibleError("template error while templating string: %s. String: %s" % (to_native(e), to_native(data)), orig_exc=e)
ansible.errors.AnsibleError: template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri="{{ broker_connection.uri }}" name="{{ broker_connection.name }}">
{% for operation in broker_connection.operation, wantlist=True) %}
            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}="{{ param.value }}" {% endfor %}/>
{% endfor %}
        </amqp-connection>. expected token 'end of statement block', got '='
fatal: [192.168.122.141]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'template'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri=\"{{ broker_connection.uri }}\" name=\"{{ broker_connection.name }}\">\n{% for operation in broker_connection.operation, wantlist=True) %}\n            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}=\"{{ param.value }}\" {% endfor %}/>\n{% endfor %}\n        </amqp-connection>. expected token 'end of statement block', got '='. template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri=\"{{ broker_connection.uri }}\" name=\"{{ broker_connection.name }}\">\n{% for operation in broker_connection.operation, wantlist=True) %}\n            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}=\"{{ param.value }}\" {% endfor %}/>\n{% endfor %}\n        </amqp-connection>. expected token 'end of statement block', got '='"
}
guidograzioli commented 4 months ago

Hello; the key for parameters under operations (and in general under any configuration that has parameters) needs to be a valid python variable name, ie. the java address-match needs to be written as address_match

        activemq_broker_connections:
          - uri: vm://0
            name: brokerA-2-brokerB
            operations:
              - type: sender
                parameters:
                  address-match: 'b.#'

I noticed the error is present in the role documentation, so relabelling as "documentation" (unless you find other issues with this configuration)

herbertkb commented 4 months ago

I changed the keys to address_match but still see the same (or very similar) error:

TASK [activemq : Create amqp broker connections configuration string] **************************************************************************************************************************************
task path: /home/kherbert/work/ansible-middleware/amq/roles/activemq/tasks/validate_config.yml:118
File lookup using /home/kherbert/work/ansible-middleware/amq/roles/activemq/templates/broker_connections.broker.xml.j2 as file
exception during Jinja2 execution: Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 975, in do_template
    t = myenv.from_string(data)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 1105, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
    self.handle_exception(source=source_hint)
  File "/usr/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<unknown>", line 2, in template
jinja2.exceptions.TemplateSyntaxError: expected token 'end of statement block', got '='

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 873, in _lookup
    ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/plugins/lookup/template.py", line 160, in run
    res = templar.template(template_data, preserve_trailing_newlines=True,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 764, in template
    result = self.do_template(
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/ansible/template/__init__.py", line 977, in do_template
    raise AnsibleError("template error while templating string: %s. String: %s" % (to_native(e), to_native(data)), orig_exc=e)
ansible.errors.AnsibleError: template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri="{{ broker_connection.uri }}" name="{{ broker_connection.name }}">
{% for operation in broker_connection.operation, wantlist=True) %}
            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}="{{ param.value }}" {% endfor %}/>
{% endfor %}
        </amqp-connection>. expected token 'end of statement block', got '='
fatal: [192.168.122.141]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'template'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri=\"{{ broker_connection.uri }}\" name=\"{{ broker_connection.name }}\">\n{% for operation in broker_connection.operation, wantlist=True) %}\n            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}=\"{{ param.value }}\" {% endfor %}/>\n{% endfor %}\n        </amqp-connection>. expected token 'end of statement block', got '='. template error while templating string: expected token 'end of statement block', got '='. String:         <amqp-connection uri=\"{{ broker_connection.uri }}\" name=\"{{ broker_connection.name }}\">\n{% for operation in broker_connection.operation, wantlist=True) %}\n            <{{ operation.type }} {% for param in lookup('ansible.builtin.dict', broker_connection.parameters, wantlist=True) %}{{ param.key | replace('_','-') }}=\"{{ param.value }}\" {% endfor %}/>\n{% endfor %}\n        </amqp-connection>. expected token 'end of statement block', got '='"
}
guidograzioli commented 4 months ago

reproduced

guidograzioli commented 4 months ago

Hello I have a PR above addressing this problem; can you test it? (or I can merge it and eventually forward fix)

herbertkb commented 4 months ago

I pulled and ran the PR branch. It fixed the template error, but now I get this error, it seems for not providing parameters user and password when defining the connection:

TASK [activemq : Create amqp broker connections configuration string] *******************************************************************************************************
task path: /home/kherbert/work/ansible-middleware/amq/roles/activemq/tasks/validate_config.yml:118
File lookup using /home/kherbert/work/ansible-middleware/amq/roles/activemq/templates/broker_connections.broker.xml.j2 as file
fatal: [192.168.122.141]: FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'user'. 'dict object' has no attribute 'user'. 'dict object' has no attribute 'user'. 'dict object' has no attribute 'user'\n\nThe error appears to be in '/home/kherbert/work/ansible-middleware/amq/roles/activemq/tasks/validate_config.yml': line 118, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create amqp broker connections configuration string\n  ^ here\n"
}

But from the documentation for that element, I see no mention of providing these credentials. https://access.redhat.com/documentation/en-us/red_hat_amq_broker/7.12/html-single/configuring_amq_broker/index#ref-br-configuring-broker-connections_configuring

I see there's a jinja if-block in roles/activemq/templates/broker_connections.broker.xml.j2, but I don't see a syntax reason for it to be failing. (But I'm still far from a jinja ninja.)

guidograzioli commented 4 months ago

Unless you setup anonymous access in artemis broker.xml, I think that for sender/receiver user and pass will always be needed (the molecule test was failing on authentication in broker connections); but yeah of course it's not marked required in the xls schema, so i'll add a check for it

guidograzioli commented 4 months ago

Should be fixed, can you please test again?

But I'm still far from a jinja ninja

That, nobody is. The bashfu mastering good'le times are gone :cry:

herbertkb commented 4 months ago

Looks good on my end. You're right, I need more configuration for it to lack user/password. I can verify that the broker connection exists.

Thank you for the fast turn around on this issue :)