command_parser relative path failing

[crosson]

ISSUE TYPE

• Bug Report

ANSIBLE VERSION

ansible --version
ansible 2.6.5
  config file = /Users/me/.ansible.cfg
  configured module search path = [u'/Users/me/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/me/Documents/git/local_ansible/venv/lib/python2.7/site-packages/ansible
  executable location = /Users/me/Documents/git/local_ansible/venv/bin/ansible
  python version = 2.7.10 (default, Oct  6 2017, 22:29:07) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

Network OS

• Mac OSX
• MBP 2017

SUMMARY

I have to populate my parser_templates folder, complete with all templates, in two locations for the playbook to succeed.

STEPS TO REPRODUCE

I have the following folder structure.

local_ansible
  -inventory
     temp
 - playbooks
    show_version.yaml

I call my playbook via

ansible-playbook -i inventory/temp playbooks/show_version.yaml -k

Under this folder struct the playbook works

local_ansible
  -inventory
     temp
  - parser_templates
     ios
     nxos
     junos
 - playbooks
     - parser_templates
        ios
        nxos
        junos
    show_version.yaml

Parser templates is in two locations

local_ansible/parser_templates local_ansible/playbooks/parser_templates

The playbook succeeds under this setup.

If I remove parser_templates from the root folder leaving it only defined in the playbooks folder the playbook fails.

Folder structure is now

local_ansible
  -inventory
     temp
 - playbooks
     - parser_templates
        ios
        nxos
        junos
    show_version.yaml

The following error is provided

TASK [Generate Version Fact as JSON] ************************************************************************************************
fatal: [myhost]: FAILED! => {"msg": "src [parser_templates/nxos/show_version.yaml] is either missing or invalid"}

Under a different setup the parser_templates folder is in the root directory. But I have removed it from the playbooks directory.

This is the structure

local_ansible
  -inventory
     temp
  - parser_templates
     ios
     nxos
     junos
 - playbooks
    show_version.yaml

I then get this error instead

TASK [Generate Version Fact as JSON] ************************************************************************************************
fatal: [myhost]: FAILED! => {"msg": "Unable to retrieve file contents\nCould not find or access '/Users/me/Documents/git/local_ansible/playbooks/parser_templates/nxos/show_version.yaml' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}

If I return the state of having the folder in both locations the playbook succeeds. Any ideas? Seems like the module is using the same relative path from two contexts thereby causing this problem.

Here is the actual playbook

- hosts: nxos
  connection: network_cli
  gather_facts: false

  tasks:
    - name: Collect Version Information
      ios_command:
        commands:
          - show version
      register: nxos_version_output

    - name: import the network-engine role
      import_role:
        name: ansible-network.network-engine

    - name: Generate Version Fact as JSON
      command_parser:
        file: "parser_templates/nxos/show_version.yaml"
        content: "{{ nxos_version_output.stdout.0 }}"

- hosts: junos
  connection: netconf
  gather_facts: false

  tasks:
    - name: Collect Version Information
      junos_command:
        commands:
          - show version
      register: junos_version_output

    - name: import the network-engine role
      import_role:
        name: ansible-network.network-engine

    - name: Generate Version Fact as JSON
      command_parser:
        file: "parser_templates/junos/show_version.yaml"
        content: "{{ junos_version_output.stdout.0 }}"

- hosts: all
  gather_facts: false

  tasks:
    - name: Display Facts
      debug:
        msg: "{{ [system_facts.version] }}"

[trishnaguha]

@crosson for this structure

local_ansible
  -inventory
     temp
  - parser_templates
     ios
     nxos
     junos
 - playbooks
    show_version.yaml

in this case the path to your parser templates should be like file: ../parser_templates/nxos/show_version.yaml as your playbook is in the other sub directory playbooks.

[crossonST]

TASK [Generate Version Fact as JSON] ************************************************************** fatal: [tuk8cs1.ilo.skytap.com]: FAILED! => {"msg": "src [../parser_templates/nxos/show_version.yaml] is either missing or invalid"}

Here is a screenshot of my folder hierarchy. It is as you've shown above.

[trishnaguha]

@crossonST this is my directory structure same as yours:

tguha@tguha ~/localtest $ tree
.
├── ansible.cfg
├── inventory
├── parser_templates
│   └── ios
│       └── show_interfaces.yaml
└── playbooks
    ├── parser_templates
    │   └── ios
    │       └── show_interfaces.yaml
    └── parse.yaml

My Playbook:

tguha@tguha ~/localtest $ cat playbooks/parse.yaml 
- hosts: ios
  connection: network_cli
  gather_facts: no

  tasks:
  - name: import network-engine role
    import_role:
      name: ~/workspace/network-engine

  - name: run command on ios and return result
    cli:
      command: show interfaces
    register: interfaces

  - name: set interface fact
    set_fact:
      interfaces: "{{ interfaces.stdout }}"

  - name: parse interfaces using command_parser engine
    command_parser:
      file: "parser_templates/ios/show_interfaces.yaml"
      content: "{{ interfaces }}"

When I run the playbook from the top dir, it works fine: ansible-playbook playbooks/parse.yaml When I run the playbook from the playbooks/ dir, it also works fine: ansible-playbook parse.yaml -i ../inventory

Which dir you are running the playbook from? Which version of Network Engine you are using?

[crossonST]

Hmm I am indeed running it in the top dir. Let me clone this over to a fresh environment. I am running in a virtual env but maybe there is another local factor. I am currently running it from my Mac.

[crossonST]

So I have reproduced this in a totally separate environment with different end devices. Can you try this with virtual env ansible 2.6?

Also if you notice your example above you provided the following tree structure.

tguha@tguha ~/localtest $ tree
.
├── ansible.cfg
├── inventory
├── parser_templates
│   └── ios
│       └── show_interfaces.yaml
└── playbooks
    ├── parser_templates
    │   └── ios
    │       └── show_interfaces.yaml
    └── parse.yaml

The idea here is that you had to place parser_templates folder in two locations. Try changing the file name of that folder and see if your test works?

tguha@tguha ~/localtest $ tree
.
├── ansible.cfg
├── inventory
├── parser_templates_x
│   └── ios
│       └── show_interfaces.yaml
└── playbooks
    ├── parser_templates
    │   └── ios
    │       └── show_interfaces.yaml
    └── parse.yaml

[trishnaguha]

@crossonST ACK. I will test this and get back to you. I couldn't test it last week as I was on PTO.

[crossonST]

Thank you! If it helps at all I was running in a virtual environment.

[mattgmoser]

Has there been a fix/update on this? Currently running into the same issue, Ansible back-end is RHEL7. Using most recent version of network-engine.

When calling the command_parser module by specifying "file," it fails to find the parser file whether relative or direct unless a file of the same name also exists in the root directory where the playbook is being run.

[trishnaguha]

@crossonST @mattgmoser No this is not fixed yet. I can make it work with providing absolute path to the parser_templates directory which is not present in playbook dir.

  - name: parse interfaces using command_parser engine
    command_parser:
      file: "~/localtest/parser_templates_x/ios/show_interfaces.yaml"
      content: "{{ interfaces }}"

[trishnaguha]

The fix https://github.com/ansible-network/network-engine/pull/222 is merged and will be released with 2.7.4 and 2.6.10 network engine this week.

[andreasbourges]

unfortunately this fix seems to break my setup ?! in my setup I had the following:

roles/vlan_facts/tasks/main.yml

---
- name: COLLECT SHOW COMMAND OUTPUT (NXOS)
  nxos_command:
    commands:
      - show run vlan
  register: output
  when: ansible_network_os == 'nxos'

- name: PARSE THE SHOW VLAN OUTPUT (NXOS)
  command_parser:
    file: "parsers/{{ ansible_network_os }}/show_vlan.yml"
    content: "{{ output.stdout[0] }}"
  when: ansible_network_os == 'nxos'

...and the corresponding parser:

parsers/nxos/show_vlan.yml

The playbook is at the top-level directory:

./otv_vlans.yml

Before upgrading to the latest network-engine everything worked fine, but now the parser is not found anymore:

fatal: [dcdelb1sd10100]: FAILED! => {
    "msg": "src [None] is either missing or invalid"
}

It seems this is caused by the method generate_source_path() method. Didn't dig too deep, since adding a '~/' to make the file-reference absolute did the trick.

[trishnaguha]

@andreasbourges Can you upgrade to the latest release 2.7.5? We have got a bugfix released related to path for command_parser with that version. If the bug still exists please open a new issue with the steps to reproduce. Thanks!

[andreasbourges]

Him Trishna,

…I am using 2.7.5 and problems started after the upgrade to 2.7.5 😉 Absolute Path is working fine, but relative path fails.

Thanks,

Andreas

Von: Trishna Guha notifications@github.com Gesendet: Montag, 18. März 2019 04:42 An: ansible-network/network-engine network-engine@noreply.github.com Cc: andreasbourges andy@bourges.de; Mention mention@noreply.github.com Betreff: Re: [ansible-network/network-engine] command_parser relative path failing (#152)

@andreasbourges https://github.com/andreasbourges Can you upgrade to the latest release 2.7.5? We have got a bugfix released related to path for command_parser with that version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ansible-network/network-engine/issues/152#issuecomment-473759470 , or mute the thread https://github.com/notifications/unsubscribe-auth/AgPMVoxs8MKz4qVETflZ3UyHGG3cAW4Mks5vXwsFgaJpZM4XJVIr .

[trishnaguha]

@andreasbourges thanks, so just to be sure it works with 2.7.4?

[andreasbourges]

Hi,

…2.7.4 does not work, but 2.7.3 works.

Thanks,

Andreas

Von: Trishna Guha notifications@github.com Gesendet: Montag, 18. März 2019 16:18 An: ansible-network/network-engine network-engine@noreply.github.com Cc: andreasbourges andy@bourges.de; Mention mention@noreply.github.com Betreff: Re: [ansible-network/network-engine] command_parser relative path failing (#152)

@andreasbourges https://github.com/andreasbourges Can you please confirm if it works with 2.7.4?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ansible-network/network-engine/issues/152#issuecomment-473954250 , or mute the thread https://github.com/notifications/unsubscribe-auth/AgPMVl6wTc682i2aaL7ne5AWP-4J0ivfks5vX64ggaJpZM4XJVIr .