zenchaos commented 3 years ago

I am attempting to update firmware on our edge switches and am unable to run the arubaoss_file_transfer. The playbook is being run on a Ubuntu 20.04 machine and all of the switches have been previously managed and updated with the older Aruba ansible modules.

I have tried both of these for the os with the same result.

ansible_network_os: arubanetworks.aos_switch.arubaoss
ansible_network_os: arubaoss

It looks to me like the file path isn't being handled correctly some how though I may not be reading the output correctly.

Here is a short playbook example.

hosts: all

serial: 1

collections:
- arubanetworks.aos_switch
tasks:
- name: copy image for 2930F arubanetworks.aos_switch.arubaoss_file_transfer: file_url: "http://10.100.1.100/WC_16_10_0012.swi" file_type: "FTT_FIRMWARE" boot_image: "BI_PRIMARY_IMAGE" action: "FTA_DOWNLOAD"

ansible-playbook firmware_update.yml -i hosts_edge_switch.yml -l SW100

PLAY [all] ***

TASK [Gathering Facts] *** ok: [SW100]

TASK [copy image for 2930F] ** fatal: [SW100]: FAILED! => {"msg": "error processing module_util ansible_collections.arubanetworks.arubaoss.plugins.module_utils.arubaoss.arubaoss_argument_spec loading redirected collection arubanetworks.arubaoss: unable to locate collection arubanetworks.arubaoss"}

PLAY RECAP *** SW100 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

pip list Package Version

ansible 2.10.7 ansible-base 2.10.5 cffi 1.14.4 cryptography 3.4.4
Jinja2 2.11.3 MarkupSafe 1.1.1
packaging 20.9
pip 20.0.2 pkg-resources 0.0.0
pycparser 2.20
pyparsing 2.4.7
PyYAML 5.4.1
setuptools 44.0.0

/home/ansible/Python/venv_2-10/lib/python3.8/site-packages/ansible_collections

Collection Version

amazon.aws 1.4.0
ansible.netcommon 1.5.0
ansible.posix 1.1.1
ansible.windows 1.4.0
arista.eos 1.3.0
awx.awx 14.1.0 azure.azcollection 1.4.0
check_point.mgmt 1.0.6
chocolatey.chocolatey 1.0.2
cisco.aci 1.1.1
cisco.asa 1.0.4
cisco.intersight 1.0.10 cisco.ios 1.3.0
cisco.iosxr 1.2.1
cisco.meraki 2.2.0
cisco.mso 1.1.0
cisco.nso 1.0.3
cisco.nxos 1.4.0
cisco.ucs 1.6.0
cloudscale_ch.cloud 1.3.1
community.aws 1.3.0
community.azure 1.0.0
community.crypto 1.4.0
community.digitalocean 1.0.0
community.docker 1.2.2
community.fortios 1.0.0
community.general 1.3.6
community.google 1.0.0
community.grafana 1.1.0
community.hashi_vault 1.1.0
community.hrobot 1.1.0
community.kubernetes 1.1.1
community.kubevirt 1.0.0
community.libvirt 1.0.0
community.mongodb 1.2.0
community.mysql 1.2.0
community.network 1.3.2
community.okd 1.0.0
community.postgresql 1.1.1
community.proxysql 1.0.0
community.rabbitmq 1.0.1
community.routeros 1.1.0
community.skydive 1.0.0
community.vmware 1.7.0
community.windows 1.3.0
community.zabbix 1.2.0
containers.podman 1.4.1
cyberark.conjur 1.1.0
cyberark.pas 1.0.5
dellemc.os10 1.0.2
dellemc.os6 1.0.6
dellemc.os9 1.0.3
f5networks.f5_modules 1.7.1
fortinet.fortimanager 1.0.5
fortinet.fortios 1.1.8
frr.frr 1.0.3
gluster.gluster 1.0.1
google.cloud 1.0.2
hetzner.hcloud 1.2.1
ibm.qradar 1.0.3
infinidat.infinibox 1.2.4
junipernetworks.junos 1.3.0
mellanox.onyx 1.0.0
netapp.aws 20.9.0 netapp.elementsw 20.11.0 netapp.ontap 20.12.0 netapp_eseries.santricity 1.1.0
netbox.netbox 1.2.1
ngine_io.cloudstack 1.2.0
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.0
openstack.cloud 1.2.1
openvswitch.openvswitch 1.1.0
ovirt.ovirt 1.3.0
purestorage.flasharray 1.6.2
purestorage.flashblade 1.4.0
servicenow.servicenow 1.0.4
splunk.es 1.0.2
theforeman.foreman 1.5.1
vyos.vyos 1.1.1
wti.remote 1.0.1

/home/ansible/.ansible/collections/ansible_collections

Collection Version

ansible.netcommon 1.2.1
arubanetworks.aos_switch 1.2.0
dellemc.os10 1.0.1

ansible-playbook --version ansible-playbook 2.10.5 config file = /etc/ansible/ansible.cfg configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /home/ansible/Python/venv_2-10/lib/python3.8/site-packages/ansible executable location = /home/ansible/Python/venv_2-10/bin/ansible-playbook python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]

tchiapuziowong commented 3 years ago

Hi @zenchaos ! Can you please verify if you're properly specifying your environment variables to include the AOS-Switch network modules? https://github.com/aruba/aos-switch-ansible-collection#setting-environment-variables

zenchaos commented 3 years ago

Yes, I have tried with and without this in the ansible.conf.

[defaults]

NETWORK_GROUP_MODULES=arubaoss

ANSIBLE_NETWORK_GROUP_MODULES=arubaoss ansible-playbook.... doesn't work either.

zenchaos commented 3 years ago

I really need to start moving to these modules so I started poking at this again. Since it looked like the path was wrong, I tried 2 things with same results.

cp -R ~/.ansible/collections/ansible_collections/arubanetworks/aos_switch ~/.ansible/collections/ansible_collections/arubanetworks/arubaoss

ln -s ~/.ansible/collections/ansible_collections/arubanetworks/aos_switch ~/.ansible/collections/ansible_collections/arubanetworks/arubaoss

ansible-playbook firmware_update.yml -i hosts_edge_switch.yml -l SW100

PLAY [all] *****

TASK [Gathering Facts] ***** ok: [SW100]

TASK [copy image for 2930F] **** fatal: [SW100]: FAILED! => {"changed": false, "msg": "Request failed: <urlopen error [Errno -2] Name or service not known>", "status": -1, "url": "http://None:80/rest/v6.0/login-sessions"}

PLAY RECAP ***** SW100 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

I actually removed this part since it was causing other modules to fail.

[defaults]

NETWORK_GROUP_MODULES=arubaoss

The module isn't correctly working this way but it looks like there is a definite problem with the path being used somewhere. The last time I saw this error, I had to add the interpreter path. The path is set and the interpreter version is correct. Obviously, the path is messed up some where. It doesn't make sense to poke at it much more to try making it work as is.

What's the best way to proceed with this?

tchiapuziowong commented 3 years ago

Hi @zenchaos I believe I know the issue and will be looking into it today, thank you for your patience and diligence on this issue

tchiapuziowong commented 3 years ago

Hi @zenchaos I believe I fixed the issue, could you please reinstall the collection and verify if the issue is still existing? ansible-galaxy collection install arubanetworks.aos_switch -f

zenchaos commented 3 years ago

Well, it ran but it's not using the ansible_host address. Other playbooks run so the host file is in tact.

ansible-playbook firmware_update.yml -i hosts_edge_switch.yml -l SW100

PLAY [all] *****

TASK [Gathering Facts] ***** ok: [SW100]

TASK [copy image for 2930F] **** fatal: [SW100]: FAILED! => {"changed": false, "msg": "Request failed: <urlopen error [Errno -2] Name or service not known>", "status": -1, "url": "http://None:80/rest/v6.0/login-sessions"}

PLAY RECAP ***** SW100 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

tchiapuziowong commented 3 years ago

When you run it you'll need to ensure you're specifying the ARUBAOSS as a part of the Network Group Modules either through the command line or through your ansible.cfg file, see here: https://github.com/aruba/aos-switch-ansible-collection#setting-environment-variables

zenchaos commented 3 years ago

OK, specifying the network group module gives me this error.

TASK [copy image for 2930F] **** fatal: [SW100]: FAILED! => {"changed": false, "msg": "Connection failure: Remote end closed connection without response", "status": -1, "url": "http://10.1.10.100:80/rest/v6.0/login-sessions"}

The switch is running WC.16.10.0010.

If I curl the uirl, I get back a response.

curl --noproxy 10.1.10.100 -X POST http://10.1.10.100/rest/v6.0/login-sessions -d '{"userName":"admin", "password":"password"}' -k {"payloadSize":65536,"uri":"/login-sessions","cookie":"sessionId=T69QVPHuZABCsQEefnImHvs9xRTRuHvK4GjFbXsuJRB2WMgQzmtXsE2qZWmMPiX"}

tchiapuziowong commented 3 years ago

I believe this might be related to the known issue with the 16.10.0010 firmware version, could you upgrade to 16.10.0011 and verify if you still see the same issue?

zenchaos commented 3 years ago

I get the same error with WC.16.10.0005 or WC.16.10.0011.

When I try to use the arubanetworks.aos_switch.arubaoss_command module I get an error with "AssertionError: socket_path must be a value", I don't know if this is related. The error occurs when ansible_connection: local is set. If ansible_connection: network_cli is set, the module runs. Do you want, I'll open a ticket for this separately.

Unfortunately, I couldn't use the arubanetworks.aos_switch.arubaoss_command module if I wanted to because you have to answer yes after some commands and there's no way to do that.

We have several hundred devices that need to be updated and I would rather not have to do it all manually.

tchiapuziowong commented 3 years ago

Okay so if I'm understanding correctly you get the following error with EVERY rest api module or is it just the arubaoss_file_transfer? fatal: [SW100]: FAILED! => {"changed": false, "msg": "Connection failure: Remote end closed connection without response", "status": -1, "url": "http://10.1.10.100:80/rest/v6.0/login-sessions"}

You get the same error above when using the WC.16.10.0011 firmware version as well?

Are you able to run any of the REST API modules? Like arubaoss_vlan?

zenchaos commented 3 years ago

I'm getting this error with the arubaoss_file_transfer module. fatal: [SW100]: FAILED! => {"changed": false, "msg": "Connection failure: Remote end closed connection without response", "status": -1, "url": "http://10.1.10.100:80/rest/v6.0/login-sessions"}

Yes, it occurs with the WC.16.10.0011 firmware.

I did a remote mirror on the switch and looked at the conversation with wireshark. The json auth request post packet is immediately followed by a ACK which is followed by a FIN. The auth token isn't returned.

I'll write up something innocuous like setting the syslog server or something then let you know.

zenchaos commented 3 years ago

The same failure occurs with the arubaoss_syslog module.

NETWORK_GROUP_MODULES='arubaoss','aruba',‘dellos10’,'os10' ansible-playbook syslog.yml -i hosts_edge_switch.yml -l SW100

fatal: [SW100]: FAILED! => {"changed": false, "msg": "Connection failure: Remote end closed connection without response", "status": -1, "url": "http://10.1.10.100:80/rest/v6.0/login-sessions"}

hosts: all collections:
- arubanetworks.aos_switch
tasks:
- name: Configure syslog server arubanetworks.aos_switch.arubaoss_syslog: server_address: 10.1.10.10 protocol: TP_UDP

alagoutte commented 3 years ago

I'm getting this error with the arubaoss_file_transfer module. fatal: [SW100]: FAILED! => {"changed": false, "msg": "Connection failure: Remote end closed connection without response", "status": -1, "url": "http://10.1.10.100:80/rest/v6.0/login-sessions"}

Yes, it occurs with the WC.16.10.0011 firmware.

I did a remote mirror on the switch and looked at the conversation with wireshark. The json auth request post packet is immediately followed by a ACK which is followed by a FIN. The auth token isn't returned.

I'll write up something innocuous like setting the syslog server or something then let you know.

do you have try 16.10.0012 ? there is a bug with REST API...

zenchaos commented 3 years ago

OK, so 16.10.0012 works. How can I get the code updated on the switches? This module won't run over ssh. The arubaoss_command won't answer yes/no.

The 2920 switches only have code up to 16.10.0011. Is there a workaround to work on these switches?

Should I put in a feature request for the arubaoss_command so that it will answer yes/no. If it could do that, we could copy the code from a tftp server, change the boot inage and reboot the switch.

tchiapuziowong commented 3 years ago

Hi @zenchaos - we're looking into adding prompt response functionality to the SSH modules, I appreciate your patience while we look into developing this. For the case of the 2920s would you be able to open a TAC case?

jmagudo commented 2 years ago

Hi, ¿any news about this problem? We are in the same situation with not working rest modules and can't use the command modules to upgrade switch to fixed version because the lack of prompt / answer support. We have 150 switches to upgrade, ¿there any other alternatives?.

zenchaos commented 2 years ago

I have several hundred that I update regularly so I ended up writing an expect script inside of an Ansible playbook to copy the firmware from a tftp server. After the firmware was uploaded, I set the boot image and reloaded the switch. With the expect script, I could easily handle the yes/no prompt.

We ran into trouble with the bugs in the json api so the modules were useless. Using expect, Ansible can ssh to the switch and copy the code. Ugly but reliable. If you need help with the playbook, I can help you out there, I just don't have access to it right now.

Chris

jmagudo commented 2 years ago

Thanks for the offer Chris. I solved it with the following task code and some conditional checks:

name: aruba upgrade firmware ansible.netcommon.cli_command: command: copy tftp flash {{tftp_server}} {{software_image}} primary prompt: Continue (y/n)? answer: y newline: false

Best regards,

Jose

tchiapuziowong commented 2 years ago

We have support using the "prompt" argument through our SSH modules, please use the following:

- name: "Upload New Image to AOS-Switch"
  hosts: all
  collections:
    - arubanetworks.aos_switch
  tasks:
    - name: Upload image to Primary via TFTP
      arubaoss_command:
        commands:
          - command: "copy tftp flash 10.0.9.87 WC_16_10_0017.swi primary oo
bm"
            prompt:
              - ".*\(y\/n\)\?.*"
            answer:
                - y

aruba / aos-switch-ansible-collection

arubaoss_file_transfer not running #6

serial: 1

/home/ansible/Python/venv_2-10/lib/python3.8/site-packages/ansible_collections

/home/ansible/.ansible/collections/ansible_collections