search

ncclient / ncclient

Python library for NETCONF clients
Apache License 2.0
679 stars 398 forks source link

Can't send commands to IOS-XR (Cisco ASR 9000) #59

Closed jejenone closed 8 years ago

jejenone commented 9 years ago

Using this simple script, connection to a Cisco IOS-XR is hung up abruptly. I am able to retrieve the capabilities, but the <get-config> does not work

from ncclient import manager
import logging
import sys

rootLogger = logging.getLogger('ncclient.transport.session')
rootLogger.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
rootLogger.addHandler(handler)

host='<hidden host>'
user='netconf'
password='<hidden password>'
m = manager.connect(host=host, port=830, username=user, password=password,
                     hostkey_verify=False, device_params={'name':'default'},
                     look_for_keys=False, allow_agent=False)
c = m.get_config(source='running')
(.venv)jfleury@36netops2:~/salt-net$ python  test_cisco.py
<SSHSession(session, initial daemon)> created: client_capabilities=['urn:ietf:params:netconf:capability:writable-running:1.0', 'urn:ietf:params:netconf:capability:rollback-on-error:1.0', 'urn:ietf:params:netconf:capability:validate:1.0', 'urn:ietf:params:netconf:capability:confirmed-commit:1.0', 'urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp', 'urn:ietf:params:netconf:base:1.0', 'urn:liberouter:params:netconf:capability:power-control:1.0', 'urn:ietf:params:netconf:capability:candidate:1.0', 'urn:ietf:params:netconf:capability:xpath:1.0', 'urn:ietf:params:netconf:capability:startup:1.0', 'urn:ietf:params:netconf:capability:interleave:1.0']
installing listener <ncclient.transport.session.HelloHandler object at 0x2bb4890>
queueing <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
starting main loop
dispatching message to <ncclient.transport.session.HelloHandler object at 0x2bb4890>: <hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <capabilities>
  <capability>urn:ietf:params:netconf:base:1.1</capability>
  <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</capability>
  <capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
  <capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
  <capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-cdp-cfg?module=Cisco-IOS-XR-cdp-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-cdp-oper?module=Cisco-IOS-XR-cdp-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-crypto-sam-cfg?module=Cisco-IOS-XR-crypto-sam-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-crypto-sam-oper?module=Cisco-IOS-XR-crypto-sam-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ha-eem-cfg?module=Cisco-IOS-XR-ha-eem-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ha-eem-oper?module=Cisco-IOS-XR-ha-eem-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ifmgr-cfg?module=Cisco-IOS-XR-ifmgr-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ifmgr-oper?module=Cisco-IOS-XR-ifmgr-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-infra-infra-cfg?module=Cisco-IOS-XR-infra-infra-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ip-domain-cfg?module=Cisco-IOS-XR-ip-domain-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ip-domain-oper?module=Cisco-IOS-XR-ip-domain-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ip-iarm-datatypes?module=Cisco-IOS-XR-ip-iarm-datatypes&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-io-cfg?module=Cisco-IOS-XR-ipv4-io-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-io-oper?module=Cisco-IOS-XR-ipv4-io-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-cfg?module=Cisco-IOS-XR-ipv4-ma-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-oper?module=Cisco-IOS-XR-ipv4-ma-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-subscriber-cfg?module=Cisco-IOS-XR-ipv4-ma-subscriber-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-cfg?module=Cisco-IOS-XR-ipv6-ma-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-oper?module=Cisco-IOS-XR-ipv6-ma-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-subscriber-cfg?module=Cisco-IOS-XR-ipv6-ma-subscriber-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-lib-keychain-cfg?module=Cisco-IOS-XR-lib-keychain-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-lib-keychain-oper?module=Cisco-IOS-XR-lib-keychain-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-man-netconf-cfg?module=Cisco-IOS-XR-man-netconf-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-man-xml-ttyagent-cfg?module=Cisco-IOS-XR-man-xml-ttyagent-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-man-xml-ttyagent-oper?module=Cisco-IOS-XR-man-xml-ttyagent-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-parser-cfg?module=Cisco-IOS-XR-parser-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-qos-ma-cfg?module=Cisco-IOS-XR-qos-ma-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-qos-ma-oper?module=Cisco-IOS-XR-qos-ma-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-rgmgr-cfg?module=Cisco-IOS-XR-rgmgr-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-rgmgr-oper?module=Cisco-IOS-XR-rgmgr-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-shellutil-cfg?module=Cisco-IOS-XR-shellutil-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-shellutil-oper?module=Cisco-IOS-XR-shellutil-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-subscriber-infra-tmplmgr-cfg?module=Cisco-IOS-XR-subscriber-infra-tmplmgr-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-cfg?module=Cisco-IOS-XR-tty-management-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-datatypes?module=Cisco-IOS-XR-tty-management-datatypes&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-oper?module=Cisco-IOS-XR-tty-management-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-server-cfg?module=Cisco-IOS-XR-tty-server-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-server-oper?module=Cisco-IOS-XR-tty-server-oper&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/Cisco-IOS-XR-tty-vty-cfg?module=Cisco-IOS-XR-tty-vty-cfg&amp;revision=2013-07-22</capability>
  <capability>http://cisco.com/ns/yang/cisco-xr-types?module=Cisco-IOS-XR-types&amp;revision=2013-07-22</capability>
  <capability>urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&amp;revision=2013-07-15</capability>
  <capability>urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&amp;revision=2013-07-15</capability>
 </capabilities>
 <session-id>10918</session-id>
</hello>
discarding listener <ncclient.transport.session.HelloHandler object at 0x2bb4890>
dispatching error to <ncclient.transport.session.HelloHandler object at 0x2bb4890>
initialized: session-id=10918 | server_capabilities=['http://cisco.com/ns/yang/Cisco-IOS-XR-crypto-sam-oper?module=Cisco-IOS-XR-crypto-sam-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-cdp-oper?module=Cisco-IOS-XR-cdp-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-qos-ma-cfg?module=Cisco-IOS-XR-qos-ma-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-infra-infra-cfg?module=Cisco-IOS-XR-infra-infra-cfg&revision=2013-07-22', 'urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-server-cfg?module=Cisco-IOS-XR-tty-server-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-man-xml-ttyagent-oper?module=Cisco-IOS-XR-man-xml-ttyagent-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ip-iarm-datatypes?module=Cisco-IOS-XR-ip-iarm-datatypes&revision=2013-07-22', 'urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&revision=2013-07-15', 'http://cisco.com/ns/yang/Cisco-IOS-XR-shellutil-cfg?module=Cisco-IOS-XR-shellutil-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-io-cfg?module=Cisco-IOS-XR-ipv4-io-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-cfg?module=Cisco-IOS-XR-ipv4-ma-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-server-oper?module=Cisco-IOS-XR-tty-server-oper&revision=2013-07-22', 'urn:ietf:params:netconf:capability:rollback-on-error:1.0', 'http://cisco.com/ns/yang/Cisco-IOS-XR-man-netconf-cfg?module=Cisco-IOS-XR-man-netconf-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-cfg?module=Cisco-IOS-XR-tty-management-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ifmgr-oper?module=Cisco-IOS-XR-ifmgr-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-subscriber-infra-tmplmgr-cfg?module=Cisco-IOS-XR-subscriber-infra-tmplmgr-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-rgmgr-oper?module=Cisco-IOS-XR-rgmgr-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-io-oper?module=Cisco-IOS-XR-ipv4-io-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ifmgr-cfg?module=Cisco-IOS-XR-ifmgr-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-cfg?module=Cisco-IOS-XR-ipv6-ma-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-lib-keychain-cfg?module=Cisco-IOS-XR-lib-keychain-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ip-domain-oper?module=Cisco-IOS-XR-ip-domain-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-vty-cfg?module=Cisco-IOS-XR-tty-vty-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-subscriber-cfg?module=Cisco-IOS-XR-ipv4-ma-subscriber-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-cdp-cfg?module=Cisco-IOS-XR-cdp-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-subscriber-cfg?module=Cisco-IOS-XR-ipv6-ma-subscriber-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ha-eem-cfg?module=Cisco-IOS-XR-ha-eem-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ha-eem-oper?module=Cisco-IOS-XR-ha-eem-oper&revision=2013-07-22', 'urn:ietf:params:netconf:capability:candidate:1.0', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-datatypes?module=Cisco-IOS-XR-tty-management-datatypes&revision=2013-07-22', 'http://cisco.com/ns/yang/cisco-xr-types?module=Cisco-IOS-XR-types&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-qos-ma-oper?module=Cisco-IOS-XR-qos-ma-oper&revision=2013-07-22', 'urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&revision=2013-07-15', 'http://cisco.com/ns/yang/Cisco-IOS-XR-lib-keychain-oper?module=Cisco-IOS-XR-lib-keychain-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-ma-oper?module=Cisco-IOS-XR-ipv4-ma-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-man-xml-ttyagent-cfg?module=Cisco-IOS-XR-man-xml-ttyagent-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ip-domain-cfg?module=Cisco-IOS-XR-ip-domain-cfg&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-tty-management-oper?module=Cisco-IOS-XR-tty-management-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-rgmgr-cfg?module=Cisco-IOS-XR-rgmgr-cfg&revision=2013-07-22', 'urn:ietf:params:netconf:capability:validate:1.1', 'http://cisco.com/ns/yang/Cisco-IOS-XR-crypto-sam-cfg?module=Cisco-IOS-XR-crypto-sam-cfg&revision=2013-07-22', 'urn:ietf:params:netconf:base:1.1', 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv6-ma-oper?module=Cisco-IOS-XR-ipv6-ma-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-shellutil-oper?module=Cisco-IOS-XR-shellutil-oper&revision=2013-07-22', 'http://cisco.com/ns/yang/Cisco-IOS-XR-parser-cfg?module=Cisco-IOS-XR-parser-cfg&revision=2013-07-22']
installing listener <ncclient.operations.rpc.RPCReplyListener object at 0x2baae50>
Traceback (most recent call last):
  File "test_cisco.py", line 16, in <module>
    c = m.get_config(source='running')
  File "/state/home/jfleury/salt-net/.venv/lib/python2.7/site-packages/ncclient/manager.py", line 156, in wrapper
    return self.execute(op_cls, *args, **kwds)
  File "/state/home/jfleury/salt-net/.venv/lib/python2.7/site-packages/ncclient/manager.py", line 226, in execute
    raise_mode=self._raise_mode).request(*args, **kwds)
  File "/state/home/jfleury/salt-net/.venv/lib/python2.7/site-packages/ncclient/operations/retrieve.py", line 87, in request
    return self._request(node)
  File "/state/home/jfleury/salt-net/.venv/lib/python2.7/site-packages/ncclient/operations/rpc.py", line 291, in _request
    self._session.send(req)
  File "/state/home/jfleury/salt-net/.venv/lib/python2.7/site-packages/ncclient/transport/session.py", line 147, in send
    raise TransportError('Not connected to NETCONF server')
ncclient.transport.errors.TransportError: Not connected to NETCONF server

Can't figure out why the connection is dropped. Any idea how to debug that ?

jejenone commented 9 years ago

after a bit of investigation, this command sent to an IOS-XR does not work:

<?xml version="1.0" encoding="UTF-8"?>
<nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8ff0220-e3bb-11e4-b950-047d7bae30d0">
<nc:get-config>
<nc:source>
<nc:running/>
</nc:source>
<nc:filter><nc:Configuration/></nc:filter>
</nc:get-config>
</nc:rpc>
]]>]]>

whereas this one, sent directly to the IOS-XR netconf session, works:

<?xml version="1.0" encoding="UTF-8"?>
<rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f8ff0220-e3bb-11e4-b950-047d7bae30d0">
<get-config>
<source>
<running/>
</source>
<filter><Configuration/></filter>
</get-config>
</rpc>
]]>]]>

removing the qualified prefix nc: fom the tags made it work. Now I'm trying to understand ncclient code to see how I could adapt it to remove the prefix with IOS-XR. Any help would be appreciated.

leopoul commented 9 years ago

Hi Jerome, The version of IOS XR you query, seems to support NETCONF 1.1. Currently ncclient supports only the 1.0 version but I'm working on a new version with 1.1 support. I have a preliminary branch that works to an extent with IOS XR and NETCONF 1.1 but I would suggest waiting for a new stable release withing the next weeks. Long story short, in 1.1 there have been some major changes especially when it comes to message delimiting and this requires quite some changes to message delimiting and parsing.

Thnx in advance, Best, Leo

jejenone commented 9 years ago

Is there any chance you can push this branch on github ? I don't really care about it not being stable, since everything I'm working on right now is experimental anyway and being multi-vendor is essential to that project. Also, I'd be more than happy to contribute to IOS-XR support starting from that branch. Appreciate your input on that specific problem.

jejenone commented 9 years ago

for reference: http://www.ciscoday.com/pdf2015/Netconf%20Yang%20Josef%20Ungerman.pdf

jejenone commented 9 years ago

definitely a capability issue with Netconf 1.1. Here is the output from the debug netconf-yang all on the ASR9K:

RP/0/RSP0/CPU0:edge01.dus01#term mon
Fri Apr 17 17:49:57.444 UTC
RP/0/RSP0/CPU0:edge01.dus01#RP/0/RSP0/CPU0:Apr 17 17:50:17.338 UTC: netconf[1116]: nc_pxs_ipc_notify_callback_fn:294 IPC_NOTIFY_OPEN
RP/0/RSP0/CPU0:Apr 17 17:50:17.338 UTC: netconf[1116]: nc_sm_pxs_notify_callback_fn:5645 New NETCONF SSH proxy client connection: 101ff164
RP/0/RSP0/CPU0:Apr 17 17:50:17.339 UTC: netconf[1116]: DBG: yfw_session_store.c:130:yfw_session_create ctx=1000a91c,assigned session-id=5615
RP/0/RSP0/CPU0:Apr 17 17:50:17.339 UTC: netconf[1116]: DBG: me.c:11929:me_session_create ctx=1000b9b8,session create: usr=17edc138,ses=100cb0a8
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: DBG: me_bk_sysdb_authorization.c:299:me_bk_sysdb_auth_user_init ctx=1000bf20,AAA authorization not configured, authorization of operations turned off.
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: TRC: me_backend_sysdb.c:1213:sysdb_backend_session_create ctx=1000bf20,auth user init succeess, caller ctx=17edc818
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: DBG: me_backend_sysdb.c:1217:sysdb_backend_session_create ctx=1000bf20,SysDB backend session successfully created (17edc1c4).
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: DBG: yfw_error.c:306:yfw_me_request_result_check ctx=1000a91c,ses=100cb0a0,op=11,ME request success.
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: TRC: me_bk_sysdb_authorization.c:180:me_bk_sysdb_auth_request_process ctx=1000bf20,Authorization request for user 'netconf' completed (rc=0), calling ME callback (user ctx=17edc1c4)
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: TRC: me.c:3595:me_session_queue_handler ctx=1000b9b8,session create queue handler woke up..
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: TRC: yfw_req_proc.c:135:yfw_session_start_cb ctx=1000a91c,YFW session start callback called (rc=0,session-id=5615), calling netconf callback.
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: nc_sm_session_start_yfw_response_cb:5233 start-session response received: 1000a91c 101a6024
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: nc_sm_session_find_session_id:1349 Found session 5615 1025edc0
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: nc_sm_session_start_yfw_response_cb:5256 start-session response for session ID 5615 (1025edc0)
RP/0/RSP0/CPU0:Apr 17 17:50:17.361 UTC: netconf[1116]: nc_pxs_send:206 SERVER->CLIENT 6557
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_rcv_eom:4089 EOM received, data (len:976): '<?xml version="1.0" encoding="UTF-8"?><hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><capabilities><capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability><capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability><capability>urn:ietf:params:netconf:capability:validate:1.0</capability><capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</capability><capability>urn(TRUNCATED)
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:writable-running:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:rollback-on-error:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:validate:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:confirmed-commit:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:liberouter:params:netconf:capability:power-control:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:candidate:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:xpath:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:startup:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:xml:ns:netconf:base:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2887 found capability: urn:ietf:params:netconf:capability:interleave:1.0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_hello_validate:2910 Error, Capability urn:ietf:params:netconf:base:1.1 not found
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_rcv_eom:4094 Hello message validation failed
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_ssh_decode_cb:573 Callback handling decoded data returned error: 'Netconf server' detected the 'warning' condition 'NC_ERRCODE_REQ_INVALID'
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_pxs_readable_msg_handle:5518 Decoding of incoming NETCONF data failed: 'Netconf server' detected the 'warning' condition 'NC_ERRCODE_REQ_INVALID'
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_pxs_readable_handle:5575 Failed to handle SSH IPC message: 'Netconf server' detected the 'warning' condition 'NC_ERRCODE_REQ_INVALID', msg:10271af4, session:1025edc0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: nc_sm_session_find_session_id:1349 Found session 5615 1025edc0
RP/0/RSP0/CPU0:Apr 17 17:50:17.541 UTC: netconf[1116]: TRC: yfw_req_proc.c:305:yfw_req_session_stop ctx=1000a91c,ses=100cb0a0,op=12,session stop success, removed session-id=5615
RP/0/RSP0/CPU0:Apr 17 17:50:17.542 UTC: netconf[1116]: DBG: me.c:12083:me_session_destroy ctx=1000b9b8,session destroy sctx=1010cb2c
RP/0/RSP0/CPU0:Apr 17 17:50:17.548 UTC: netconf[1116]: TRC: me_bk_sysdb_bag_decoder.c:3386:me_bk_sysdb_bag_cache_flush ctx=1000bf20,Flushing bag cache.
RP/0/RSP0/CPU0:Apr 17 17:50:17.548 UTC: netconf[1116]: TRC: me_bk_sysdb_packed_decoder.c:1119:me_bk_sysdb_pack_cache_flush ctx=1000bf20,Flushing pack cache.
RP/0/RSP0/CPU0:Apr 17 17:50:17.548 UTC: netconf[1116]: DBG: me_backend_sysdb.c:1287:sysdb_backend_session_drop ctx=1000bf20,SysDB backend session dropped (17edc1c4).
RP/0/RSP0/CPU0:Apr 17 17:50:17.548 UTC: netconf[1116]: nc_sm_session_close:6008 Closing IPC hndl: 101ff164
RP/0/RSP0/CPU0:Apr 17 17:50:17.583 UTC: SSHD_[65893]: %SECURITY-SSHD-3-ERR_GENERAL : Read from pty failed, No error
RP/0/RSP0/CPU0:Apr 17 17:50:20.948 UTC: netconf[1116]: nc_pxs_ipc_notify_callback_fn:298 IPC_NOTIFY_CLOSE
RP/0/RSP0/CPU0:Apr 17 17:50:20.948 UTC: netconf[1116]: nc_sm_pxs_notify_callback_fn:5794 Client 101ff164 closed connection
RP/0/RSP0/CPU0:Apr 17 17:50:20.948 UTC: netconf[1116]: nc_sm_pxs_notify_callback_fn:5797 Session matching with IPC client handle 101ff164 was already removed
RP/0/RSP0/CPU0:Apr 17 17:50:20.948 UTC: netconf[1116]: nc_sm_session_pxs_find:1387 No sessions stored in sessions hashmap
jejenone commented 9 years ago

So I managed to pass the hello sequence by advertising fake 1.1 capability (creating an IOS-XR device type). That said it is obvious as you explained that Netconf 1.1 protocol is fairly different, sending chunk messages instead of using the ]]>]]> end sequence of 1.0

Fri Apr 17 19:18:43.551 UTC
[04/17/15 19:18:39.869 UTC bba99 344899954] nc_px_netconf_data_flush:698 CLIENT->SERVER 247
[04/17/15 19:18:39.869 UTC bba9a 70906225] nc_ssh_frame_decode_chunked:124 Expected byte 10, received 60
[04/17/15 19:18:39.869 UTC bba9b 70906225] nc_ssh_decode_cb:563 Failed to extract frame from input buffer
[04/17/15 19:18:39.869 UTC bba9c 70906225] nc_sm_pxs_readable_msg_handle:5518 Decoding of incoming NETCONF data failed: 'Netconf server' detected the 'warning' condition 'Invalid argument'
[04/17/15 19:18:39.869 UTC bba9d 70906225] nc_sm_pxs_readable_handle:5575 Failed to handle SSH IPC message: 'Netconf server' detected the 'warning' condition 'Invalid argument', msg:1024f9e4, session:102539e4
jejenone commented 9 years ago

I managed to add support for:

Netconf 1.1 framing Netconf 1.0 over TTY (IOS-XR can only spawn a 1.0 server from TTY) IOS-XR

the branch is available here. Thinking about doing a PR but code isn't 100% clean, though it's working fine so far. Comment are welcome.

https://github.com/jejenone/ncclient/tree/jerome/netconf11

nnakamot commented 9 years ago

Leo, is it close to merge the branch? I have also added support for v1.1 framing and often get a request from the users to push it.

I have forked/merged with recent ncclient version and fixed some bugs in this repository: https://github.com/vbajpai/ncclient

einarnn commented 8 years ago

New pull request #98 waiting to merge as of today. This is an alternative to #85 submitted by nnakamot as it contains all his code as well. Has been extensively tested with the 1.1 implementation on IOS-XR 5.3.0 and later. Also interoperates with Tail-F ConfD-based agents.

Cheers,

Einar

p-alik commented 8 years ago

98 works well with ASR9000 IOS-XR 5.3.1

einarnn commented 8 years ago

Good to know, thanks!

On 14 Dec 2015, 07:43 +0000, Алексей Пастуховnotifications@github.com, wrote:

98(https://github.com/ncclient/ncclient/pull/98)works well with ASR9000 IOS-XR 5.3.1

— Reply to this email directly orview it on GitHub(https://github.com/ncclient/ncclient/issues/59#issuecomment-164370046).

einarnn commented 8 years ago

Now that the netconf:1.1 support has been merged, can this issue be closed?

Cheers,

Einar

alvarezsantiago commented 8 years ago

When can we expect these enhancements in PyPI package?

leopoul commented 8 years ago

Now it's on Pypi, v 0.4.7

alvarezsantiago commented 8 years ago

Great!! Thanks.

azlux commented 8 years ago

Still have the same problem. The ncclient send :

queueing <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability></nc:capabilities></nc:hello>

and the cisco command netconf echo format show : 

<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <capabilities>
        <capability>
            urn:ietf:params:netconf:base:1.0
        </capability>
        <capability>
            urn:ietf:params:netconf:capability:candidate:1.0
        </capability>
        <capability>
            urn:ietf:params:netconf:capability:notification:1.0
        </capability>
    </capabilities>
    <session-id>
        285212672
    </session-id>
</hello>
]]>]]>

And I have this error : ncclient.transport.errors.SessionCloseError: Unexpected session close So I think it's because of the nc: so how can I choose the remove it into the python code ? Which parameter do that ?

p-alik commented 8 years ago

get config example works fine without any changes against ASR 9001 with Cisco IOS XR Software, Version 6.0.2

The edit config example was well tested in same environment.

111pontes commented 8 years ago

Make sure that NETCONF/YANG agent is configured on the router:

ssh server v2 ssh server netconf vrf default ! netconf-yang agent ssh !

and make sure you generate your keys:

RP/0/RP0/CPU0:router#crypto key generate rsa Sun Oct 23 21:06:52.620 UTC The name for the keys will be: the_default Choose the size of the key modulus in the range of 512 to 4096 for your General Purpose Keypair. Choosing a key modulus greater than 512 may take a few minutes.

How many bits in the modulus [2048]: 4096 Generating RSA keys ... Done w/ crypto generate keypair [OK]

RP/0/RP0/CPU0:router#

p-alik commented 8 years ago

crypto key is a nice sugar feature but not mandatory

111pontes commented 8 years ago

It is.

nkaliape commented 7 years ago

@azlux Did you get solution for the problem mentioned on Oct 17 2016

111pontes commented 7 years ago

Make sure you use latest ncclient and the XR config described on Oct 23, 2016.

azlux commented 7 years ago

@nkaliape In fact, no. I still have this weird log and I run this program. The router is a Cisco IOS XR Software, Version 5.3.3 , and I use python3.6

p-alik commented 7 years ago

@azlux, on Linux your program runs fine

azlux commented 7 years ago

Tried on a debian VM, still having this error. Maybe I need to open another issue ? @p-alik

djhochjr commented 7 years ago

Along these line when working with IOS-XR, I am having difficulty when i have set ssh server capability netconf-xml. If I don't have it set, I am able to get server_capabilites and get_config. When it is set, ncclient seems to have issue about halfway trough the capabilities dump. ```Here are the Remote Devices Capabilities urn:ietf:params:netconf:base:1.0 urn:ietf:params:netconf:base:1.1 urn:ietf:params:netconf:capability:writable-running:1.0 urn:ietf:params:netconf:capability:candidate:1.0 urn:ietf:params:netconf:capability:confirmed-commit:1.0 urn:ietf:params:netconf:capability:rollback-on-error:1.0 urn:ietf:params:netconf:capability:startup:1.0 urn:ietf:params:netconf:capability:url:1.0 urn:ietf:params:netconf:capability:validate:1.0 urn:ietf:params:netconf:capability:xpath:1.0 urn:ietf:params:netconf:capability:notification:1.0 urn:liberouter:params:netconf:capability:power-control:1.0 urn:ietf:params:netconf:capability:interleave:1.0 2017-06-07T16:40:46: %AETEST-INFO: STEP 4: TEST VALIDATION - Errored 2017-06-07T16:40:46: %AETEST-ERROR: Caught exception during execution: 2017-06-07T16:40:46: %AETEST-ERROR: Traceback (most recent call last): 2017-06-07T16:40:46: %AETEST-ERROR: File "testcases/agilis_core/misc_snmp_netconf/misc_snmp_netconf_admin_group_affinity_membership_per_interface.py", line 338, in misc_snmp_netconf_admin_group_affinity_membership_per_interface_test_setup 2017-06-07T16:40:46: %AETEST-ERROR: big = m.get_config('running') 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/manager.py", line 162, in wrapper 2017-06-07T16:40:46: %AETEST-ERROR: return self.execute(op_cls, *args, *kwds) 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/manager.py", line 232, in execute 2017-06-07T16:40:46: %AETEST-ERROR: raise_mode=self._raise_mode).request(args, **kwds) 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/operations/retrieve.py", line 97, in request 2017-06-07T16:40:46: %AETEST-ERROR: return self._request(node) 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/operations/rpc.py", line 326, in _request 2017-06-07T16:40:46: %AETEST-ERROR: raise self._error 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/transport/ssh.py", line 530, in run 2017-06-07T16:40:46: %AETEST-ERROR: self._parse11() 2017-06-07T16:40:46: %AETEST-ERROR: File "/usr/local/lib/python3.6/site-packages/ncclient/transport/ssh.py", line 173, in _parse11 2017-06-07T16:40:46: %AETEST-ERROR: raise Exception 2017-06-07T16:40:46: %AETEST-ERROR: Exception



```<SSHSession(session, initial daemon)> created: client_capabilities=<dict_keyiterator object at 0x7f667ecddb38>
2017-06-02T20:28:08: %NCCLIENT-DEBUG: <SSHSession(session, initial daemon)> created: client_capabilities=<dict_keyiterator object at 0x7f667ecddb38>
installing listener <ncclient.transport.session.NotificationHandler object at 0x7f667cdbe5f8>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: installing listener <ncclient.transport.session.NotificationHandler object at 0x7f667cdbe5f8>
installing listener <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: installing listener <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>
queueing <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: queueing <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
starting main loop
2017-06-02T20:28:09: %NCCLIENT-DEBUG: starting main loop
dispatching message to <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: dispatching message to <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
dispatching message to <ncclient.transport.session.NotificationHandler object at 0x7f667cdbe5f8>: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: dispatching message to <ncclient.transport.session.NotificationHandler object at 0x7f667cdbe5f8>: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:liberouter:params:netconf:capability:power-control:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability></nc:capabilities></nc:hello>
discarding listener <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>
2017-06-02T20:28:09: %NCCLIENT-DEBUG: discarding listener <ncclient.transport.session.HelloHandler object at 0x7f6682aa6048>
initialized: session-id=0 | server_capabilities=<dict_keyiterator object at 0x7f667cdb9728>
2017-06-02T20:28:09: %NCCLIENT-INFO: initialized: session-id=0 | server_capabilities=<dict_keyiterator object at 0x7f667cdb9728>
> /agilis/testcases/agilis_core/misc_snmp_netconf/misc_snmp_netconf_admin_group_affinity_membership_per_interface.py(328)misc_snmp_netconf_admin_group_affinity_membership_per_interface_test_setup()
-> c = cisco_manager.get_config(source='running').data_xml```

My goal is to run rpc requests against the box using ncclient/netconf natively. In speaking with others on my team, we are wondering if there's a way to pass a subsystem arg in to ncclient so it doesn't freak on our ssh config?

thx,
Dave
nnakamot commented 7 years ago

djhochjr, It looks like there are several issues in netconf-xml agent that cause ncclient not work with it. I will inform the developer of the issues. In the meantime, I have following temporarily patch in ncclient to make it work with netconf-xml agent if you'd like to try out:

diff --git a/ncclient/transport/session.py b/ncclient/transport/session.py
index ea01b19..80715c5 100644
--- a/ncclient/transport/session.py
+++ b/ncclient/transport/session.py
@@ -94,7 +94,6 @@ class Session(Thread):
         self.add_listener(NotificationHandler(self._notification_q))
         listener = HelloHandler(ok_cb, err_cb)
         self.add_listener(listener)
-        self.send(HelloHandler.build(self._client_capabilities, self._device_hand
         logger.debug('starting main loop')
         self.start()
         # we expect server's hello message, if server doesn't responds in 60 seco
@@ -107,6 +106,7 @@ class Session(Thread):
             raise error[0]
         #if ':base:1.0' not in self.server_capabilities:
         #    raise MissingCapabilityError(':base:1.0')
+        self.send(HelloHandler.build(self._client_capabilities, self._device_hand
         logger.info('initialized: session-id=%s | server_capabilities=%s' %
                     (self._id, self._server_capabilities))

diff --git a/ncclient/transport/ssh.py b/ncclient/transport/ssh.py
index 9cbd1da..a6af4a8 100644
--- a/ncclient/transport/ssh.py
+++ b/ncclient/transport/ssh.py
@@ -119,10 +119,18 @@ class SSHSession(Session):
             buf.seek(0)
             msg, _, remaining = buf.read().decode('UTF-8').partition(MSG_DELIM)
             msg = msg.strip()
+            trimmed = ''
+            msglines = msg.splitlines(True)
+            if ' UTC' in msglines[0]:
+                # IOS-XR netconf-xml agent sends timestamp first so remove it
+                logger.debug("message (1st line) trimmed")
+                trimmed = ''.join(msglines[1:])
+            else:
+                trimmed = ''.join(msglines[0:])
             if sys.version < '3':
-                self._dispatch_message(msg.encode())
+                self._dispatch_message(trimmed.encode())
             else:
-                self._dispatch_message(msg)
+                self._dispatch_message(trimmed)
             # create new buffer which contains remaining of old buffer
             self._buffer = StringIO()
             self._buffer.write(remaining.encode())
@@ -539,11 +547,13 @@ class SSHSession(Session):
                 if not q.empty() and chan.send_ready():
                     logger.debug("Sending message")
                     data = q.get()
+                    is_hello = True
                     try:
                         # send a HELLO msg using v1.0 EOM markers.
                         validated_element(data, tags='{urn:ietf:params:xml:ns:net
                         data = "%s%s"%(data, MSG_DELIM)
                     except XMLError:
+                        is_hello = False
                         # this is not a HELLO msg
                         # we publish v1.1 support
                         if 'urn:ietf:params:netconf:base:1.1' in self._client_cap
@@ -554,7 +564,11 @@ class SSHSession(Session):
                                 elif 'urn:ietf:params:netconf:base:1.0' in self._
                                     # send using v1.0 EOM markers
                                     data = "%s%s"%(data, MSG_DELIM)
-                                else: raise Exception
+                                    logger.debug("Sending v1.0: %s", data)
+                                else: 
+                                    #raise Exception
+                                    # Assume v1.0 instead of exception
+                                    data = "%s%s"%(data, MSG_DELIM)
                             else:
                                 logger.debug('HELLO msg was sent, but server capa
                                 raise Exception
@@ -563,12 +577,32 @@ class SSHSession(Session):
                             # send using v1.0 EOM markers
                             data = "%s%s"%(data, MSG_DELIM)
                     finally:
-                        logger.debug("Sending: %s", data)
-                        while data:
-                            n = chan.send(data)
-                            if n <= 0:
-                                raise SessionCloseError(self._buffer.getvalue(), 
-                            data = data[n:]
+                        # Check if the server hello is from IOS-XR netconf-xml 
+                        # agent
+                        is_netconf_xml = False
+                        for cap in self._server_capabilities:
+                            if ' urn:ietf:params:netconf:base:1.0' in cap:
+                                is_netconf_xml = True
+                                break
+                        if is_hello and is_netconf_xml:
+                            # Skip sending client hello as netconf-xml agent
+                            # can't handle it
+                            logger.info("Sending client hello is skipped")
+                        else:
+                            if is_netconf_xml:
+                                # Replace nc:rpc with rpc as IOS-XR netconf-xml
+                                # agent does not recognize nc:rpc
+                                data = data.replace('nc:rpc', 'rpc')
+                                data = data.replace(
+                                'xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.
+                                'xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"'
+                                data += '\n'
+                            logger.debug("Sending: %s", data)
+                            while data:
+                                n = chan.send(data)
+                                if n <= 0:
+                                    raise SessionCloseError(self._buffer.getvalue
+                                data = data[n:]
         except Exception as e:
             logger.debug("Broke out of main loop, error=%r", e)
             self._dispatch_error(e)
djhochjr commented 7 years ago

Norio, Thank you so much. I will put the patch in place and give you feedback afterwards if you like. Lifesaver.

Dave

On Thu, Jun 8, 2017 at 2:01 PM, Norio Nakamoto notifications@github.com wrote:

djhochjr, It looks like there are several issues in netconf-xml agent that cause ncclient not work with it. I will inform the developer of the issues. In the meantime, I have following temporarily patch in ncclient to make it work with netconf-xml agent if you'd like to try out:

diff --git a/ncclient/transport/session.py b/ncclient/transport/session.py index ea01b19..80715c5 100644 --- a/ncclient/transport/session.py +++ b/ncclient/transport/session.py @@ -94,7 +94,6 @@ class Session(Thread): self.add_listener(NotificationHandler(self._notification_q)) listener = HelloHandler(ok_cb, err_cb) self.add_listener(listener)

  • self.send(HelloHandler.build(self._client_capabilities, self._device_hand logger.debug('starting main loop') self.start()

    we expect server's hello message, if server doesn't responds in 60 seco

    @@ -107,6 +106,7 @@ class Session(Thread): raise error[0]

    if ':base:1.0' not in self.server_capabilities:

     #    raise MissingCapabilityError(':base:1.0')
  • self.send(HelloHandler.build(self._client_capabilities, self._device_hand logger.info('initialized: session-id=%s | server_capabilities=%s' % (self._id, self._server_capabilities))

diff --git a/ncclient/transport/ssh.py b/ncclient/transport/ssh.py index 9cbd1da..a6af4a8 100644 --- a/ncclient/transport/ssh.py +++ b/ncclient/transport/ssh.py @@ -119,10 +119,18 @@ class SSHSession(Session): buf.seek(0) msg, _, remaining = buf.read().decode('UTF-8').partition(MSG_DELIM) msg = msg.strip()

  • trimmed = ''
  • msglines = msg.splitlines(True)
  • if ' UTC' in msglines[0]:

  • IOS-XR netconf-xml agent sends timestamp first so remove it

  • logger.debug("message (1st line) trimmed")
  • trimmed = ''.join(msglines[1:])
  • else:
  • trimmed = ''.join(msglines[0:]) if sys.version < '3':
  • self._dispatch_message(msg.encode())
  • self._dispatch_message(trimmed.encode()) else:
  • self._dispatch_message(msg)
  • self._dispatch_message(trimmed)

    create new buffer which contains remaining of old buffer

         self._buffer = StringIO()
     self._buffer.write(remaining.encode())

    @@ -539,11 +547,13 @@ class SSHSession(Session): if not q.empty() and chan.send_ready(): logger.debug("Sending message") data = q.get()

  • is_hello = True try:

    send a HELLO msg using v1.0 EOM markers.

                     validated_element(data, tags='{urn:ietf:params:xml:ns:net
                 data = "%s%s"%(data, MSG_DELIM)
             except XMLError:
  • is_hello = False

    this is not a HELLO msg

                     # we publish v1.1 support
                 if 'urn:ietf:params:netconf:base:1.1' in self._client_cap

    @@ -554,7 +564,11 @@ class SSHSession(Session): elif 'urn:ietf:params:netconf:base:1.0' in self._

    send using v1.0 EOM markers

                                 data = "%s%s"%(data, MSG_DELIM)
  • else: raise Exception
  • logger.debug("Sending v1.0: %s", data)
  • else:

  • raise Exception

  • Assume v1.0 instead of exception

  • data = "%s%s"%(data, MSG_DELIM) else: logger.debug('HELLO msg was sent, but server capa raise Exception @@ -563,12 +577,32 @@ class SSHSession(Session):

    send using v1.0 EOM markers

                         data = "%s%s"%(data, MSG_DELIM)
             finally:
  • logger.debug("Sending: %s", data)
  • while data:
  • n = chan.send(data)
  • if n <= 0:
  • raise SessionCloseError(self._buffer.getvalue(),
  • data = data[n:]

  • Check if the server hello is from IOS-XR netconf-xml

  • agent

  • is_netconf_xml = False
  • for cap in self._server_capabilities:
  • if ' urn:ietf:params:netconf:base:1.0' in cap:
  • is_netconf_xml = True
  • break
  • if is_hello and is_netconf_xml:

  • Skip sending client hello as netconf-xml agent

  • can't handle it

  • logger.info("Sending client hello is skipped")
  • else:
  • if is_netconf_xml:

  • Replace nc:rpc with rpc as IOS-XR netconf-xml

  • agent does not recognize nc:rpc

  • data = data.replace('nc:rpc', 'rpc')
  • data = data.replace(
  • 'xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.
  • 'xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"'
  • data += '\n'
  • logger.debug("Sending: %s", data)
  • while data:
  • n = chan.send(data)
  • if n <= 0:
  • raise SessionCloseError(self._buffer.getvalue
  • data = data[n:] except Exception as e: logger.debug("Broke out of main loop, error=%r", e) self._dispatch_error(e)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ncclient/ncclient/issues/59#issuecomment-307181190, or mute the thread https://github.com/notifications/unsubscribe-auth/ATtoAcjEpBKVKL7hKTkuCThISQsWF0jiks5sCDb7gaJpZM4EAj_Z .

-- Dave Hochstrasser 919.218.0060 m djhochjr@gmail.com

nnakamot commented 7 years ago

Dave, sure, please do let me know if you see any issues with the patch. I'll take a look. Please turn on these log and collect them if you see an issue.

logging.getLogger("ncclient.transport.ssh").setLevel(logging.DEBUG) logging.getLogger("ncclient.transport.session").setLevel(logging.DEBUG) logging.getLogger("ncclient.operations.rpc").setLevel(logging.DEBUG)

gageorsburn commented 7 years ago

@nnakamot I think I found the issue he was having.

nnakamot commented 7 years ago

It looks like you receive a hello message I did not expect. Can you add these lines to dump the received message?

diff --git a/ncclient/transport/ssh.py b/ncclient/transport/ssh.py index 9cbd1da..d682ea1 100644 --- a/ncclient/transport/ssh.py +++ b/ncclient/transport/ssh.py @@ -118,11 +118,22 @@ class SSHSession(Session): if MSGDELIM in buf.read().decode('UTF-8'): buf.seek(0) msg, , remaining = buf.read().decode('UTF-8').partition(MSG_DELIM)

Thanks, Norio

From: Gage Orsburn notifications@github.com Reply-To: ncclient/ncclient reply@reply.github.com Date: Thursday, June 8, 2017 at 17:11 To: ncclient/ncclient ncclient@noreply.github.com Cc: Nakamoto Norio nnakamot@cisco.com, Mention mention@noreply.github.com Subject: Re: [ncclient/ncclient] Can't send commands to IOS-XR (Cisco ASR 9000) (#59)

@nnakamothttps://github.com/nnakamot After implementing your patch, this was the result.

<SSHSession(session, initial daemon)> created: client_capabilities=<dict_keyiterator object at 0x7f4722326188>

2017-06-09T00:08:23: %NCCLIENT-DEBUG: <SSHSession(session, initial daemon)> created: client_capabilities=<dict_keyiterator object at 0x7f4722326188>

2017-06-09T00:08:23: %NCCLIENT-DEBUG: starting thread (client mode): 0x22324630

2017-06-09T00:08:23: %NCCLIENT-DEBUG: Local version/idstring: SSH-2.0-paramiko_2.1.2

2017-06-09T00:08:23: %NCCLIENT-DEBUG: Remote version/idstring: SSH-2.0-Cisco-2.0

2017-06-09T00:08:23: %NCCLIENT-INFO: Connected (version 2.0, client Cisco-2.0)

2017-06-09T00:08:23: %NCCLIENT-DEBUG: kex algos:['ecdh-sha2-nistp521', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp256', 'diffie-hellman-group14-sha1', 'diffie-hellman-group1-sha1'] server key:['ssh-rsa'] client encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr'] server encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr'] client mac:['hmac-sha2-512', 'hmac-sha2-256', 'hmac-sha1'] server mac:['hmac-sha2-512', 'hmac-sha2-256', 'hmac-sha1'] client compress:['none'] server compress:['none'] client lang:[''] server lang:[''] kex follows?False

2017-06-09T00:08:23: %NCCLIENT-DEBUG: Kex agreed: diffie-hellman-group1-sha1

2017-06-09T00:08:23: %NCCLIENT-DEBUG: Cipher agreed: aes128-ctr

2017-06-09T00:08:23: %NCCLIENT-DEBUG: MAC agreed: hmac-sha2-256

2017-06-09T00:08:23: %NCCLIENT-DEBUG: Compression agreed: none

2017-06-09T00:08:24: %NCCLIENT-DEBUG: kex engine KexGroup1 specified hash_algo

2017-06-09T00:08:24: %NCCLIENT-DEBUG: Switch to new keys ...

2017-06-09T00:08:24: %NCCLIENT-DEBUG: userauth is OK

2017-06-09T00:08:24: %NCCLIENT-INFO: Authentication (password) successful!

2017-06-09T00:08:24: %NCCLIENT-DEBUG: [chan 0] Max packet in: 32768 bytes

2017-06-09T00:08:24: %NCCLIENT-DEBUG: [chan 0] Max packet out: 32768 bytes

2017-06-09T00:08:24: %NCCLIENT-DEBUG: Secsh channel 0 opened.

2017-06-09T00:08:24: %NCCLIENT-DEBUG: [chan netconf-subsystem-0] Sesch channel 0 request ok

installing listener <ncclient.transport.session.NotificationHandler object at 0x7f47223355c0>

2017-06-09T00:08:24: %NCCLIENT-DEBUG: installing listener <ncclient.transport.session.NotificationHandler object at 0x7f47223355c0>

installing listener <ncclient.transport.session.HelloHandler object at 0x7f47223351d0>

2017-06-09T00:08:24: %NCCLIENT-DEBUG: installing listener <ncclient.transport.session.HelloHandler object at 0x7f47223351d0>

starting main loop

2017-06-09T00:08:24: %NCCLIENT-DEBUG: starting main loop

2017-06-09T00:08:24: %NCCLIENT-DEBUG: parsing netconf v1.0

2017-06-09T00:08:24: %NCCLIENT-DEBUG: parsing netconf v1.0

2017-06-09T00:08:25: %NCCLIENT-DEBUG: parsing netconf v1.0

2017-06-09T00:08:25: %NCCLIENT-DEBUG: parsing netconf v1.0

error parsing dispatch message: Document is empty, line 1, column 1 (, line 1)

2017-06-09T00:08:25: %NCCLIENT-ERROR: error parsing dispatch message: Document is empty, line 1, column 1 (, line 1)

2017-06-09T00:08:25: %NCCLIENT-DEBUG: Trying another round of parsing since there is still data

2017-06-09T00:08:25: %NCCLIENT-DEBUG: parsing netconf v1.0

2017-06-09T00:09:24: %NCCLIENT-DEBUG: Broke out of main loop, error=SessionCloseError("Unexpected session close\nIN_BUFFER: b'\\r\\n'",)

dispatching error to <ncclient.transport.session.NotificationHandler object at 0x7f47223355c0>

2017-06-09T00:09:24: %NCCLIENT-DEBUG: dispatching error to <ncclient.transport.session.NotificationHandler object at 0x7f47223355c0>

dispatching error to <ncclient.transport.session.HelloHandler object at 0x7f47223351d0>

2017-06-09T00:09:24: %NCCLIENT-DEBUG: EOF in transport thread

2017-06-09T00:09:24: %NCCLIENT-DEBUG: dispatching error to <ncclient.transport.session.HelloHandler object at 0x7f47223351d0>

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ncclient/ncclient/issues/59#issuecomment-307260757, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHYUXih5_DnOWFilSO0esj6sXNZUsZqLks5sCI2ZgaJpZM4EAj_Z.

gageorsburn commented 7 years ago

I checked the router configuration again and it's setup to use a different port.

ssh server v2
ssh server vrf mgmt
ssh server netconf port 1500
ssh server netconf vrf mgmt
ssh server netconf vrf default

I had ncclient use that instead of 22 which I guess the xml agent was running on and there were no issues at all.

I don't think there was a particular reason or requirement xml agent. I think that is still a bug in ncclient though.

nnakamot commented 7 years ago

Ok, do you have ‘netconf agent ssh’ configured? And I assume you have ‘netconf-yang agent ssh’ also configured.

‘ssh server netconf port 1500’ is to enable netconf subsystem for netconf-yang agent with port 1500. To enable subsystem for netconf-xml agent, you need ‘ssh server capability netconf-xml’ configured. Since netconf-yang agent works fine with ncclient, you did not have an issue when using port 1500 because you connect to netconf-yang agent.

To summarize;

  1. This enables netconf-xml agent on port 22 netconf agent tty ! ssh server capability netconf-xml

  2. This enables netconf-yang agent on the specified port (830 if port is not specified) netconf-yang agent ssh ! ssh server netconf port <port#>

Thanks, Norio

From: Gage Orsburn notifications@github.com Reply-To: ncclient/ncclient reply@reply.github.com Date: Thursday, June 8, 2017 at 18:11 To: ncclient/ncclient ncclient@noreply.github.com Cc: Nakamoto Norio nnakamot@cisco.com, Mention mention@noreply.github.com Subject: Re: [ncclient/ncclient] Can't send commands to IOS-XR (Cisco ASR 9000) (#59)

I checked the router configuration again and it's setup to use a different port.

ssh server v2

ssh server vrf mgmt

ssh server netconf port 1500

ssh server netconf vrf mgmt

ssh server netconf vrf default

I had ncclient use that instead of 22 which I guess the xml agent was running on and there were no issues at all.

I don't think there was a particular reason or requirement xml agent. I think that is still a bug in ncclient though.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ncclient/ncclient/issues/59#issuecomment-307268261, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHYUXgczpxdVXKcZ5MZ_NWZo4iG_7FaZks5sCJvQgaJpZM4EAj_Z.

leopoul commented 7 years ago

Can you please try to run with correct device parameters? Instead of:

device_params={'name': 'default'},

use:

device_params={'name': 'iosxr'},