search

CESNET / netopeer2

NETCONF toolset
BSD 3-Clause "New" or "Revised" License
300 stars 189 forks source link

Cannot login via netconf #1526

Closed nhathaway closed 10 months ago

nhathaway commented 10 months ago

I am getting this

root@5399f5a44b43:/# netopeer2-cli
get_netconf_dir: Configuration directory "/root/.netopeer2-cli" did not exist, created.
load_history: No saved history.
load_config: No saved configuration.
> connect --host localhost --login netconf
The authenticity of the host 'localhost' cannot be established.
ssh-rsa key fingerprint is 41:8e:c1:f1:10:31:70:58:42:ab:19:2f:05:51:30:9b:f0:da:75:53.
Are you sure you want to continue connecting (yes/no)? yes
netconf@localhost password:
ly ERROR: Augment target node "private-key-type" in grouping "asymmetric-key-pair-with-certs-grouping" was not found. (Path "/ietf-keystore:keystore/{uses='keystore-grouping'}/asymmetric-keys/asymmetric-key/{uses='ks:asymmetric-key-pair-with-certs-grouping'}/{uses='ct:asymmetric-key-pair-with-certs-grouping'}".)
cmd_connect: Connecting to the localhost:830 as user "netconf" failed.
>

It seems to imply that there is a problem with the yangs supplied with netopeer. What is going wrong?

I am using the version set from the August 2023 release.

nhathaway commented 10 months ago

I notice that we have 2 versions of ietf-crypto-types

ietf-crypto-types@2019-04-29.yang ietf-crypto-types@2019-07-02.yang

Could it be picking up the wrong version? In previous versions, it was OK. This has only happened since we upgraded.

nhathaway commented 10 months ago

According to RFC 7950, if a yang file imports another yang file without specifying the revision, it will import the latest version available. In this case it should be 2019-07-02. However, both have 'asymmetric-key-pair-with-certs-grouping', so it shouldn't fail like this.

Roytak commented 10 months ago

Hi, this is most likely due to picking up the wrong version of the module as you said. It is weird that you don't have the latest version of ietf-crypto-types, that is ietf-crypto-types@2023-12-28.yang. Check the directory of the installed modules (/usr/local/share/yang/modules/libnetconf2/ by default) to see if the latest revision is there, if not try installing libnetconf2 again. If it is, try removing the offending revisions. If that doesn't work, Michal should be here tomorrow to help.

nhathaway commented 10 months ago 
root@5399f5a44b43:/# ls /usr/share/yang/modules/netopeer2
iana-crypt-hash@2014-08-06.yang    ietf-ip@2018-02-22.yang                  ietf-netconf-nmda@2019-01-07.yang      ietf-restconf@2017-01-26.yang                  ietf-tcp-client@2019-07-02.yang  ietf-tls-server@2019-07-02.yang         ietf-yang-push@2019-09-09.yang
ietf-crypto-types@2019-07-02.yang  ietf-keystore@2019-07-02.yang            ietf-netconf-server@2019-07-02.yang    ietf-ssh-common@2019-07-02.yang                ietf-tcp-common@2019-07-02.yang  ietf-truststore@2019-07-02.yang         ietf-yang-schema-mount@2019-01-14.yang
ietf-datastores@2018-02-14.yang    ietf-netconf-acm@2018-02-14.yang         ietf-netconf@2013-09-29.yang           ietf-ssh-server@2019-07-02.yang                ietf-tcp-server@2019-07-02.yang  ietf-x509-cert-to-name@2014-12-10.yang  nc-notifications@2008-07-14.yang
ietf-interfaces@2018-02-20.yang    ietf-netconf-monitoring@2010-10-04.yang  ietf-network-instance@2019-01-21.yang  ietf-subscribed-notifications@2019-09-09.yang  ietf-tls-common@2019-07-02.yang  ietf-yang-patch@2017-02-22.yang         notifications@2008-07-14.yang
root@5399f5a44b43:/#
Roytak commented 10 months ago

The modules were moved from netopeer2 to libnetconf2 in the latest release. Try removing everything from this directory and installing netopeer2 again. You may want to do the same for libnetconf2, if it is still causing issues.

nhathaway commented 10 months ago

I am not using the 'lastest' release, but the one from August 2023. According to github, you only made that release yesterday.

Roytak commented 10 months ago

Sorry, I completely missed that part and rushed to conclusions! Will get back to you.

michalvasko commented 10 months ago

Yes, it is definitely a problem with YANG module revisions. Try running verb verbose before connecting on the CLI, it should then print the path to the problematic YANG and you can remove it. But, it would be best to then remove all the old revisions because they can cause issues even if they succeed to load.

nhathaway commented 10 months ago 
root@5399f5a44b43:/# netopeer2-cli
load_config: No saved configuration.
> verb verbose
> connect --host localhost --login netconf
nc VERBOSE: Trying to connect via IPv4 to 127.0.0.1:830.
nc VERBOSE: Successfully connected to localhost:830 over IPv4.
[2024/01/18 08:21:09.916268, 2] ssh_config_parse_line:  Unapplicable option: SendEnv, line: 51
[2024/01/18 08:21:09.916286, 1] ssh_config_parse_line:  Unsupported option: HashKnownHosts, line: 52
[2024/01/18 08:21:09.916302, 2] ssh_connect:  libssh 0.9.6 (c) 2003-2021 Aris Adamantiadis, Andreas Schneider and libssh contributors. Distributed under the LGPL, please refer to COPYING file for information about your rights, using threading threads_pthread
[2024/01/18 08:21:09.916313, 2] ssh_connect:  Socket connecting, now waiting for the callbacks to work
[2024/01/18 08:21:09.916342, 1] socket_callback_connected:  Socket connection callback: 1 (0)
[2024/01/18 08:21:09.959923, 2] ssh_client_connection_callback:  SSH server banner: SSH-2.0-libssh_0.9.6
[2024/01/18 08:21:09.959953, 2] ssh_analyze_banner:  Analyzing banner: SSH-2.0-libssh_0.9.6
[2024/01/18 08:21:09.961153, 2] ssh_kex_select_methods:  Negotiated curve25519-sha256,rsa-sha2-512,aes256-gcm@openssh.com,aes256-gcm@openssh.com,aead-gcm,aead-gcm,none,none,,
[2024/01/18 08:21:09.963284, 2] ssh_init_rekey_state:  Set rekey after 4294967296 blocks
[2024/01/18 08:21:09.963314, 2] ssh_init_rekey_state:  Set rekey after 4294967296 blocks
[2024/01/18 08:21:09.963324, 2] ssh_packet_newkeys:  Received SSH_MSG_NEWKEYS
[2024/01/18 08:21:09.963400, 2] ssh_packet_newkeys:  Signature verified and valid
[2024/01/18 08:21:09.965558, 1] ssh_packet_userauth_failure:  Access denied for 'none'. Authentication that can continue: publickey,password
[2024/01/18 08:21:09.965601, 2] ssh_packet_userauth_failure:  Access denied for 'none'. Authentication that can continue: publickey,password
nc VERBOSE: Publickey athentication.
nc VERBOSE: No key pair specified.
nc WARNING: Authentication denied.
nc VERBOSE: Password authentication (host "localhost", user "netconf").
netconf@localhost password:
nc VERBOSE: Authentication successful.
[2024/01/18 08:21:21.474881, 2] channel_open:  Creating a channel 43 with 64000 window and 32768 max packet
[2024/01/18 08:21:21.475358, 2] ssh_packet_channel_open_conf:  Received a CHANNEL_OPEN_CONFIRMATION for channel 43:43
[2024/01/18 08:21:21.475379, 2] ssh_packet_channel_open_conf:  Remote window : 32000, maxpacket : 35000
[2024/01/18 08:21:21.475766, 2] channel_request:  Channel request subsystem success
ly VERBOSE: Searching for "ietf-inet-types" in "/".
ly VERBOSE: Newer revision than "ietf-inet-types@2013-07-15" not found, using this as the latest revision.
ly VERBOSE: Searching for "ietf-yang-types" in "/".
ly VERBOSE: Newer revision than "ietf-yang-types@2013-07-15" not found, using this as the latest revision.
[2024/01/18 08:21:21.477064, 2] grow_window:  growing window (channel 43:43) to 1280000 bytes
nc VERBOSE: Capability for <get-schema> support found.
nc VERBOSE: Capability for yang-library support found.
nc VERBOSE: Capability for XPath filter support found.
nc VERBOSE: Capability for NMDA RPCs support not found.
nc VERBOSE: Reading module "ietf-netconf@<latest>" from local file "/usr/share/yang/modules/netopeer2/ietf-netconf@2013-09-29.yang".
nc VERBOSE: Reading module "ietf-netconf-acm@2018-02-14" from local file "/usr/share/yang/modules/netopeer2/ietf-netconf-acm@2018-02-14.yang".
nc VERBOSE: Reading module "ietf-yang-metadata@2016-08-05" from local file "/usr/share/yang/modules/libyang/ietf-yang-metadata@2016-08-05.yang".
nc VERBOSE: Reading module "ietf-yang-library@2019-01-04" from local file "/usr/share/yang/modules/sysrepo/ietf-yang-library@2019-01-04.yang".
nc VERBOSE: Unable to identify revision of the import module "ietf-datastores" from the available server side information.
nc VERBOSE: Reading module "ietf-datastores@<latest>" from server via get-schema.
[2024/01/18 08:21:21.481364, 2] channel_rcv_change_window:  Adding 1248206 bytes to channel (43:43) (from 31782 bytes)
nc VERBOSE: Reading module "sysrepo@2023-06-16" from local file "/usr/share/yang/modules/sysrepo/sysrepo.yang".
nc VERBOSE: Reading module "ietf-factory-default@2020-08-31" from local file "/usr/share/yang/modules/sysrepo/ietf-factory-default.yang".
nc VERBOSE: Reading module "sysrepo-factory-default@2023-02-23" from local file "/usr/share/yang/modules/sysrepo/sysrepo-factory-default.yang".
nc VERBOSE: Reading module "sysrepo-monitoring@2023-08-11" from local file "/usr/share/yang/modules/sysrepo/sysrepo-monitoring.yang".
nc VERBOSE: Reading module "sysrepo-plugind@2022-08-26" from local file "/usr/share/yang/modules/sysrepo/sysrepo-plugind.yang".
nc VERBOSE: Reading module "ietf-netconf-with-defaults@2011-06-01" from local file "/usr/share/yang/modules/sysrepo/ietf-netconf-with-defaults.yang".
nc VERBOSE: Reading module "ietf-netconf-notifications@2012-02-06" from local file "/usr/share/yang/modules/sysrepo/ietf-netconf-notifications.yang".
nc VERBOSE: Reading module "ietf-origin@2018-02-14" from local file "/usr/share/yang/modules/sysrepo/ietf-origin.yang".
nc VERBOSE: Reading module "ietf-netconf-nmda@2019-01-07" from local file "/usr/share/yang/modules/netopeer2/ietf-netconf-nmda@2019-01-07.yang".
nc VERBOSE: Reading module "nc-notifications@2008-07-14" from local file "/usr/share/yang/modules/netopeer2/nc-notifications@2008-07-14.yang".
nc VERBOSE: Reading module "notifications@2008-07-14" from local file "/usr/share/yang/modules/netopeer2/notifications@2008-07-14.yang".
nc VERBOSE: Reading module "ietf-x509-cert-to-name@2014-12-10" from local file "/usr/share/yang/modules/netopeer2/ietf-x509-cert-to-name@2014-12-10.yang".
nc VERBOSE: Reading module "ietf-keystore@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-keystore@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-crypto-types@2019-04-29" from server via get-schema.
nc VERBOSE: Reading module "ietf-truststore@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-truststore@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-tcp-common@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-tcp-common@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-ssh-server@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-ssh-server@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-ssh-common@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-ssh-common@2019-07-02.yang".
nc VERBOSE: Reading module "iana-crypt-hash@2014-08-06" from local file "/usr/share/yang/modules/netopeer2/iana-crypt-hash@2014-08-06.yang".
nc VERBOSE: Reading module "ietf-tls-server@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-tls-server@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-tls-common@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-tls-common@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-netconf-server@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-netconf-server@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-tcp-client@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-tcp-client@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-tcp-server@2019-07-02" from local file "/usr/share/yang/modules/netopeer2/ietf-tcp-server@2019-07-02.yang".
nc VERBOSE: Reading module "ietf-interfaces@2018-02-20" from local file "/usr/share/yang/modules/netopeer2/ietf-interfaces@2018-02-20.yang".
nc VERBOSE: Reading module "ietf-ip@2018-02-22" from local file "/usr/share/yang/modules/netopeer2/ietf-ip@2018-02-22.yang".
nc VERBOSE: Reading module "ietf-network-instance@2019-01-21" from local file "/usr/share/yang/modules/netopeer2/ietf-network-instance@2019-01-21.yang".
nc VERBOSE: Reading module "ietf-subscribed-notifications@2019-09-09" from local file "/usr/share/yang/modules/netopeer2/ietf-subscribed-notifications@2019-09-09.yang".
nc VERBOSE: Reading module "ietf-restconf@2017-01-26" from local file "/usr/share/yang/modules/netopeer2/ietf-restconf@2017-01-26.yang".
nc VERBOSE: Reading module "ietf-yang-push@2019-09-09" from local file "/usr/share/yang/modules/netopeer2/ietf-yang-push@2019-09-09.yang".
nc VERBOSE: Reading module "ietf-yang-patch@2017-02-22" from local file "/usr/share/yang/modules/netopeer2/ietf-yang-patch@2017-02-22.yang".
nc VERBOSE: Reading module "ietf-ipv4-unicast-routing@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "ietf-routing@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "ietf-ipv6-unicast-routing@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "ietf-ipv6-router-advertisements@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "ietf-system@2014-08-06" from server via get-schema.
nc VERBOSE: Reading module "iana-if-type@2017-01-19" from server via get-schema.
nc VERBOSE: Reading module "iana-hardware@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "o-ran-wg4-features@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-usermgmt@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-operations@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "ietf-hardware@2018-03-13" from server via get-schema.
nc VERBOSE: Reading module "o-ran-supervision@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-interfaces@2021-12-01" from server via get-schema.
nc VERBOSE: Reading module "o-ran-file-management@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-software-management@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-hardware@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "o-ran-trace@2022-08-15" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-nrcelldu@2021-10-28" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-yang-types@2022-02-09" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-managed-function@2022-01-07" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-top@2019-06-17" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-measurements@2021-07-22" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-trace@2022-04-27" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-managed-element@2021-01-16" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-subscription-control@2021-01-16" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-fm@2021-08-08" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-gnbdufunction@2021-10-28" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-rrmpolicy@2020-11-05" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-5g-common-yang-types@2021-08-05" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-nrcellcu@2021-01-25" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-gnbcucpfunction@2021-11-06" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-gnbcuupfunction@2020-11-05" from server via get-schema.
nc VERBOSE: Reading module "node-h-interfaces@2023-11-14" from server via get-schema.
nc VERBOSE: Reading module "node-h-pmstats-control@2022-10-12" from server via get-schema.
nc VERBOSE: Reading module "node-h-leds@2023-01-04" from server via get-schema.
nc VERBOSE: Reading module "node-h-licensing@2023-04-18" from server via get-schema.
nc VERBOSE: Reading module "node-h-simulation@2022-09-30" from server via get-schema.
nc VERBOSE: Reading module "o-ran_3gpp-nr-nrm-nrcelldu@2020-09-25" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-bwp@2020-11-17" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-ep@2022-01-07" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-common-ep-rp@2020-06-08" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-nrfreqrelation@2020-04-23" from server via get-schema.
nc VERBOSE: Reading module "_3gpp-nr-nrm-nrcellrelation@2021-01-25" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-ep@2023-05-15" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-nrcellcu-extension@2023-01-31" from server via get-schema.
nc VERBOSE: Reading module "o-ran-cu-security-handling@2021-07-04" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-nrcelldu-extension@2024-01-06" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-nrcellrelation-extension@2023-01-31" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-gnbdufunction-extension@2023-01-31" from server via get-schema.
nc VERBOSE: Reading module "node-h-5g-gnbcuupfunction-extension@2023-01-31" from server via get-schema.
[2024/01/18 08:21:21.594083, 2] grow_window:  growing window (channel 43:43) to 1280000 bytes
ly ERROR: Augment target node "private-key-type" in grouping "asymmetric-key-pair-with-certs-grouping" was not found. (Path "/ietf-keystore:keystore/{uses='keystore-grouping'}/asymmetric-keys/asymmetric-key/{uses='ks:asymmetric-key-pair-with-certs-grouping'}/{uses='ct:asymmetric-key-pair-with-certs-grouping'}".)
cmd_connect: Connecting to the localhost:830 as user "netconf" failed.
>
michalvasko commented 10 months ago

Can you post the output of sysrepoctl -l? My guess is that one of your custom YANG modules is importing the older revision ietf-crypto-types@2019-04-29, but netopeer2 requires ietf-crypto-types@2019-07-02.yang.

nhathaway commented 10 months ago

Yes, that is what is happening, and the standards require that 2 different yangs at 2 different versions can be loaded at the same without the system breaking. This used to work OK on netopeer2, but now it doesn't.

ietf-crypto-types                   | 2019-07-02 | i     |              |               |                                 |
ietf-crypto-types                   | 2019-04-29 | i     |              |               |                                 |
nhathaway commented 10 months ago

Attached are some of the yangs. o-ran-software-management calls up o-ran-file-management which in turn calls up the older ietf-crypto-types. You can try it out for yourself.

yangs.zip

michalvasko commented 10 months ago

So, I have installed o-ran-software-management (with all its dependencies), then netopeer2-server (which installed its modules) and it worked fine. However, I have used the current versions where the newer ietf-crypto-types is implemented because there are some features that need to be enabled, which is why I think it worked. So, just manually implement ietf-crypto-types@2019-07-02 (using sysrepoctl -i <yang>) and I think it will work for you as well.

nhathaway commented 10 months ago 
root@5399f5a44b43:/# sysrepoctl -i /etc/sysrepo/yang/ietf-crypto-types\@2019-07-02.yang
root@5399f5a44b43:/# netopeer2-cli
load_config: No saved configuration.
> connect --host localhost --login netconf
netconf@localhost password:
ly ERROR: Augment target node "private-key-type" in grouping "asymmetric-key-pair-with-certs-grouping" was not found. (Path "/ietf-keystore:keystore/{uses='keystore-grouping'}/asymmetric-keys/asymmetric-key/{uses='ks:asymmetric-key-pair-with-certs-grouping'}/{uses='ct:asymmetric-key-pair-with-certs-grouping'}".)
cmd_connect: Connecting to the localhost:830 as user "netconf" failed.
>
nhathaway commented 10 months ago

Which version are you using? Your latest release from a couple of days ago, or the one from last August?

michalvasko commented 10 months ago

I am (always) using the current ones from the devel branch even. But there are no significant changes from the latest release. But like I said, I was hoping it would not matter (although it is always good to use the latest release if it is not a problem), what is your output of sysrepoctl -l now?

nhathaway commented 10 months ago 
ietf-crypto-types                   | 2019-07-02 | I     | root:root    | 600           |                                 |
ietf-crypto-types                   | 2019-04-29 | i     |              |               |                                 |
michalvasko commented 10 months ago

Okay, I have verified it does not work using the previous release for some reason. So if you still want to use it (if you upgraded just now, why not use the latest release), you will probably have to specify the revision-date for imports of ietf-crypto-types to ietf-keystore and ietf-truststore. And fully reinstall all the projects so that the modified modules are really used.

nhathaway commented 10 months ago

We will move to your latest software release. Thanks for confirming.

nhathaway commented 10 months ago

Unfortunately, the new version requires openssl 3.0.x, and we are stuck with openssl 1.1.1 for now. But we've managed to get newer versions of the ORAN yangs, and they've fixed the need for the older yang.