CESNET / netopeer

NETCONF Protocol Toolset
117 stars 65 forks source link

Requests timing out after upgrade to Netopeer 1.1.53 #195

Closed jagmsing closed 3 years ago

jagmsing commented 3 years ago

We are using Netopeer2-1.1.39 and have our regression setup working fine, We are upgrading to 1.1.53 version and found multiple failures after upgrade to 1.1.53 version.

  1. Subscription requests for default stream with different filters are timing out on same session . This is working in the 1.1.39.

In 1.1.39:

subscribe --filter-xpath /o-ran-troubleshooting:troubleshooting-log-generated OK subscribe --filter-xpath /o-ran-file-management:file-upload-notification OK

In 1.1.53

subscribe --filter-xpath /o-ran-troubleshooting:troubleshooting-log-generated OK subscribe --filter-xpath /o-ran-file-management:file-upload-notification _cli_sendrecv: Timeout for receiving a reply expired.

  1. Edit-config requests are timing out with below error. 'ERROR type: application tag: operation-failed severity: error message: Callback event "rpc" with ID 25 processing timed out.

Can you please help here understanding the change in behaviour and reason for these timeouts.

Netopeer and sysrepo versions listed below:

Netopeer2-1.1.53 libyang-1.0.215 sysrepo-1.4.104 libssh-0.9.5-1

michalvasko commented 3 years ago

Please update to netopeer2 1.1.70, there is no reason not to. In version 1.1.53 there were some serious bugs connected to the new features (in sysrepo) so updating may solve your issue. But looking at the use-case specifically, it seems you are subscribing on one NETCONF session a second time. That was incorrectly allowed on previous netopeer2 version and should produce the correct error in the latest version, that it is forbidden.

jagmsing commented 3 years ago

Thanks Michal , I am clear about the subscription part and indeed was the expectation.

Regarding the 1.1.53 version, Is there any way to know about the features impacted by bugs in 1.1.53 version. Or Does it impacts all the enabled features in sysrepo ?

michalvasko commented 3 years ago

Does it impacts all the enabled features in sysrepo ?

Not sure what you mean by "enabled features", I am simply talking about changes that made it to that release. So yes, there were some quite serious bugs, in general, so you should update everything to the latest release. That is what you should always do, ideally.

jagmsing commented 3 years ago

Thanks Michal.

Regarding the subscription part , I tried from different session and even then the failure is observed for second session.

connect --host 127.0.0.1 --login tesuser testuser@127.0.0.1 password: subscribe --filter-xpath /o-ran-troubleshooting:troubleshooting-log-generated _cli_sendrecv: Timeout for receiving a reply expired.

Is it possible to list any such issue(s) in 1.1.53 build for reference so that we have strong reason to upgrade to 1.1.70 version.

regards, Jagmeet

jagmsing commented 3 years ago

Hi Michal,

I realised I raised issue on netopeer repo instead of netopeer2 repo. Upon checking I found issue 843 which is related to my query. Instead of crash I am observing timeout issue.

So my question is whether new subscription over the new cli session with the same user would be allowed ? or does that also has any impact of this change.

regards, Jagmeet

michalvasko commented 3 years ago

I have missed you created this in netopeer as well. And no, you cannot have several subscriptions on a single session, the simple netopeer2-cli does not support more sessions.

jagmsing commented 3 years ago

Thanks Michal , I have upgraded netopeer version to 1.70 .

I have some queries about parallel operations in Netopeer. Are parallel operations supported by Netopeer i.e case below :

Netconf client 1: Edit conf session is ongoing.

Netconf client 2: Get conf /Edit conf operation is triggered.

Does the new requests queued up or allowed in parallel to the ongoing operations. Is there any related documentation/RFC I can refer to for this.

michalvasko commented 3 years ago

The concurrency is currently limited by sysrepo. And in the case you mentioned, the operations can be, for the most part, executed in parallel. Only once the edit-config reaches the point of writing into the datastore, WRITE locks are held so no other operation can access the data until they are updated.

jagmsing commented 3 years ago

Thanks Michal .