TLS to Nokia router difficulties

bmah888 commented 6 years ago

Having some issues trying to gNMI_Subscribe.py to establish a connection using TLS to a Nokia 7750src4 running TiMOS-B-15.0.R4. Slightly anonymized output (using PR #4):

netlab-collector:gnmi% bin/python pygnmi/gNMI_Subscribe.py --server netlab-7750src4-rt1-es1.example.net:57400 --username bmah --tls --cert ~/Cert/netlab-7750src4-rt1-es1.example.net/grpc.cert --submode 0
18/04/24 15:14:11,241 Sending SubscribeRequest
subscribe {
  subscription {
    path {
    }
    sample_interval: 10000000000
  }
}

18/04/24 15:14:11,250 grpc.RpcError received:
<bound method _Rendezvous.details of <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>>

Any ideas about how I would go about debugging this? Weirdly, the Arista golang gNMI client at https://github.com/aristanetworks/goarista/tree/master/cmd/gnmi works fine:

netlab-collector:gnmi% ~/go/bin/gnmi -addr netlab-7750src4-rt1-es1.example.net:57400 -cafile ~/Cert/netlab-7750src4-rt1-es1.example.net/grpc.cert -username bmah subscribe /
/configure/qos/sap-ingress[policy-id=2]/policy-id = 2
/configure/qos/sap-ingress[policy-id=2]/description = "ESnet SAP Ingress QoS Policy"
/configure/qos/sap-ingress[policy-id=2]/default-priority = "high"
/configure/qos/sap-ingress[policy-id=2]/queue[queue-id=2]/queue-id = 2
/configure/qos/sap-ingress[policy-id=2]/queue[queue-id=2]/queue-type = "non-expedited"
...

Also, gNMI_Subscribe.py connecting to an Arista 7504r running EOS 4.20.3F works with no issues.

netlab-collector:gnmi% bin/python pygnmi/gNMI_Subscribe.py --server netlab-7504r-rt1.example.net:6030 --username bmah --tls --cert ~/Cert/netlab-7504r-rt1.example.net/grpc.cert --submode 0
18/04/24 15:30:25,56  Sending SubscribeRequest
subscribe {
  subscription {
    path {
    }
    sample_interval: 10000000000
  }
}

18/04/24 15:30:25,86  Update received
update {
  timestamp: 1520544456089936237
  update {
    path {
      elem {
...

Thanks for any insights!

wisotzky commented 6 years ago

Hi Bruce,

Do you have any contact in Nokia? I've compiled info about the SROS gNMI in a slide-deck, which I am trying to keep up-to-date. It contains all the info needed to use this python tool and trouble-shooting guidance.

Generally the tool supports some debugging. You need to use the option "-v" while the number of v's defined the log-level. So you could try it with "-vvvvvv" if needed.

I have not yet looked into the Arista tool, but likely they have just hijacked the tooling from OpenConfig contributed by Google.

From my experience, the "golang" solutions have the benefit, that Google has built a dedicated protocols stack (gRPC/TLS). So the logging is much better and you can even run those tools in "insecure" mode - which basically means that the certificates are not required to be valid or you can accept the server certificate without checking/having the CA cert. So clearly, there is no surprise here, that golang works nicer than python based tools.

About tool versions, you need to be careful. The latest version of this python tool only support gNMI version 0.4 (and newer). Major changes have been introduced from 0.3 to 0.4 as the path encoding has changed. In conclusion, this python tool does not longer support nodes implementing gNMI 0.2 or 0.3...

As you are using a node in 15.0.R4 - I think the gNMI version is 0.3. Therefore you should try the previous version of this python tool to ensure compatibility.

In SROS 16.0 we are introducing quite some changes in SROS. We will support "insecure" mode (unencrypted) to avoid issues with certificates - especially in lab and DevOps environments. Also there is support for Get/Set and on_change coming.

/wiso

PS: Please come back to me, if you can not get it running. Else I would close the ticket.

bmah888 commented 6 years ago

Hey wiso,

Thanks for the quick reply! I appreciate your insights on the changes in the gNMI standard between version 0.3 and 0.4, I somehow had the impression that SROS 15.0.R4 supported gNMI 0.4, but I just went back and checked the documentation and sure enough it only supports gNMI 0.3. So we know not to try this anymore, at least not until we get a newer SROS.

I think supporting an insecure mode of operation would be very helpful (it exactly matches our current use case). One other thing I've discovered so far is that I wasn't able to import a self-signed certificate generated by libreSSL (on macOS), however a certificate created with exactly the same operation on OpenSSL (on FreeBSD) worked fine.

I can tell you who we've been speaking to at Nokia about these topics, but I'd rather not spam their names into a public GitHub issue tracker. Is it OK if we take this conversation to email? My email address is bmah@es.net. I can also tell you a little more about what we're doing in case you're interested.

Thanks again!

Bruce.

PS. It's fine to close this issue, but I'll leave that up to your discretion.

nokia / pygnmi

TLS to Nokia router difficulties #5