Memory Usage

[timho93]

Hey,

i have some issues with memory usage when using establish_subscription. I subscribe to data of configured IP SLAs. Every second the I am getting a notification. And from the moment of subscription my script has increasing memory usage (ca. 5 MB per minute with only on subscription). Some explanation?

def establish_subscription(self, netconf_client, xpath, context, period=None, dampening_period=None):
    try:
        if all(var is None for var in [period, dampening_period]):
            logging.warning("default dampening period of 1s will be used" % (self.identification))
            dampening_period = 100
        else:
            s = netconf_client.establish_subscription(
                getattr(self, "%s_callback" % context),
                getattr(self, "default_errback"),
                period=period,
                dampening_period=dampening_period,
                xpath=xpath)
            if s.subscription_result.endswith('ok'):
                logging.info("successfully established subsrciption to xpath %s for node %s " % (xpath, self.identification))
                return s.subscription_id
    except Exception as e:
        raise Exception("failed to establish subscription on node %s \n %s" % (self.identification, str(e)))

def monitor_rtt_callback(self, notif):
    #logging.info('<<-- Received a notification from node %s !!!' % self.identification )
    # show data included in notification:
    #logging.info(etree.tostring(notif.datastore_ele, pretty_print=True).decode('utf-8'))
    # namespaces which should be exluded in json formatted notification
    namespaces = {
        "urn:ietf:params:xml:ns:yang:ietf-yang-push": None,
        "http://cisco.com/ns/yang/Cisco-IOS-XE-ip-sla-oper": None
    }
    try:
        json_format = xmltodict.parse(
            etree.tostring(notif.datastore_ele, pretty_print=True).decode('utf-8'),
            process_namespaces=True,
            namespaces=namespaces
        )
    except Exception as e:
        logging.error(str(e))
    datastore = json_format["datastore-contents-xml"]
    ipsla_stats = datastore["ip-sla-stats"]["sla-oper-entry"]
    measurement_list = []
    measurement = {}
    measurement.update({"measurement": ipsla_stats["oper-type"]})
    measurement.update({"tags": {"oper-id": ipsla_stats["oper-id"]}})
    measurement.update({"fields": {}})
    measurement["fields"].update({"rtt": int(ipsla_stats["rtt-info"]["latest-rtt"]["rtt"])})
    measurement_list.append(measurement)
    ...

[einarnn]

Tim,

We haven’t done any memory profiling on this. If you’re able to run your script with cProfile or pympler, and share your results, that would be great.

Cheers,

Einar

[einarnn]

Ok, done a little bit of investigation and I think I've isolated the issue. This is caused by there being two consumers of notifications currently, basic RFC 5277 support introduced a while ago, and pre-RFC YANG Push support. What is happening is that the notification is getting pushed onto the RFC 5277 notification queue here, but your code is processing a notification that is created here, using the same underlying data buffer. The first notification is never consumed, and so lingers on the _notification_q, consuming the memory indefinitely.

I'll look into how to best fix this, but there is potentially a fundamental design issue with ncclient in that it, by design, allows multiple listeners for the same <rpc-reply> content today. This has always made me a little bit uncomfortable, so now I need to dig into this more and look at unifying the RFC 5277 and YANG Push behaviors.

If this is causing you critical problems in the short term, you can patch by disabling the RFC 5277 listener, which you can do by commenting out this line.

Cheers,

Einar

[timho93]

Hi Einar,

thank your for your efforts. It seems to work perfect for now with this workaround. Does the workaround have any other effects to functionality of ncclient?

Regards, Tim

[einarnn]

Tim,

Yes, the workaround impacts basic RFC 5277 support (‘create-subscription’), effectively meaning that those events, if generated, will be sent to whatever YANG Push callbacks are installed, at which point they will be dropped as they will not have a ‘subscription-id’ tag to enable the notification to be mapped to a callback.

Cheers,

Einar On Nov 3, 2018, 23:29 +0000, Tim Hofer notifications@github.com, wrote:

Hi Einar, thank your for your efforts. It seems to work perfect for now with this workaround. Does the workaround have any other effects to functionality of ncclient? Regards, Tim — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.