rticommunity / rticonnextdds-connector-py

RTI Connector for Connext DDS is a lightweight technology that enables DDS data to be accessed with Python.
Other
34 stars 21 forks source link

increase in memory consumption #155

Closed aelhadee closed 2 years ago

aelhadee commented 2 years ago

Hello, I am using the writer and reader sample codes to test data transmission between two devices. I noticed one of the devices (receiver) memory consumption (using top in linux) is increasing over time.

Any thoughts why might that be happening? I used the same exact code with other protocols but didn't observe this behavior.

Thanks, AE

samuelraeburn commented 2 years ago

At what point in the application are you noticing the memory growth? Connector uses Connext DDS under the hood, and we try to use static allocations wherever possible once an application reaches steady state. Steady state would be that the entities have discovered each other and resource pools have been allocated.

One way you can try to reduce memory growth is via the Resource Limits Qos policy (supported in Connector). You should apply any changes to the XML file that you are using to load Connector.

Have you made any modifications to the example code here?

I don't quite understand what you mean by this part:

I used the same exact code with other protocols but didn't observe this behavior.

Connector is designed on top of DDS, it is not possible to use a different protocol.

aelhadee commented 2 years ago

I don't quite understand what you mean by this part:

I used the same exact code with other protocols but didn't observe this behavior.

I should have clarified more. The code I was referring to is the reset of the code that I added, which I used with another protocol such as socket but didn't see this behavior, so it seems to be caused by DDS not the addition I made.

Have you made any modifications to the example code here?

Yes, I put each script in a while loop and had the writer generate a string and publish it at a faster rate like every 10 milliseconds or less. So the memory might be dynamically allocated by default and I need to use the resource limits, will give that a try. Is there any examples that you can share for the Resources limits QoS?

At what point in the application are you noticing the memory growth?

It's happens from the beginning when the script starts, the reader script only though, so it's seems like there is buffer or history being kept.

Thank you!

samuelraeburn commented 2 years ago

Ok, and you are using the same type as the example, right?

        <struct name="ShapeType" extensibility="extensible">
            <member name="color" stringMaxLength="128" type="string" key="true"/>
            <member name="x" type="long"/>
            <member name="y" type="long"/>
            <member name="shapesize" type="long"/>
        </struct>

If so the issue is that the color string is marked as a key in the type definition. In DDS a key is used to distinguish instances of the same type. Since you are creating many instances of the shape type, each instance has some overhead related with it.

The following QoS can help you; max_instances.

aelhadee commented 2 years ago

Setting key to False resolved the issue, thanks a lot!