Ram usage is increasing continuously while running ecal python code

prakhar114 commented 1 month ago

Problem Description

When I try to run ecal webcam_grabber.py from ecal-camera-samples/ecal_camera_send_python , the ram keeps on increasing and then if i keep running it for longer time,ram gets filled up completely.

Ram usage before running the webcam_grabber.py

Ram usage after running webcam_grabber.py for 5 min

ecal monitor

Protobuff packet size is ~0.6MB

How to reproduce

clone this repo https://github.com/eclipse-ecal/ecal-camera-samples follow the python setup code in Readme connect a full hd(1920,1080) camera via usb (/dev/video0) run the webcam_grabber.py --compressed

Protobuff packet size is ~0.6MB

How did you get eCAL?

Download from Release Page

Environment

eCAL v5.12.5
OS Ubuntu 22.04.4 LTS
Python 3.10.12
OpenCV version 4.9.0

eCAL System Information

------------------------- SYSTEM ---------------------------------
Version                  : v5.12.5 (2024-04-24 14:43:08 +0200)
Platform                 : linux

------------------------- CONFIGURATION --------------------------
Default INI              : /etc/ecal/ecal.ini

------------------------- NETWORK --------------------------------
Host name                : car
Host group name          : car
Network mode             : local
Network ttl              : 2
Network sndbuf           : 5 MByte
Network rcvbuf           : 5 MByte
Multicast cfg version    : v1
Multicast group          : 239.0.0.1
Multicast mask           : 0.0.0.15
Multicast ports          : 14000 - 14010
Multicast join all IFs   : off
Bandwidth limit (udp)    : not limited

------------------------- TIME -----------------------------------
Synchronization realtime : "ecaltime-localtime"
Synchronization replay   : 
State                    :  synchronized 
Master / Slave           :  Master 
Status (Code)            : "everything is fine." (0)

------------------------- PUBLISHER LAYER DEFAULTS ---------------
Layer Mode INPROC        : off
Layer Mode SHM (ZEROCPY) : auto
Layer Mode TCP           : off
Layer Mode UDP MC        : auto

------------------------- SUBSCRIPTION LAYER DEFAULTS ------------
Layer Mode INPROC        : on
Layer Mode SHM           : on
Layer Mode TCP           : on
Layer Mode UDP MC        : on
Npcap UDP Reciever       : off

------------------------- EXPERIMENTAL ---------------------------
SHM Monitoring           : off
SHM Monitoring (Domain)  : ecal_mon
SHM Monitoring (Queue)   : 1024
Network Monitoring       : on
Drop out-of-order msgs   : off

KerstinKeller commented 1 month ago

Hi, thanks for reporting. First we need to analyse, where this RAM build-up is coming from. It could come from the sample code itself (unlikely, since it's pure Python), from the eCAL binding, from Protobuf, or from opencv which is used to read the webcam images.

We can check pure eCAL samples (e.g. person_snd) to see if that builds up memory. Maybe also the VideoCapture is building up data, and not freeing it.

prakhar114 commented 1 month ago

Hello, I tested the following:-

Ran cpp based person_snd(protobuff), with which there was no ram build up issue.
I closed opencv and tried sending array = np.random.randint(10, size=(1920,1080,3)) with the ecal_snd_python code(protobuff), but the ram buildup issue was still there.

Observation:-

When I am trying to send big proto files over ecal in python, I am facing the ram build up issue, but there is not such issue when i am using cpp to do the same.
Also I did an experiment where I converted the proto message to string before sending and then sent the string using simple eCAL string publisher, In which I noticed there was no ram build up issue.
I also noticed that when the size of protobuf packet is significantly small like np.random.randint(10, size=(10,10,3)), there is no ram build up issue.

prakhar114 commented 4 weeks ago

@Kerstin-Keller any possible solution/workaround to fix this issue?

eclipse-ecal / ecal