search

ros-event-camera / metavision_driver

driver for event based cameras using the MetaVision SDK (Prophesee and CenturyArk)
Apache License 2.0
42 stars 12 forks source link

The process of events #9

Closed mosfe closed 2 years ago

mosfe commented 2 years ago

The temporal of this camera is 1280*720，the number is much larger than the other event cameras that the process to these events is difficult. When I want to reconstruct images or execute classification tasks, the large number of events makes real time displaying difficult ,that is to say, the reponse delay is so large. Could you give some suggestions? Hope your answer !

berndpfrommer commented 2 years ago

We noticed this too: the new sensor produces a very large number of events. You can cut this down by using event rate control (I merged support for this into the driver) but that only goes so far. It turns out when the sensor is producing data at maximum rate, the ROS driver as it is right now cannot keep up with the incoming events. For this reason I am currently completely rewriting the driver to only process raw packets and don't decode any events. This way at least one is able to store the raw events in a ROS compatible fashion. However the raw event stream will then have to be decoded downstream, which will put additional load on the processing pipeline.

mosfe commented 2 years ago

Thanks for your reply! You said that the rewriting driver only process raw packets and do not decode events, does it mean that we can not get (x,y,p,t) from the ROS topic but only the raw data format? I have a suggestion that like the padding operation in the neuron networks, we only choose one pixel in an area, and the camera only fire the event of that pixel, which can achieve the goal of compression of camera size，and the chosen pixel can be dynamic adjusted，such as cycle by time in an area. What is your opinion on this idea，hope your answer，thanks

berndpfrommer commented 2 years ago

That's exactly what it means: you cannot get the (x,y,p,t) directly. We will provide a decoder though that the receiving node can call to obtain (x,y,p,t). The camera (and the ROS driver) already supports ROI (region of interest). I don't think it can be dynamically adjusted quickly. And doing custom algorithms for event rate compression inside the ROS driver is definitely not a good idea. That should be done outside the driver, best in a nodelet that takes the raw packets, decodes them, and does whatever fancy event rate control it deems best.

mosfe commented 2 years ago

Thanks to your answer！