search

uzh-rpg / rpg_esim

ESIM: an Open Event Camera Simulator
MIT License
586 stars 124 forks source link

using ESIM for object detection #27

Closed germal closed 5 years ago

germal commented 5 years ago

Hi Could be possible to use ESIM to process realtime video and use the events to make object recognition from drones performing trajectories at high speed ? Thanks ! Germal

supitalp commented 5 years ago

Hello,

Using an event camera to detect moving objects on drones is doable, yes (see here: http://rpg.ifi.uzh.ch/docs/RAL19_Falanga.pdf).

However, converting the frames from a standard camera to a virtual event stream (instead of using a real event camera) for this task does not seem like a good idea: you would essentially lose the advantages of event cameras since the frames would suffer from motion blur, under- over- exposure, etc.

While it would be possible to adapt ESIM to perform what you describe (see relevant simulation code here: https://github.com/uzh-rpg/rpg_esim/blob/master/event_camera_simulator/esim/src/event_simulator.cpp#L21), note that the simulated events would not be accurate since nothing guarantees that the optical flow between two frames in the video is small enough to yield accurate events.