SensorsINI / DHP19

Repository for the Dynamic Vision Sensor 3D Human Pose Dataset (DHP19).
MIT License
42 stars 10 forks source link

May I ask how you are viewing the aedat file? When I view them using AEViewer, the images from the 4 DVS cameras overlap on one screen and I can't see the movements. #24

Closed lsjws2008 closed 7 months ago

lsjws2008 commented 8 months ago

微信图片_20240327153505

tobidelbruck commented 8 months ago

Where did you get the file exactly? I can have a look to see if we can remember how in jAER the data is stored.

As I recall they are stored as 4 AEDAT files with one .idx file that lists the four files. Then you open the idx file that should open 4 AEViewers that should be synchronized.

lsjws2008 commented 8 months ago

Where did you get the file exactly? I can have a look to see if we can remember how in jAER the data is stored.

As I recall they are stored as 4 AEDAT files with one .idx file that lists the four files. Then you open the idx file that should open 4 AEViewers that should be synchronized.

The image shows the file "DVS_movies\S5\session3\mov1.aedat." I directly opened the file using AEViewer. Is this operation correct?

tobidelbruck commented 8 months ago

No, you should open the aeidx file, that will open four viewers for the 4 files. That is what I recall. Let me know if that doesn't work.

lsjws2008 commented 8 months ago

I get what you mean. However, in the files synchronized from Resilio Sync, none of the filenames contain "aeidx". 微信图片_20240328203457

tobidelbruck commented 8 months ago

hmm, then I'm not sure. You can try to make an .aeidx file. It just has one file per line. I will give it a try later, am trying to finish overdue report today for one of our projects.

tobidelbruck commented 8 months ago

BTW how are the files named, what is the camera index? There should be 4 DVS cameras per session

tobidelbruck commented 7 months ago

OK, I finally had a chance to look and I'm sorry to have misinformed you. The correct way to view the recordings in jAER is to use the AEChip MultiDAVIS346BCameraChip. With this AEChip, the 2 lsbs of the address are decoded to provide the X address offset. The clue is in the paper where it is stated:

  DVS data format. The dataset contains only DVS data
  from the four DAVIS cameras. We adapted the standard
  DVS data format to the multi-camera setup case. We
  merged the streams of DVS events from each camera to 
  ensure monotonic timestamp ordering, 
  and included the identification (ID) number of each 
  camera in the two least significant bits of the raw address.
   In this way, each event is
  represented by a tuple e = (x, y, t, p, c). Where (x, y) is
  the address in the pixel array, t is the time information in
  microsecond resolution, p is the polarity of the brightness
  change, and c is the camera ID number. This arrangement
  makes it much easier to process all the DVS data together
  in single data files.

Searching around in the jAER commit history for some time, I finally found the class that Gemma made called MultiDavisCameraChip. This base class is subclassed for MultiDAVIS346BCameraChip.

multicamerachip

Go to the menu AEChip/Customize AEChip menu and add the AEChip for DAVIS346 cameras. When you select this chip, enter 4 when asked for number of cameras.

You should then be able to play these recordings.

Here is a screenshot of playing one of them:

multicamerachip dhp19

Here is a YouTube video playing one DHP19 recording

tobidelbruck commented 7 months ago

Updated https://sites.google.com/view/dhp19/jaer to provide this information

lsjws2008 commented 7 months ago

Thank you very much for your reply. After upgrading jAER to version 2.6.1, I obtained the expected correct results.