Dataset

[wang149876]

Very interesting work and thank you for sharing. I downloaded the pre-generated events of the NTURain dataset you provided but found that all the images are black. Is it because of the upload error or my viewing method is wrong?

[sunshangquan]

Hi, @wang149876 Sorry for the late reply. The way to visualize event data cannot be trivially done since they are highly sparse. At a pixel position, there would be only several value changes. You could save them into .npy file to see that the images are not all zero.

To visualize it, you could make a threshold like 1 (for positive change) or -1 (for negative change) so that making the value greater than 1 be (255,0,0) of red color and the value less than -1 be (0,0,255) of blue color.

[wang149876]

Thank you very much for your reply, I can indeed get effective visualizations with the methods you provided. I still have a question about the event information, the theoretical polarity of the event information should be 1 or -1, but there is a situation of polarity accumulation in the generated results, is this because the frame rate of the ESIM simulator and the video frame rate are different? Does the proposed network limit the polarity value to 1 and -1 or retain the polarity for the input events?

[sunshangquan]

Hi @wang149876 , glad to solve your issue. Yes, you are right. The polarity is +1 or -1 in one single time step of event camera (in microseconds). However, the time step of RGB camera is in milliseconds. So there will be accumulation of event polarity between two RGB frames. The proposed method does not limit the accumulated polarity values to 1 or -1.

[wang149876]

Ok, thank you very much for your reply.

[wang149876]

Ok, thank you very much for your reply.