Open Noah-Jrh opened 1 year ago
Hi Noah, we are up to upload hd5 files for the digits, already splitter and ready to use. I'll back to you soon.
De: Noah @.> Enviado: lunes, 12 de junio de 2023 17:44 Para: RTC-research-group/LIPSFUS-Event-driven-dataset @.> Cc: Subscribed @.***> Asunto: [RTC-research-group/LIPSFUS-Event-driven-dataset] How to get the event data containing a single word? (Issue #2)
As far as I know, an AEDAT file was recorded for the whole presentation from the word ”one” to the ”fox” sentence [1]. I want to get the event data that contains a single word, which requires knowing the start and end points of each word in the AEDAT files. I carefully read the code about the data processing but I didn‘t ’find relevant information or code to achieve this purpose. The Matlab code under the NoisyRoom folder just splits the AEDAT file into video and audio data but does not split the words. The create_visual_frame_dataset.py under the scripts folder processes the AEDAT file containing a single word, because the file_name in the code already contains label information, but the code in MATLAB does not add the label information to the file name. I observed in show_dataset_examples.py that the AEDAT file name after word segmentation should be like Alejandro_0degrees_BCIroom_word_audio_Eight_th_4600_prets_100000_ts_30555117_32055117.aedat. However, I don't know what to do to get it, maybe I'm overlooking something.
Finally, obtaining event data containing a single word is important for the use of this dataset, and it would be helpful if the authors could provide relevant information or code.
[1] Rios-Navarro A, Piñero-Fuentes E, Canas-Moreno S, et al. LIPSFUS: A neuromorphic dataset for audio-visual sensory fusion of lip reading[J]. arXiv preprint arXiv:2304.01080, 2023.
— Reply to this email directly, view it on GitHubhttps://github.com/RTC-research-group/LIPSFUS-Event-driven-dataset/issues/2, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADWOVSX5XEOVEDZE5OJOWY3XK42NHANCNFSM6AAAAAAZDSKLWE. You are receiving this because you are subscribed to this thread.Message ID: @.***>
Thanks for your reply. I look forward to the new uploads.
Hi Noah, we have uploaded the .aedat for the digits. Each aedat corresponds to the audio / visual recording of a spoken digit. You can see how we took all the samples into two zip files (train and test). hdf5 files were constructed for a particular application (under test now) with some preprocessing to the aedat (ie. reducing the traffic with particular filter). We use the pynavis (https://github.com/jpdominguez/pyNAVIS) and tonic (https://github.com/neuromorphs/tonic) to create the needed hdf5 files. [https://opengraph.githubassets.com/7cba3e3d05c6f5b45027a37a5f07fe4e858526a48ba2464912858064d6f24efd/neuromorphs/tonic]https://github.com/neuromorphs/tonic GitHub - neuromorphs/tonic: Publicly available event datasets and transforms.https://github.com/neuromorphs/tonic Publicly available event datasets and transforms. Contribute to neuromorphs/tonic development by creating an account on GitHub. github.com
[https://repository-images.githubusercontent.com/149469035/ae565280-700d-11ea-8f2f-c9201e4e3d75]https://github.com/jpdominguez/pyNAVIS GitHub - jpdominguez/pyNAVIS: An open-source cross-platform package to analyze and post-process spiking information obtained from neuromorphic cochleashttps://github.com/jpdominguez/pyNAVIS An open-source cross-platform package to analyze and post-process spiking information obtained from neuromorphic cochleas - GitHub - jpdominguez/pyNAVIS: An open-source cross-platform package to an... github.com
Best.
De: Noah @.> Enviado: martes, 13 de junio de 2023 17:32 Para: RTC-research-group/LIPSFUS-Event-driven-dataset @.> Cc: Alejandro Linares-Barranco @.>; Comment @.> Asunto: Re: [RTC-research-group/LIPSFUS-Event-driven-dataset] How to get the event data containing a single word? (Issue #2)
Thanks for your reply. I look forward to the new uploads.
— Reply to this email directly, view it on GitHubhttps://github.com/RTC-research-group/LIPSFUS-Event-driven-dataset/issues/2#issuecomment-1589549521, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADWOVSTXAZADAKSBLU4TC5LXLCBYZANCNFSM6AAAAAAZDSKLWE. You are receiving this because you commented.Message ID: @.***>
Thanks for the uploaded .aedat files. However, I try to run the create_visual_frame_dataset.py and the create_audio_frame_dataset.py with the uploading .aedat files. I've encountered a new error ("ValueError: Addresses are not in range. Could be due to bad decoding") in the line spikes = Loaders.loadAEDAT(file_path, SETTINGS). I'm wondering if it's an error in the uploaded file, or some setting error in the code?
As far as I know, an AEDAT file was recorded for the whole presentation from the word ”one” to the ”fox” sentence [1]. I want to get the event data that contains a single word, which requires knowing the start and end points of each word in the AEDAT files. I carefully read the code about the data processing but I didn‘t ’find relevant information or code to achieve this purpose. The Matlab code under the NoisyRoom folder just splits the AEDAT file into video and audio data but does not split the words. The create_visual_frame_dataset.py under the scripts folder processes the AEDAT file containing a single word, because the file_name in the code already contains label information, but the code in MATLAB does not add the label information to the file name. I observed in show_dataset_examples.py that the AEDAT file name after word segmentation should be like Alejandro_0degrees_BCIroom_word_audio_Eight_th_4600_prets_100000_ts_30555117_32055117.aedat. However, I don't know what to do to get it, maybe I'm overlooking something.
Finally, obtaining event data containing a single word is important for the use of this dataset, and it would be helpful if the authors could provide relevant information or code.
[1] Rios-Navarro A, Piñero-Fuentes E, Canas-Moreno S, et al. LIPSFUS: A neuromorphic dataset for audio-visual sensory fusion of lip reading[J]. arXiv preprint arXiv:2304.01080, 2023.