Open lixinghe1999 opened 1 month ago
Hi @lixinghe1999 , our model is mainly trained on natural sound like bird chirping, dog barking and train passing, so it is hard to distinguish human speech. Here are two solutions to enhance it:
Thank you for your rapid reply. However, it still outputs meaningless results for other sounds, like musical instrument sounds. Can you give me some hints to solve it? I believe it is not necessary to retrain
Does it possible for the audio duration? Since the IMU duration is fixed to 2 seconds, I also fix the audio duration to 2 seconds
It may also be related to the sampling length. We sample 1024 frames in total. https://github.com/csuhan/OneLLM/blob/913638c0d385ff706aaed945ec87ee42bab4debb/data/data_utils.py#L81-L86
I slightly modify the eval code of audio to run on my dataset, however, the outputs are vague even the audio is speech. There are all like the blow ones:
I attach my code below