hecomi / uMicrophoneWebGL

Enable microphone input buffer access in Unity WebGL builds
https://tips.hecomi.com/entry/2024/01/28/010109
MIT License
32 stars 4 forks source link

Recorded clip seems to be reproduced at half the proper pitch. #1

Open HunterProduction opened 8 months ago

HunterProduction commented 8 months ago

I'd like to thank you for this resource, but I'm facing a weird problem (even in the Recorder sample scene) where my audio clip is being reproduced "slowed down", and by tweeking the AudioSource parameters I managed to understand that if I double the pitch it gets back to its original pitch. Also, isn't there the possibility to simply override the Unity Microphone class so that the logic could be left equal? For example, is it possible to avoid updating the data every frame and retrieve the final AudioClip as a return of the Start method, and then use it when the record ends?

hecomi commented 8 months ago

Thank you for trying to use this library.

It sounds like you might be encountering an issue with the sample rate determination (there could be a bug where, for example, a 96000 Hz sampling rate is being treated as 48000 Hz on Mac). Could you please share the environment you're currently using?

Regarding your suggestion for compatible behavior with Unity's Microphone, I appreciate the feedback. There are indeed several existing libraries made by other developers that achieve this compatibility. However, aiming for compatibility tends to increase the latency in microphone buffer acquisition due to the need for communication with the JavaScript side. This library's policy focuses on real-time processing, which is why I've opted for a callback approach. For more details, please refer to my blog I've written on this subject (it's in Japanese, so you might need to use some translation service):

HunterProduction commented 8 months ago

Thank you very much for you clarifications.

Regarding my environment, I'm currently working on a windows virtual machine with a Xeon E5-2678 v3, 12GB of RAM and a GTX1080. Running your Recorder sample scene outputs me this log about the selected device:

Device[0]: External Microphone Headphone (Shadow Virtual Audio Device) (sampleRate: 48000, ID: External Microphone Headphone (Shadow Virtual Audio Device))

so the sample rate seems to be correct, but I don't know if the 96000Hz problem that you mentioned can be detected by this log or happens in a different moment of the execution. Another test I did, If I record a clip in which each second I count from 1 to 10, when I play it sounds at half the speed but it still lasts real 10 seconds, so it stops around when I ear a slowed "five". This means that the dimension of the buffer seems to be preserved with the maxDuration value, but still the sample rate seems to be compromised in some way.

EDIT: It seems like doubling the sample frequency in this way: var freq = microphoneWebGL.selectedDevice.sampleRate *2; both in OneEnd and OnBegin methods results in a normal record of maxDuration seconds. Do you have some ideas on why this happens?

EDIT 2: After another session of tests, I noticed that this problem happens only in the editor on my virtual machine (I couldn't test the project on another local machine). After building, doubling that freq variable breaks the creation of the AudioClip, so I tried to build without that correction and both on a desktop browser and a mobile browser it seems to work correctly. I'm leaving you all this reports just in case they could be useful for you to improve the tool.

Also, on the virtual machine browser the only mic detected results with no name, a value of 0 for the sample rate, and a long alphanumeric string for the ID. This could probably not depend on your implementation because it is a very specific platform situation, but anyway I'm telling you just in case it could be helpful!

hecomi commented 7 months ago

Thank you for providing the detailed information and I apologize for the delay in responding. I may have identified the issue; it seems that my current implementation might not support multi-channel microphones. I will conduct tests with a microphone that has more than two channels.

BioeJD commented 6 months ago

I came here to mention the same. The issue is that multiple channels aren't accounted for. I heard the same, set the channels to 2 for my particular mic, and it played back as expected.