search

videokit-ai / videokit

Low-code, cross-platform media SDK for Unity Engine. Register at https://videokit.ai
https://videokit.ai
Apache License 2.0
107 stars 14 forks source link

Audio corrupt / weird behaviour #116

Closed marvpaul closed 5 months ago

marvpaul commented 6 months ago

When recording video + audio (AudioListener) it dosen't seem to work (0.0.18 alpha 7/8). On Mac editor it has a weird sound in the final recording, even though I play a harmonic meditation music in the scene. On iOS it's even worse, when I click record, I hear a weird sound directly on my phone and the audio which I currently play inside my app will be completely silent.

This is a sample project and when I record within the editor using this sample project, it results in those videos (no error log): https://github.com/marvpaul/videokitbug

https://github.com/natmlx/videokit/assets/10696948/c1719b15-11e4-4a6e-9efb-804bc59fbe5d

On iOS the final recording is without any audio. Also I got this error log

VideoKit: MP4Recorder prepared video encoder at resolution 720x1,558
VideoKit: MP4Recorder prepared audio encoder with format 2Hz 24,000ch
Main Thread Checker: UI API called on a background thread: -[UIView bounds]
PID: 1145, TID: 108321, Thread name: (none), Queue name: com.apple.root.default-qos.overcommit, QoS: 0
Backtrace:
4   UnityFramework                      0x000000010e22e89c UnityIsFullscreen + 172
5   UnityFramework                      0x000000010e887f50 _ZN14CrashReporting13CrashReporter24PopulateReportSharedDataERNS_11CrashReportEbb + 928
6   UnityFramework                      0x000000010e8879d8 _ZN14CrashReporting13CrashReporter9HandleLogERK21DebugStringToFileData7LogTypeb + 1140
7   UnityFramework                      0x000000010eb4f5b8 _ZN14CallbackArray3IRK21DebugStringToFileData7LogTypebE6InvokeES2_S3_b + 112
8   UnityFramework                      0x000000010eb4f064 _Z40DebugStringToFilePostprocessedStacktraceRK21DebugStringToFileData + 228
9   UnityFramework                      0x000000010e6c2f28 _ZN9Scripting23LogExceptionFromManagedE21ScriptingExceptionPtriPKcbPKNS_30LogExceptionFromMangedSettingsE + 520
10  UnityFramework                      0x000000010e6c1db0 _ZN9Scripting12LogExceptionE21ScriptingExceptionPtriPKcb + 132
11  UnityFramework                      0x000000010e6c1394 _ZN19ScriptingInvocation6InvokeEP21ScriptingExceptionPtrb + 172
12  UnityFramework                      0x000000010e834eb4 _ZN17AudioCustomFilter12readCallbackEP14FMOD_DSP_STATEPfS2_jii + 324
13  UnityFramework                      0x000000010f0711c0 _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 1348
14  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
15  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
16  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
17  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
18  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
19  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
20  UnityFramework                      0x000000010f080dec _ZN4FMOD12DSPSoundCard4readEPvPj16FMOD_SPEAKERMODEij + 132
21  UnityFramework                      0x000000010f08e944 _ZN4FMOD6Output3mixEPvj + 648
22  UnityFramework                      0x000000010f0905ac _ZN4FMOD15OutputCoreAudio11updateMixerEv + 92
23  UnityFramework                      0x000000010f0a9c58 _ZN4FMOD6Thread8callbackEPv + 180
24  libsystem_pthread.dylib             0x00000001f6c8da90 _pthread_start + 136
25  libsystem_pthread.dylib             0x00000001f6c8cfcc thread_start + 8
=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView bounds]
PID: 1145, TID: 108321, Thread name: (none), Queue name: com.apple.root.default-qos.overcommit, QoS: 0
Backtrace:
4   UnityFramework                      0x000000010e22e89c UnityIsFullscreen + 172
5   UnityFramework                      0x000000010e887f50 _ZN14CrashReporting13CrashReporter24PopulateReportSharedDataERNS_11CrashReportEbb + 928
6   UnityFramework                      0x000000010e8879d8 _ZN14CrashReporting13CrashReporter9HandleLogERK21DebugStringToFileData7LogTypeb + 1140
7   UnityFramework                      0x000000010eb4f5b8 _ZN14CallbackArray3IRK21DebugStringToFileData7LogTypebE6InvokeES2_S3_b + 112
8   UnityFramework                      0x000000010eb4f064 _Z40DebugStringToFilePostprocessedStacktraceRK21DebugStringToFileData + 228
9   UnityFramework                      0x000000010e6c2f28 _ZN9Scripting23LogExceptionFromManagedE21ScriptingExceptionPtriPKcbPKNS_30LogExceptionFromMangedSettingsE + 520
10  UnityFramework                      0x000000010e6c1db0 _ZN9Scripting12LogExceptionE21ScriptingExceptionPtriPKcb + 132
11  UnityFramework                      0x000000010e6c1394 _ZN19ScriptingInvocation6InvokeEP21ScriptingExceptionPtrb + 172
12  UnityFramework                      0x000000010e834eb4 _ZN17AudioCustomFilter12readCallbackEP14FMOD_DSP_STATEPfS2_jii + 324
13  UnityFramework                      0x000000010f0711c0 _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 1348
14  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
15  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
16  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
17  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
18  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
19  UnityFramework                      0x000000010f070dac _ZN4FMOD9DSPFilter4readEPPfPiPj16FMOD_SPEAKERMODEij + 304
20  UnityFramework                      0x000000010f080dec _ZN4FMOD12DSPSoundCard4readEPvPj16FMOD_SPEAKERMODEij + 132
21  UnityFramework                      0x000000010f08e944 _ZN4FMOD6Output3mixEPvj + 648
22  UnityFramework                      0x000000010f0905ac _ZN4FMOD15OutputCoreAudio11updateMixerEv + 92
23  UnityFramework                      0x000000010f0a9c58 _ZN4FMOD6Thread8callbackEPv + 180
24  libsystem_pthread.dylib             0x00000001f6c8da90 _pthread_start + 136
25  libsystem_pthread.dylib             0x00000001f6c8cfcc thread_start + 8
Uploading Crash Report
ArgumentException: Cannot append audio buffer with sample rate {audioBuffer.sampleRate}Hz to recorder with sample rate {sampleRate}Hz
Parameter name: audioBuffer
  at VideoKit.MediaRecorder.Append (VideoKit.AudioBuffer audioBuffer) [0x00000] in <00000000000000000000000000000000>:0 
  at VideoKit.Sources.SceneAudioSource+<>c__DisplayClass10_0.<.ctor>b__0 (System.Single[] data, System.Int32 channels) [0x00000] in <00000000000000000000000000000000>:0 

ArgumentException: Cannot append audio buffer with sample rate {audioBuffer.sampleRate}Hz to recorder with sample rate {sampleRate}Hz
Parameter name: audioBuffer
  at VideoKit.MediaRecorder.Append (VideoKit.AudioBuffer audioBuffer) [0x00000] in <00000000000000000000000000000000>:0 
  at VideoKit.Sources.SceneAudioSource+<>c__DisplayClass10_0.<.ctor>b__0 (System.Single[] data, System.Int32 channels) [0x00000] in <00000000000000000000000000000000>:0 

ArgumentException: Cannot append audio buffer with sample rate {audioBuffer.sampleRate}Hz to recorder with sample rate {sampleRate}Hz
Parameter name: audioBuffer
  at VideoKit.MediaRecorder.Append (VideoKit.AudioBuffer audioBuffer) [0x00000] in <00000000000000000000000000000000>:0 
  at VideoKit.Sources.SceneAudioSource+<>c__DisplayClass10_0.<.ctor>b__0 (System.Single[] data, System.Int32 channels) [0x00000] in <00000000000000000000000000000000>:0 

ArgumentException: Cannot append audio buffer with sample rate {audioBuffer.sampleRate}Hz to recorder with sample rate {sampleRate}Hz
Parameter name: audioBuffer
  at VideoKit.MediaRecorder.Append (VideoKit.AudioBuffer audioBuffer) [0x00000] in <00000000000000000000000000000000>:0 
  at VideoKit.Sources.SceneAudioSource+<>c__DisplayClass10_0.<.ctor>b__0 (System.Single[] data, System.Int32 channels) [0x00000] in <00000000000000000000000000000000>:0
marvpaul commented 6 months ago

@lanreolokoba I tested with your 0.0.18alpha13 again on Android and now I get the following error when I start recording through VideoKitRecorder component :/ 05-22 15:56:52.637 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:52.637 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:52.637 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:52.637 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:52.637 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:53.008 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:53.008 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:53.008 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:53.008 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:53.008 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:53.295 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:53.295 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:53.295 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:53.295 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:53.295 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:53.516 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:53.516 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:53.516 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:53.516 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:53.516 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:53.866 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:53.866 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:53.866 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:53.866 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:53.866 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:54.180 16597 16621 E Unity : NullReferenceException: Object reference not set to an instance of an object 05-22 15:56:54.180 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CreateSessionToken (System.String buildToken) [0x0017f] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:146 05-22 15:56:54.180 16597 16621 E Unity : at VideoKit.Internal.VideoKitClient.CheckSession () [0x00046] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Internal/VideoKitClient.cs:61 05-22 15:56:54.180 16597 16621 E Unity : at VideoKit.MediaRecorder.Create (VideoKit.MediaRecorder+Format format, System.Int32 width, System.Int32 height, System.Single frameRate, System.Int32 sampleRate, System.Int32 channelCount, System.Int32 videoBitRate, System.Int32 keyframeInterval, System.Single compressionQuality, System.Int32 audioBitRate, System.String prefix) [0x0001a] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/MediaRecorder.cs:176 05-22 15:56:54.180 16597 16621 E Unity : at VideoKit.VideoKitRecorder.StartRecording () [0x0024b] in ./Library/PackageCache/ai.natml.videokit@0.0.18-alpha.13/Runtime/Components/VideoKitRecorder.c 05-22 15:56:57.641 1245 3584 I ActivityManager: PendingStartActivityUids startActivity to updateOomAdj delay:23392ms, uid:10368

olokobayusuf commented 6 months ago

@marvpaul does your app have internet access? Seems like a runtime session token couldn't be created.

olokobayusuf commented 5 months ago

Audio recording bugs have been fixed in the latest alpha builds. I'm closing this issue pending the 0.0.18 update that will go out either later today or tomorrow. Feel free to open a new issue if you face any new bugs around audio recording.