search

RenderHeads / UnityPlugin-AVProMovieCapture

AVPro Movie Capture is a Unity Plugin for advanced video capture to AVI/MP4/MOV files
https://renderheads.com/products/avpro-movie-capture/
48 stars 8 forks source link

Drop in FPS when audio source set to manual or unity audio mixer #189

Open woodrowproctor opened 2 years ago

woodrowproctor commented 2 years ago

When I have audio recording enabled I get just about 2 seconds in the recording the frame rate drops to about one frame per two seconds. When audio is disabled I do not experience this performance dip. I am using a capture from camera component. I have tried Unity as well as Unity Audio Mixer as audio outputs.

Steps to reproduce the behavior:

  1. Run the application in the editor or a build
  2. Start recording realtime capture with audio
  3. Wait 2 seconds
  4. Watch framerate hit 0

No errors are shown in the console

woodrowproctor commented 2 years ago

I have tried the capture screen component and it has the same issue. If recording audio is enabled performance crashes to 0 every time

Chris-RH commented 2 years ago

Hi @woodrowproctor, Are you getting the same result with a new unity project just containing AVPro MovieCapture demos, without changing any options? Which audio codec are you using? Have you got a Windows 10 machine that you can test the exact same build on?

woodrowproctor commented 2 years ago

@Chris-RH I have not tried it in an empty project yet. I was mistaken in my original post, the machine I am on is actually Windows 10, sorry for the typo.

From what I can tell I am unable to change Audio codecs. Changing the order of them in the list always still results in AAC being used when the filming begins. Also setting it to use a codec with the manual slider doesn't change it. It always uses AAC.

Chris-RH commented 2 years ago

@woodrowproctor Yes, please could you try the demo scenes in a new project and see how they work for you :)

eduhalfen commented 2 years ago

I have the same problem. When I build for windows, there are many and many drop frames. I am using Capture from Camera and I capture Audio in Manual with a separated audio file.

eduhalfen commented 2 years ago

I forgot to say that in Play mode in MacOS (Editor) it is ok. The problem occurs when put Audio Capture in Manual. If it is Capture from Audio Listener is better.

Chris-RH commented 2 years ago

How are you pushing the audio into the plugin?

eduhalfen commented 2 years ago

The problem occurs when I use Capture Audio (From Audio Clip).

Chris-RH commented 2 years ago

So, both Manual and Unity Audio Mixer need specific data thrown at them, otherwise they just stall while waiting for it (hence FPS nosedive)

Unity Audio Mixer is intended for realtime capture and needs quite a bit of setting up from the user in order to get the data passing through it. It seems to be empty by default. (Unity Audio Mixer = https://www.renderheads.com/content/docs/AVProMovieCapture/articles/component-capture-audio-from-audiomixer.html )

Manual - CaptureAudio(FromAudioClip) is designed for offline capture mode (there is a note in the c#). You need to make sure that the AudioClip sample rate and channel count is the same as in the Capture component.

We could do with improving the error handling and relevant documentation. So its something that we will look at doing in the future.

eduhalfen commented 2 years ago

I needed too much to use Manual capture with Audio Clip in real-time mode because the audio in the final video it is not what the people is listening in the capture moment. In Mac OS it works very well but when build for Windows, the result is terrible.

woodrowproctor commented 2 years ago

Update on my end, I have discovered that some of my performance woes were coming from recording video + loading addressable content at the same time. We have a set of environments that are addressable prefabs. When I switch environments while recording it kills FPS. I have started disabling recording during the environment switches.

Chris-RH commented 2 years ago

Ahh, that's helpful to know. Thanks for that update

Kyoungsu-VIRNECT commented 1 year ago

Hi @Chris-RH, I have a similar problem.

In my CaptureFromCamera compnent case in Unity Editor, AudioSource:Unity / start capture -> pause -> stop capture -> start capture -> drop FPS Could you check this recording flow? (PC OS : Windows)

drprexl commented 10 months ago

I also have the same issue with another configuration. 

Unity version: 2022.3.13f
AVPro Movie Capture version: 5.1.8
Operating system version: Windows 10
Capture component used: Capture from Texture
Audio Capture component used: Capture Audio from AudioRenderer
Capture settings (resolution, frame-rate, codec): 1920x1080, 30fps, H264, I also have tried everything for audio

Steps to reproduce the behaviour:

Run the application in the editor or a Windows standalone build
Start recording capture with audio in offline renderer with manual update

With the same code, I do not have any issues under iOS. Please let me know if I need to correct a different configuration.

Edit: I also want to mention, that we have a video player component within the scene. When I deactivate the video player I do not have any issues... I only do not understand why the video player is influencing the audio recording. The audio output of the video player is set to direct.

drprexl commented 10 months ago

I also have the same issue with another configuration.

Unity version: 2022.3.13f
AVPro Movie Capture version: 5.1.8
Operating system version: Windows 10
Capture component used: Capture from Texture
Audio Capture component used: Capture Audio from AudioRenderer
Capture settings (resolution, frame-rate, codec): 1920x1080, 30fps, H264, I also have tried everything for audio

Steps to reproduce the behaviour:

Run the application in the editor or a Windows standalone build
Start recording capture with audio in offline renderer with manual update

With the same code, I do not have any issues under iOS. Please let me know if I need to correct a different configuration.

Edit: I also want to mention that we have a video player component within the scene. When I deactivate the video player, I do not have any issues... I need to understand why the video player is influencing the audio recording. The audio output of the video player is set to direct.

Do I get any support as a paying customer on this matter?

Chris-RH commented 10 months ago

@drprexl

  1. Does the finished video have FPS issues, or is it just while recording?
  2. Have you tried disabling manual update?
  3. Do you get the same result from capturefromscreen or capturefromcamera?
  4. Am I right in thinking that there are multiple audio sources? Have you tried a video without audio?
drprexl commented 10 months ago

@drprexl

1. Does the finished video have FPS issues, or is it just while recording?

The rendered video has no issue. It is only at the recording and rendering takes around 5 sec per frame

  1. Have you tried disabling manual update? I need the manual update to have a synchronous rendering in the video. I will test it and give feedback on this matter.
  2. Do you get the same result from capturefromscreen or capturefromcamera? I did not tested it, but I also do not need it from screen or camera. I will test it and give you feedback on this matter.
  3. Am I right in thinking that there are multiple audio sources? Have you tried a video without audio? A video without audio is no issue. I think the issue is, when you have a videoplayer and a audio render in the same scene.
Chris-RH commented 10 months ago

OK, well if the rendered video comes out as expected that is good. The reason why this is already labelled as an enhancement is so that we can look into improving how long it takes.