Closed freemanfromgodotengine closed 7 years ago
The sound engine will be rewritten for 3.0 I think
@timoschwarzer I afraid it will not. That was a plan some time ago, but with recent decisions to skip 2.2 and go for 3.0, @reduz aims for the renderer rewrite for 3.0 and not audio rewrite now, but it would be good to hear it from him personally. I expect it will be done later, but I did the tests and decided to describe audio problems anyway, so we can have reference how better Godot will be in the future.
285ms? Are you sure it's that long? I don't really notice this I think... But obviously it shouldn't be like that, jesus :P
Also, I could be talking about of my ass but is this related to the audio's actual api, or could it be just a signal lag from ui events -> play audio? (input lag. like minuscule)
@freemanfromgodotengine you mentioned that using the low level audio AudioServer can improve this. How exactly can AudioServer be used, and how much of an improvement does it make?
I did some additional tests. This time measuring AudioServer's performance. Frankly, I expected much better results. The best result of AudioServer's latency I was able to get was 235 ms.
I know it's 50ms less then results of SamplePlayer test I did, but from my point of view that is still bad performance.
Wow, this is big. I always thought it was my sound files that were distorted. I usually open them up in Audacity and try to shave off the first part of the sound so it would play faster.... I guess this explains the lag. F*ck
@beamer159 By using AudioServer I meant using this: http://docs.godotengine.org/en/stable/classes/class_audioserver.html can create voices and samples, and play them. It's also able to change in time parameters of played voices, for example volume, in a much smoother way. Like I explained in post above, it tends to have less latency than SamplePlayer.
@Dillybob92 I checked my editing. OK, there was 0.007 ms silence at the beginning of the file in the 1st test I did - the one with SamplePlayer, so it would give us 278 ms instead 285 ms. :) 2nd test - AudioServer test - audio had no silence at the beginning, so 235 ms stands.
How are you measuring this? Have you tried to bypass GDScript (by connecting the signal directly to the play
function of the SamplePlayer)?
@vnen I do it in the most simple and reliable way - "analog". My test looks like this: I run audio playing Godot project on my computer. I set external high quality audio recording device, that, using microphones, records everything that comes out of computer speakers. I bring computer mouse close the microphone and speaker. Mouse has audible click, that microphone is able to record. I start to record the sounds and I click play" button in Godot project. Godot plays the sound, but my mouse click is also recorded by the microphone. I copy audio from audio recording device to the computer. I open audio file in audio editor and look at the time pass between recorded mouse click and sound played by Godot in the timeline (most audio editors provide it and are able to show the time in ms or samples). That's is.
What signal and how would I directly connect "play" function to it?
@vnen I did another simple test, just in Godot editor. I went to SamplePlayer inspector and recorded clicking in UI on "play" choosing the sample. Time measured between releasing the mouse button and sound being played was more or less similar to value of 285 ms. So this delay is no created by code. Anyway my test project has nothing more then just TouchScreenButtons mechanism and playing sounds.
@freemanfromgodotengine on your OP you mentioned func _on_TouchScreenButton_pressed()
which I assume is a signal connected to that function. With the dialog to connect signal you can select any node and type any function name, so you can select play
function from the SamplePlayer node, without any script attached to it.
But since the sample player in editor does the same, I guess there's overhead from GDScript there.
So if I understand correctly, ETA for sound improvements is >6 month? (Considering that 3.0 will come out at late winter/early spring and nobody will care about sound for some time). I wanted to play around with timed inputs in my game, I guess that mechanic will have to wait until this is resolved.
OK, I guess it's a test day today :) so I decided to do one more test to check if problem might be related to operating with UI - things like clicking the button with a mouse, or touching the screen of mobile device.
In the test the sound was played on colliding two objects. There was no input from me, just simple code
func _on_RigidBody2D_body_enter( body ):
get_node("SamplePlayer").play("sample")
and godot's physics. It was a Rigidbody2d (character mode) bouncing on Staticbody. I run the test and recorded computer's screen with my camera 30fps. Then I checked the movie frame by frame. There was a difference of approx. 6.5 frames between moment of visual contact of the objects and the sound played. Assuming that 1 frame of video recorded at 30 fps is taken every 0.0333333 of a sec., 6.5 frame delay would give approx. 217 ms . It's better than 285 ms when UI interactions were taking place, but still confirms, more or less, rather noticeable lag.
As I never noticed this delay as a problem, i've made some tests too under Windows 8.1, and compared Godot delay to the delay I'm used to experience in the Left4Dead2 video game.
So I recorded the sound (using a camcorder) near my laptop, then I measured the delay between the sound of the mouse click and the sound emitted by the computer using Audacity.
Under windows 8.1, with a Godot 2.0.4.1 project and default audio settings, I get a delay of approximately 200ms.
With Left4Dead2 into the UI, the delay between the mouse click and the sound of the UI is of around 100ms. In game, the delay between the mouse click and the sound of the gun is of around 250ms.
around 250ms is what i measured for Left4Dead2 which uses Valve's Source Engine, and for which around 250ms is not ruining the game immersion, IMO.
The speed of sound in real life, it's 340 m/s. so, we can hear 250ms delayed sound if something happens about 85 meters out. Can we get this reality with new audio engine? :)
Here are some possible workarounds, for those feeling adventurous and not willing to wait till this is fixed:
.wav
sample, since many samples have. (probably you already tried it tho)I don't know for the rest of all of you, but when i play the plateformer demo or the simple shooter demo, my gaming experience is okay despite the "200ms" delay ...
@GodotIsAMess : could you upload a minimal project that shows when this delay is an issue ?
Here is an example project similar to DDR. Use WASD for up, down, left, right. When the player hits an arrow, it plays a sound effect. Ideally, the sound would play immediately when the arrow is hit, but the delay makes this type of game unplayable.
In Tap.gd, look at lines 87 and 119. These lines play the sound effect. Right now 87 is commented out. If you uncomment this line, the sound will play when it is supposed to (without any input from the player)
For comparison, look at Osu's Taiko game mode:
https://www.youtube.com/watch?v=kthG-0Un6fA
From experience, I can say that there is essentially no noticeable delay between hitting the key and hearing the sound effect.
@beamer159 : I made a similar demo/game with a early version of Godot 1, 2 years ago, and this delay did not appeared as a problem because, first my game was slower than yours, and secondly because I adapted my logic to workaround the delay. Even at the speed and precision level required by your game, if you consider the delay is a constant, it should be all about math, IMO.
I did consider the delay as a constant. That is how I got the sound to play when I wanted at line 87, by playing the sound 200 ms before I wanted it to sound. However, I can't know when the player is going to press the button, so I can't start playing the sound before they press the button.
The problem still stands. I have tried a number of Godot-created games, and in every one, if a sound plays as a result of a player input, this ~200 ms delay exists.
indeed it's not about guessing when the player is going to press the button. It would be more about making the player synchronizes his reflexes on what happens on the screen ... There would still be a delay between the user inputs and the sound, but the music and the sounds could still be synchronized.
And if the player wants to synchronizes his reflexes on the music, with some training, his/her brain should just perfectly anticipate the delay ... after all, when a drummer/percussionist wants to synchronize with the rest of the music (like in a cover for instance), he/she has to move his hands and arms before the sound is actually emitted (not just moving his fingers), and his/her brain perfectly anticipates this larger delay ...
I'm not saying that the 200 to 250ms delay should/could not be reduced, nor that it does not pose some little issue/difficulties, but I'm just still not convinced that it is a real handicap that deserve the gigantic warning blinking neon sign that our dear @GodotIsAMess would like to put on the homepage, nor that there is no workaround or tiny compromises possible to keep working on a "6 month long project" ... IMO.
@GodotIsAMess How about having a blind test with your friends to see if they notice the delay if you have to release soon.
If you think you can fix this then let's have this conversation in your pull request thread, not here.
On 28 November 2016 at 21:50, GodotIsAMess notifications@github.com wrote:
@SuperUserNameMan https://github.com/SuperUserNameMan Well, I was just saying that because I don't think it's fair for people who have spent 6 + months (like myself) only to finally realize about this audio bug and now it's preventing me from publishing the game. Which is very big. It's really depressing to be honest, but it's the risk you take for using open source projects I guess. I just don't understand how the main developers here don't view it as a huge problem but will put off fixing it for Q1-Q2 of next year. That makes absolutely no sense to me, but I guess I am the special snowflake here and it only affects my project only. (/s)
Obviously a neon sign on the front page would be dumb, but just saying that's how I feel personally. Even I were to fix the bug, akien wouldn't even merge it because he's so delusionally picky about whitespaces. I would spend more time fixing the damn whitespace in code than actually fixing the bug in C/C++ code.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6965#issuecomment-263442758, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPV5UnxXiAf7-Ln3gEhRyMJ10oE4-ks5rC3a6gaJpZM4KkJRU .
Wow, I did not create this issue to make havoc nor bash devs :) I would like to thank you all for confirming my finds. I also assume reading the comments, that the issue is relevant for other Godot users.
@SuperUserNameMan When such latency may not interfere with your gameplay in some different genre of games, it will be noticeable or even ruining the experience in others, therefore if we want Godot to be more versatile game engine, the issue should be resolved as much as possible.
I am sorry, but I have to say, that you had no idea what you were talking about when you described brain anticipating the larger delay. It's exactly opposite. Brains know very well where things should be in time and space and latency even as small as 25 ms is noticed in the sound anticipating cognitive processes. If two sounds will be played with such delay apart, your brain will clearly be able to recognize them as two separate audio events. Latency below that (12 ms) gives just a feeling, that can't be described as separate audio events, but brains "feel" that something is not right. So, brain compensating for bigger delays can only be described as less and less comfortable task, that probably ends with completely ruining the game. How ofter did you hear online FPS players with ping 250 ms that were angry and swearing after being killed or not able to headshot enemies because of 250 ms game delay?
Maybe itself it's not yet an issue worth of a gigantic red neon sign saying that "Godot sucks" (because it clearly doesn't), but Godot's audio inefficiencies are evident, as I was trying to state by posting this issue as well as #6963 and #6964
@GodotIsAMess I am glad you see the latency problem as well. It does not make me standing alone here. However I would constrain myself from blaming Godot devs for your fault of not taking care of choosing the right tool for creating your game. If someone spends building something 6 months with the tool he did not check, it's his own fault if the tool is not the right for the job. If you say that it's not fair for people who have spent 6 + months only to finally realize about this audio bug, I would say, that Godot's dev's will be glad to fully refund your Godot purchase. ;)
@bojidar-bg about your list of possible workarounds.
@volzhs they will notice the delay, unless the game will have a strange, visually misleading mechanics or something else, that would be able visually justify the reaction latency. It's possible, but needs creativity in game design :)
@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc.. After all GDScript was created to be simpler and easier to learn. It's logical, that Godot will attract many users who were not able to learn other, more complicated and difficult languages, frameworks, game engines. Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.
@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc.. [...] Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.
( for the sake of minding others' business and of subtly making the unobvious more obvious, i just would like to point out that the GodotIsAMess' message to which Punto replied by mail is not exactly the same as the one currently displayed in github .... In the initial version, there was a second paragraph :
Even I were to fix the bug, akien wouldn't even merge it because he's so delusionally picky about whitespaces. I would spend more time fixing the damn whitespace in code than actually fixing the bug in C/C++ code.
.... )
@freemanfromgodotengine Well, our response about "go fix it yourself" was result from @GodotIsAMess's ramble on another (totally unrelated) issue. As we told him multiple times, this is planned to be fixed when the audio engine is rewritten (or just refactored) in 3.0/3.1.
Till then, we can't really suggest something else to people whose projects are blocked by this, as devs are giving away their free time on the issues they freely choose from -- and if they think something is more important than this, we can't really blame anyone.
Anyway, if you feel adventurous (not forcing anyone here), just go in there, find every C++ function which is called, print_line
the millisecond time in all of them, and identify the delay somewhere in the pipeline -- this would be really helpful for a swift fix. :smiley:
@freemanfromgodotengine I don't expect anything from anyone, but my advice is that if you want to be a successful game developer of course you need to learn c++, especially when nobody can fix your issue after complaining so loudly about it. Even AAA studios with millions of dollars in budgets prefer to have in house people that can fix their engine issues, instead of depending on a 3rd party to solve their problems. We do our best, and as I said on the other thread I agree that this is a problem, and ideally I'd like to see it fixed, but we can't be responsible for your game. Especially if he brags about being able to fix it, why are we having this discussion?
On 5 December 2016 at 12:13, Rémi Verschelde notifications@github.com wrote:
Remove above comments that went a bit far from the initial topic (+ edited @SuperUserNameMan https://github.com/SuperUserNameMan's answer to add the proper quote that makes it more understandable).
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6965#issuecomment-264879367, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPSZLGd2X2V1AWKq2QESnVVpCpatCks5rFCoSgaJpZM4KkJRU .
Okay, so I dug around a bit, and the audio driver buffer size is based on audio/output_latency
in the project settings. By default it'll be 25ms @ 44100Hz, giving 1102.5 buffer size but because it's rounded to the next power of two, it'll actually be a 2048 size buffer or 46.4ms latency. What about the audio mixer? Same principle with audio/mixer_latency
, but this time 10ms is actually 11.5ms. Not too bad.
All in all, buffer size alone, you're talking about 57.9ms latency.
Now comes the interesting part. In main/main.cpp
I saw that the main thread operates by processing Input -> Rendering -> Audio Mixer -> Script
. That means, from mouse click to actual output, it has to go Input -> [Rendering] -> [Audio] -> Script -> [Input] -> [Rendering] -> Audio
. Two whole frames. At say, 60fps, that's 33ms. At the captured video 30fps it's worse, at 66ms.
EDIT: Digging a bit more and maybe I might be wrong here, but it should at least be 16-33ms.
Now, because the audio driver is updated in a separate thread you'll get a vsync equivalent, giving an overall total of 33 + 58 + (+46.4 / -0) = 114.2 ± 23.2 ms
.
So yeah, quite a bit of wastage that can be shaved off with some tweaking.
The other 100ms I'd say is down to OS latency in the audio system (~10-50ms unless you have a realtime kernel patch), input (wireless mouse?) and stuff I can't account for without actually touching the code, which I might get around to at some point.
Just tested it myself, using my phone as a mic. For control, my old laptop has a 400ms (!! it's old though) delay. Reducing the audio/output_latency
to 20ms reduced it to 350ms. Seems like that variable isn't working 1-to-1 to actual latency. Possibly a race condition somewhere?
EDIT: Arrrgghhh, I realised I forgot to set click-on-press. No, and with repeat testing it seems to only be shaving off 5-10ms
Nice debugging work, @leezh! I guess you are from the "feeling adventurous" group, no? :smiley:
Haha, yeah. I have very creative yet oddly productive ways of procrastinating actual game making. XD
Anyways, I found that doing one or two tests wasn't enough so I recorded multiple tests :P (n=8
):
25ms: x̄=280ms σ=26ms
20ms: x̄=244ms σ=30ms
Seems like my earlier prediction was right! That said, it's only a small improvement in the grand scheme of things.
Now, before you go about reducing the output_latency to something ridiculously small, I should warn that if it's too low you'd get lots of popping sounds during playback if the CPU can't keep up.
Nice experiments @leezh
I played with audio buffer settings to. Like you, I noticed, that it's able to set the latency back only a little and can cause other sound related problems, especially on mobile.
That's awesome (I guess I was wrong when I said it'd involve seriously breaking the AudioServer, only the main loop :p ). Does it make any difference if you play the voice from the _input callback instead of _process? I don't remember if the dispatch order of those events is assured to be the same across all platforms..
On 6 December 2016 at 11:21, Freeman notifications@github.com wrote:
Nice experiments @leezh https://github.com/leezh I played with audio buffer settings to. Like you, I noticed, that it's able to set the latency back only a little and can cause other sound related problems, especially on mobile.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6965#issuecomment-265161020, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPTo4CaZwMGXi-CUgo4PWRafwAatDks5rFW9cgaJpZM4KkJRU .
@punto- Some time ago @vnen was asking about
"Have you tried to bypass GDScript (by connecting the signal directly to the play function of the SamplePlayer)?"
so I
"I went to SamplePlayer inspector and recorded clicking in UI on "play" choosing the sample. Time measured between releasing the mouse button and sound being played was more or less similar to value of 285 ms. So this delay is no created by code. "
I guess if such latency exists in the editor itself when I click on the SamplePlayer node to play the sample, it is not related to the _input nor _process, right?
@freemanfromgodotengine The editor is built using the same GUI tools provided, so it definitely would be related to _input()
@punto- So, yeah, after finally digging around more I am starting to better understand how the main loop is set up. In platform/x11/os_x11.cpp
the input is polled and _input()
dispatched right at the start of the frame. Only then does it go through the regular stuff in main/main.cpp
. In other words, it's Input -> _input() -> Rendering -> Audio
. That's one frame of delay. Moving AudioServer::update()
to before VisualServer::update()
should improve things.
Also, to take into account OS latency that can't be dealt with, I did some research and found that they're typically around 20ms
total. Heck, I even tried using aplay
on the command line with a tiny wav file, and got only a 50ms
delay.
In other words, we're actually still looking at a fairly large chunk of latency that we don't know the cause of. :/
@leezh Again, I'd guess it is some thread-locking and syncing that's bottlenecking here... But that would be so only if AudioServer is multithreaded (might be wrapped in a thread as well).
Also, I'd guess, we should test directly dispatching AudioServer from GDScript in _input. That way, we can be sure if the latency is from the sampleplayer or somethere else :smile:
btw, how long does it takes to Android (and also to other OS maybe) to tell the difference between a simple touch/press/click, a double tap/click, a swipe, a drag and drop, and a middle finger triple backflip click, etc ?
edit : nevermind, Android will send onTouch events when they occurs, and will detect gesture in parallel, so its not blocking.
So I am very annoyed by this problem and messed around with it today. I measured the times at some points by printing out the result from OS::get_ticks_msec
. By doing this, I found out that there are two bottlenecks:
The time that passes between the call of AudioServer::voice_play()
(which happens almost immediately after receiving the triggering _input event) and the queued CMD_PLAY command getting processed within AudioServerSW::driver_process_chunk()
is about 10-50 ms. Which is the less significant amount of the whole 200-300 ms delay.
After tracking it down to the lowest level, where the driver is being handled, I came to the comclusion that the problem must be there. And I was right.
I am using Linux with the PulseAudio driver. After verifying that AudioDriverPulseAudio is actually being used, I played around with that file a bit, finding out that there is function which returns the latency from PulseAudio. Printing it out showed a latency as high as like 200 ms, just as big as expected. Reading about the PulseAudio API, I played around with the initialization parameters, especially with the buffering attributes pa_buffer_attr
, which currently are set to NULL for default initialization.
I then set the pa_buffer_attr::tlength
to a low value, compiled, and... The delay was gone! So the problem is a too large set buffer size. On the project page of PulseAudio (http://0pointer.de/lennart/projects/pulseaudio/doxygen/structpa__buffer__attr.html) it says that
The server tries to assure that at least tlength bytes are always available in the per-stream server-side playback buffer. It is recommended to set this to (uint32_t) -1, which will initialize this to a value that is deemed sensible by the server. However, this value will default to something like 2s, i.e. for applications that have specific latency requirements this value should be set to the maximum latency that the application can deal with. [...]
So it seems that the server sets that to a very high value by default, which causes the high latency. Using the global settings value "audio/output_latency" for the tlength setting (which is being loaded in that file anyway), I managed to fix this problem within a few lines of code. The buffer size does now depend on this setting, which is by default 25 ms (which is totally fine).
So I made a pull request, but also decided to put it up here for discussion. While I was able to fix this, I didn't really know what I was doing (which is probably normal when you are trying to fix bugs in a huge unkown code base). And it would fix the problem only for pulseaudio. I will take a look into ALSA and maybe the windows driver too later, but I definitely can't look into the iOS, Android and Mac drivers (since I can't test it). It is a bit weird anyway that this problem occurs with all of the audio drivers which are independently form each other. So what do you think about this?
Edit: pull request is #7425
@lonesurvivor Yay! Finally someone adventurous enough to want to do this! :+1:
Seems like ALSA already does most of the buffer thing: https://github.com/godotengine/godot/blob/master/drivers/alsa/audio_driver_alsa.cpp#L99 What is weirder is that RTAudio also seems to be doing it, though it still suffers from the bug :open_mouth: https://github.com/godotengine/godot/blob/master/drivers/rtaudio/audio_driver_rtaudio.cpp#L138
The difference with ALSA is that the output_latency setting which is used there has actually an effect on the delay (unlike with PulseAudio). If you set it to like 10 ms or less, the delay is nearly gone. But if you set it to 50, it is already like half a second. With 100 it is even more, nearly a second ( all measured by hearing). So it seems that there is a multiplicative factor somewhere in there. I will take a look at this tomorrow.
@bojidar-bg I've done a quick search on RTAudio and the documentation at https://www.music.mcgill.ca/~gary/rtaudio/classRtAudio.html#a6907539d2527775df778ebce32ef1e3b says that the buffer is measured in sample frames. I don't know if those are milliseconds but I think they aren't.
Same for ALSA: http://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m___h_w___params.html#ga9162045265f283c532634506456cab09
PulseAudio has the tlength
field, which is temporal.
@timoschwarzer and @lonesurvivor :
I don't remember all the things i tried, but just to share a little of my last month experiments about RtAudio with you, i tried various settings for RtAudio::StreamOptions options;
without noticing any improvements.
I think that options.flags
should at least be set to this :
options.flags = RTAUDIO_MINIMIZE_LATENCY | RTAUDIO_SCHEDULE_REALTIME;
Reducing unsigned int buffer_size
tends to lead to buffer underflows, and increasing the number of options.numberOfBuffers = 16;
did not improved things neither.
I hope you'll find the solution.
I tested on Windows 8.1.
edit : also, I measured the latency between the hardware click-sound of my USB wireless mouse, and the sound emited by my computer. So, my latency equation was something like this : mouse wireless latency + receiver wireless latency +USB latency + Godot latency + Rtaudio lantency + audio driver latency + hardware latency
@bojidar-bg ALSA is fixed as well now(at least it works as it should for me now). It turned out that the buffer size wasn't set at all, instead the period size was set by the output_latency setting (in the line you linked above). Added it to the pull request.
@lonesurvivor Nice progress so far, do you think you can handle RTAudio as well? :smiley_cat:
I will at least take a look at it. At the moment compiling with support for RTAudio on Linux is failing, need to get that going first.
Operating system or device - Godot version: Godot 2.1-stable on Xubuntu 14.10 64-bit. Project also tested on Android 4 to 6.
Issue description (what happened, and what was expected): Delay between action on the screen (for example clicking the button) and hearing the sounds, that is played by pressing the button with something as simple as
is terribly long. From my experiments it's around 285 ms.
Latency like that basically prevents people (who don't know how to use low level audio AudioServer), from creating interactive audio games.