search

phaserjs / phaser

Phaser is a fun, free and fast 2D game framework for making HTML5 games for desktop and mobile web browsers, supporting Canvas and WebGL rendering.
https://phaser.io
MIT License
36.74k stars 7.07k forks source link

Audio Tag/HTML Audio causing my VO to be cut off (mostly an IE issue) #2718

Closed rroylance closed 7 years ago

rroylance commented 7 years ago

This Issue is about one of these, not sure which;

I have created my audiosprite that contains the VO for my game and when loading into the game it plays fine in Web Audio but several of the lines get cut off when in Audio Tag. For example I'll have the line "Can you please find the thing that is" "12" "units long?" each being it's own sprite, one being triggered after the other in the onStop (I also tried manually timing it out using the durationMS property and a game timer, same issue); in Web Audio it plays fine, however in Audio Tag I'll hear "Can you please find the thing that" "12" "units lo".

It seems as though maybe there's a slight delay before the audio actually starts but the onStop is called as if it started playing instantly (the audio is all preloaded and decoded and everything before being used). I've tried everything I can think of, hopefully you can see something I don't.

Notes;

Thanks.

EDIT:

EDIT 2:

photonstorm commented 7 years ago

How much 'padding' have you allowed in your audio sprite? I.e. 'white noise' before and after each clip? The error you describe sounds like the problem that Audio nodes have of just taking too long to seek to the right point in the audio sprite before it plays. During this 'head movement' you lose precious ms. So if you've got a tightly packed audio sprite, with no buffer after each sound, it runs the risk of getting prematurely clipped. The fact this happens in Howler too makes me think that is exactly what happens here, and you could mitigate it by adding some padding in, and adjusting the audio sprite timings accordingly.

rroylance commented 7 years ago

I generate my audiosprites with 1 second padding between all sprites, so I doubt that's it.

photonstorm commented 7 years ago

Clearly not:

10: {
start: 0,
end: 0.6,
loop: false
},

That's 600ms.

rroylance commented 7 years ago

The sound is less than 1 second, yes, but there is a 1 second gap (more in most cases) between the end of that sound and the start of the next.

10: {
start: 0,
end: 0.6,
loop: false
},
20: {
start: 2,
end: 2.7,
loop: false
},
photonstorm commented 7 years ago

It's the short duration values causing this. The fact you have the padding between each clip already makes your task a whole lot easier. Just increase each clip duration by X ms and re-test it.

I'd go for 0.25 (250ms), but it depends (I've seen some older, crapper browsers, on certain systems, sadly take even longer than that to seek to the right point in the audio). There is no way to track when it actually starts playing, so if there's 100ms seeking going on, then you effectively lose 100ms from your duration, even though the sound hasn't actually started yet, hence the clipping.

rroylance commented 7 years ago

This happens on audio of all durations, from a few hundred milliseconds to several seconds. So the issue isn't the sound duration; it's a slow, seemingly random, playhead and the only way to avoid this (for the most part) is to pad each sound a bit extra to accommodate a random amount of seek time?

I had done some testing and found that if I Math.ceil all the end values (creating padding anywhere from 1 ms to 999ms) the issue was mostly solved albeit with adding in a tonne of extra gaps in the audio (and for VO centric experiences this wasn't ideal). I was hoping that idea wasn't the only option, but I guess it is.

Just FYI; what I've been doing is having 1 set of my audiosprites that are put together normally, no padded sounds (padding between sounds still) for browsers using Web Audio (so that they sound normal and work fine) and having a second 'rounded' set of audiosprites that are used when Audio Tag is used so that (in our case, IE can not clip as much as possible).

Thanks.

photonstorm commented 7 years ago

it's a slow, seemingly random, playhead and the only way to avoid this (for the most part) is to pad each sound a bit extra to accommodate a random amount of seek time?

Yes, that's exactly the only way to solve it. Your Math.ceil is really no different to my suggestion, just with a random additional value instead of a consistent one. The principal is the same. Being an integer makes no difference. The seek time isn't quite as random as it seems, the further the jump, the longer it appears to take. Feel free to ask MS why.

It's effectively a 'lesser of two evils' situation, you have clipped audio, or you have slight pauses after each piece. This is why for audio heavy projects on IE9/10 specifically we still use a SWF for audio playback, controlled via Phaser, even today.

rroylance commented 7 years ago

Your Math.ceil is really no different to my suggestion, just with a random additional value instead of a consistent one. The principal is the same. Being an integer makes no difference.

Ya, I had done that weeks ago and just moved on until I heard from you. It's nice having some context for why this is happening, now.

Do you know of any blog posts, browser bug tracker reports, anything like that that talks about this issue ? I've always had trouble trying to find anything about it.

photonstorm commented 7 years ago

I've never read anything about it, just bitter experience I'm afraid. I guess when IE9/10 were around, no-one really cared, because they weren't using Audio for games, they were using Flash.

rroylance commented 7 years ago

This is why for audio heavy projects on IE9/10 specifically we still use a SWF for audio playback, controlled via Phaser, even today.

Is this a part of Phaser already? I haven't noticed Flash fallback in there but I may have just missed it, if it's not, care to elaborate ?

photonstorm commented 7 years ago

No it's not part of the core, we built it for a client project, and then just kept re-using it. I could probably release it, but it'd be yet another thing to support, which I'd rather avoid tbh.

rroylance commented 7 years ago

Understandable. Thanks for the help.

photonstorm commented 7 years ago

The CreateJS libs has a SWF sound player as part of them, which would be trivial to adapt though. Or I can send you the source, I don't mind, I just don't want to release it to GitHub.

hilts-vaughan commented 7 years ago

You could probably create a Phaser plugin that uses the CreateJS audio player backend as well if you wanted. Or add another audio backtoend to phaser via plugins.

On Tue, Sep 27, 2016 at 5:00 PM, Richard Davey notifications@github.com wrote:

The CreateJS libs has a SWF sound player as part of them, which would be trivial to adapt though. Or I can send you the source, I don't mind, I just don't want to release it to GitHub.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/photonstorm/phaser/issues/2718#issuecomment-249997116, or mute the thread https://github.com/notifications/unsubscribe-auth/ABB3p1VxI2pXWvaeQ8BhYrSVg_MuxgbCks5quYPogaJpZM4JtFny .

rroylance commented 7 years ago

@photonstorm I wouldn't mind taking a look at your solution. Promise not to bug you about it, haha. Thanks.