search

playcanvas / engine

JavaScript game engine built on WebGL, WebGPU, WebXR and glTF
https://playcanvas.com
MIT License
9.67k stars 1.35k forks source link

Mac/iOS Safari Positional Sound Problem #2866

Open Kulodo opened 3 years ago

Kulodo commented 3 years ago

In the linked project there is a sound component set to positional. The camera has an audio listener component. On Mac Chrome the sound is played ok, but on Safari on Mac and iOS the sound is distorted and deafening.

https://playcanvas.com/project/765249/overview/safari-positional-sound-problem

See Forum discussion: https://forum.playcanvas.com/t/safari-positional-sound-problem/17779

I noticed that the Sound component properties for the failing case are refDistance=1, maxDistance=10, distanceModel=Linear, rolloffFactor=100. However, when I change the values to refDistance=1, maxDistance=10000, distanceModel=Linear, rolloffFactor=1, then the sound is no longer loud distorted but normal volume.

iOS 13.6.1 Safari Version 13.1.2 (15609.3.5.1.3) Mac OS Catalina 10.15.6 Chrome Version 88.0.4324.150

The following shows the difference between Safari and Chrome:

https://user-images.githubusercontent.com/74154551/107914710-eda9cc80-6f5a-11eb-889c-7445e6738116.mp4

https://user-images.githubusercontent.com/74154551/107914728-f4d0da80-6f5a-11eb-97bd-af1eaba85e87.mp4

Kulodo commented 3 years ago

This error also happen on iOS 13.6.1.

Kulodo commented 3 years ago

I upgraded my Mac from: MacOS Catalina 10.15.6 / Safari 13.1.2 to: MacOS Catalina 10.15.7 / Safari 14.1

and the issue is no longer present.

Kulodo commented 3 years ago

I will try on iOS14 maybe tomorrow. If this is just an OS/Safari issue then I'll need to figure out a workaround set of properties values on the sound component that don't cause the error.

Kulodo commented 3 years ago

The error is not present on iOS 14.3.

Adolar13 commented 3 years ago

Hey, I noticed some audio crackling in safari (osX) in several projects, even the playcanvas basic audio sample: https://developer.playcanvas.com/en/tutorials/basic-audio/ This might be related to this issue? Note that this also happens when creating a new audio context and plain html audio elements from playcanvas scripts.

Kulodo commented 3 years ago

The issue I first raise went away after I upgraded to Mac Safari 14.1 and on iOS when I upgraded to iOS14. There where no changes to the engine ( I think ) so I put this down to a Safari Issue. So this crackling could still be a Safari issue, but I don't have the knowledge to know if the engine could be the cause. In the basic-audio link above I think there is slight crackling on Mac Chrome also, but is much more obvious on Mac Safari.

Kulodo commented 3 years ago

@Adolar13 I'm seeing this issue too in iOS14.5 and Safari 14.1. Here is a simple project that shows the issue https://playcanv.as/p/LKgsRYu6/ https://playcanvas.com/editor/scene/1170102

To a very minor extent there is some crackling on Chrome Android and Mac.

@yaustar is there any chance that someone at playcanvas could take a quick look and see whether this is an Apple issue or potentially a PC issue?

yaustar commented 3 years ago

I'm currently on holiday at the moment, if someone from the team doesn't get a chance to look at by next week, I take a look myself.

Kulodo commented 3 years ago

Cheers. Enjoy the rest of your holiday.

yaustar commented 3 years ago

@Kulodo, I can't reproduce this on Mac Safari 14.1. On iOS 14.4, the OK and FAIL sound the same to my ears. Given the sound effect, could it be a speaker quality issues?

https://playcanvas.com/project/765249/overview/safari-positional-sound-problem

Will try these projects next as they have a cleaner audio: https://playcanv.as/p/LKgsRYu6/ https://playcanvas.com/editor/scene/1170102

Kulodo commented 3 years ago

@yaustar I agree that the original error i reported in https://playcanvas.com/project/765249/overview/safari-positional-sound-problem went way when upgrading to Mac safari 14.1 and iOS14.

The new project https://playcanv.as/p/LKgsRYu6/ does show bad crackling especially on iOS ( I'm on 14.5 ). This may just be a different Apple bug to the original one I reported.

yaustar commented 3 years ago

14.4 on an iPhone XS seems fine. Updating to 14.6 to see if that shows a different outcome

Kulodo commented 3 years ago

https://user-images.githubusercontent.com/74154551/121917690-4896d900-cd2d-11eb-9499-68c52557480b.mp4

Here is demo of iPhone XR 14.5.1. I also have iPhone X 14.4.2 that doesn't have the issue. https://playcanv.as/p/LKgsRYu6/ https://user-images.githubusercontent.com/74154551/121917264-dcb47080-cd2c-11eb-928c-1a1dbc9afe0d.mp4

Note that the error only occurs when "positional" is selected on the sound component

yaustar commented 3 years ago

Just tried on 14.6, I don't get that issue shown in the video 🤔 (This build: https://playcanv.as/p/LKgsRYu6/ )

yaustar commented 3 years ago

Hmm. I've let it play longer than the first 3-4 secs I do get crackling but can't tell if its as much or worse than other platforms 🤔 Android seems to be pretty bad too compared to Desktop Chrome.

Kulodo commented 3 years ago

OK I'll update the device to 14.6 and see if it gives the same results as you and probably we need to blame this on Apple.

Kulodo commented 3 years ago

Also the the "new fold" website from the other thread in room6 on 14.5.1 also manifested this error . The music gets progressively more crackling until around the 3 minutes mark it completes dies. That scene has several positional audio elements.

yaustar commented 3 years ago

Doesn't seem to be an issue with the audio itself as no matter the play offset, it generally takes the same amount of time to get really crackly

Edit: Checked with another audio file with the same issue. Preview (Female).mp3.zip

I don't think this is going to be a quick fix as this area hasn't been touched for a long while so will require someone to get up to speed with the engine and browser API.

It may be a buffer related issue as it seems to worsens as it goes

Edit: same happens with a WAV too

Kulodo commented 3 years ago

I've updated both my iOS devices to 14.6 and they both display the problem with https://playcanv.as/p/LKgsRYu6/. 14.4.2 was OK. I understand that it is not a quick fix, and looks suspiciously like a possible OS issue. I'll check with the creative half of my team and see how we can work around it for now.

yaustar commented 3 years ago

I'm putting this as a background task for myself and see how far I get

LeXXik commented 3 years ago

I'm afraid this is not PlayCanvas related, but a general issue with WebAudio implementation of browsers. In particular with positional sound. I've stumbled upon this before and was trying to find the cause to no avail. I also recorded various adio tracks in different formats in a hope that maybe it is an issue with some codec that doesn't go well with positional audio. It didn't really work. It made it worse that at some point the problem goes away on one pc/mobile, but appears on another. Or appears on the same, but with a different file, etc. Weird problem that I gave up trying to fix :)

Adolar13 commented 3 years ago

Yep, several WebAudio related bugs in the webkit bug database: For example:

Adolar13 commented 2 years ago

Just a follow up on the issue. I was playing around with an implementation of Google Resonance for spatial audio and it got crackling and distortion in a playcanvas project as well, however I haven't had those issues in a standalone html version nearly using the same codebase except getting the audio context from app.systems.sound when using playcanvas. I'll do some more experimenting and set up an example project.

Adolar13 commented 2 years ago

Here is a project for comparing PC audio with a pure webAudio implementation. It seems that using WebAudio directly doesn't seem to cause the distortion problem: https://playcanvas.com/project/867811/overview/webaudio-test

There seems to be a gap when the audio loops in safari though when using WebAudio directly.

Update: Pure WebAudio with using Playcanvas audio context still causing the issue just a lot later.

Adolar13 commented 2 years ago

It seems a fix landed in webkit but isn't yet shipped in current safari. However I was trying safari tech preview with no issues for a period of ~15min.

yaustar commented 2 years ago

Awesome, thanks for keeping an eye on this and testing the tech preview!

Adolar13 commented 2 years ago

Seems to be fixed in non tech preview version 15.2 as well. Will do some longer running tests and report back.

dataexcess commented 1 year ago

Hey hey, just reporting in that I see different behaviour on Safari for positional audio with distanceModel 'exponential' or 'inverse'. The rolloffFactor seems to be having less of an impact than on Chrome/Firefox. Because of this my website is broken on Safari: all the volumes are calculated on the distance between my positional audio instances - using distanceModel 'exp', working perfectly and mixing smoothly on Chrome / Firefox, but playing all at once and thus too loud on Safari!

Site: https://yggdrasil-908ca4owk-dataexcess.vercel.app/

Any help/tips much appreciated. When using distanceModel linear there is no issue, so I think it has to do with the rolloff Factor ?