search

supermedium / moonrider

🌕🏄🏿 Surf the musical road among the stars. Side project built by two people in a few months to demonstrate WebXR.
https://moonrider.xyz
MIT License
570 stars 222 forks source link

Sync is an issue Quest2 #125

Open NoSloppy opened 3 years ago

NoSloppy commented 3 years ago

Hi. Definitely a sync issue as of 12/30/20 on a brand new Oculus quest 2. Notes hit target timing before the song's beat. Unfortunately, this makes having music practically pointless as it's strictly a visual cue to match your actions to. To recreate, load up Imperial March by Celldweller. You need to hit the notes exactly on the upbeat of the prior beat. So if the count is 1 and 2 and 3 and 4 and..... <---- that last "and" is where you need to hit the note for the next 1. Hard to put in words, but very reproducible.

dmarcos commented 3 years ago

@NoSloppy Can you give it another try? Thanks!

NoSloppy commented 3 years ago

Hi thanks for the reply! Reloaded the page, still not right, if anything seems a bit worse? Very hard to demonstrate, but I attempted a video: https://photos.app.goo.gl/73mDrPWjYLYGYgtB7

On Thu, Dec 31, 2020 at 12:13 AM Diego Marcos notifications@github.com wrote:

@NoSloppy https://github.com/NoSloppy Can you give it another try? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-752858541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I65WD4DHON5LLG7BMDSXQJA5ANCNFSM4VPC3KTQ .

NoSloppy commented 3 years ago

Also, would a calibration menu item be a good idea? This would allow the user to adjust the sync issue due to different variables...device used, browser version, internet connectivity etc...

On Thu, Dec 31, 2020, 2:05 AM Brian Conner nyquist747@gmail.com wrote:

Hi thanks for the reply! Reloaded the page, still not right, if anything seems a bit worse? Very hard to demonstrate, but I attempted a video: https://photos.app.goo.gl/73mDrPWjYLYGYgtB7

On Thu, Dec 31, 2020 at 12:13 AM Diego Marcos notifications@github.com wrote:

@NoSloppy https://github.com/NoSloppy Can you give it another try? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-752858541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I65WD4DHON5LLG7BMDSXQJA5ANCNFSM4VPC3KTQ .

danjferg commented 3 years ago

I also have this issue on my new Quest 2. In the default browser my notes are also arriving early. I tried scooting back in my play volume and delaying my strikes, but it's so bad the blocks fail before I swing.

I tried Firefox, and the timing was good. But the transformations in Firefox were bad, especially when turning your head.

I've seen threads complaining about audio delay in other Oculus apps, so it could just be an Oculus problem. So this is probably a feature request. My first thought was also to have a menu item to adjust the sync. Something like the "Track Synchronization" in VLC would be great.

If you don't have a quest 2 to troubleshoot, let me know if I can provide any additional information. I'm new to VR and A-Frame, but I could probably setup debugging if I knew what to look for.

fellowbak commented 3 years ago

Unfortunately, I have the same problem with my Oculus Quest 2. I'd no issues on my Quest 1.

cabanier commented 3 years ago

What happens if you run moon rider at a lower frame rate? You can do this by passing in "low-refresh-rate" during requestSession.

It would be interesting to see if the situation is the same, or reversed.

dmarcos commented 3 years ago

@cabanier Thanks! Is there a chrome://flags option to force low-refresh-rate? It would be handy.

cabanier commented 3 years ago

No. Unfortunately it's only exposed by javascript. I was looking at the moonrider code. How does it know that the beat of the music is lining up correctly? It looks like there are some hardcoded offsets...

dmarcos commented 3 years ago

@NoSloppy Do you see the sync issue in any other songs or only that specific one? Do you have a desktop PC you can use with Oculus Link and compare the results with Chrome / Firefox? Do you happen to have access to an Oculus 1 to compare to? Thanks so much for helping investigate

NoSloppy commented 3 years ago

Thanks for the reply. Unfortunately, it was not my Oculus and I no longer have access. I can say for sure that it was not just that song...that one just felt like the most calculatable offset, probably due to the BPM, but it was uncomfortable to play any song due to lack of sync. The audio was always behind the beat of the visual notes. Even if I had the device in hand, I do not have a PC to link to, and no Oculus 1 to compare, sorry.

On Wed, Jan 6, 2021 at 3:36 PM Diego Marcos notifications@github.com wrote:

@NoSloppy https://github.com/NoSloppy Do you see the sync issue in any other songs or only that specific one? Do you have a desktop PC you can use with Oculus Link and compare the results with Chrome / Firefox? Do you happen to have access to an Oculus 1 to compare to? Thanks so much for helping investigate

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-755654117, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I4HIFIXPUAU4BC6VIDSYTCTHANCNFSM4VPC3KTQ .

cabanier commented 3 years ago

If the audio is behind, that means that the rendering is ahead. Since Quest 2 has a faster processor, it makes sense that the frame is finished faster.

The Oculus SDK has the ability to tell the application when the frame is going to be presented to the user. However, WebXR does not have this ability. It just reports the time when the frame callback is called. I'm going to think about what we can do about this...

NoSloppy commented 3 years ago

Would it be difficult to have a calibration slider in a settings menu? I know other games have a manual "tap to set" sort of calibration to compensate for lag in the video signal chain, where hitting a button when the visual frame hits, then when a click track ticks sets the offset.

On Wed, Jan 6, 2021 at 4:51 PM Rik Cabanier notifications@github.com wrote:

If the audio is behind, that means that the rendering is ahead. Since Quest 2 has a faster processor, it makes sense that the frame is finished faster.

The Oculus SDK has the ability to tell the application when the frame is going to be presented to the user. However, WebXR does not have this ability. It just reports the time when the frame callback is called. I'm going to think about what we can do about this...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-755735515, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I3EIKVKVE24BEYB4KTSYTLO7ANCNFSM4VPC3KTQ .

cabanier commented 3 years ago

@danjferg do you remember where you saw the complaints about the delays? @NoSloppy how much would you say the delay is? 1/10 of a second? 1 second?

NoSloppy commented 3 years ago

Tough to say. About .5 sec? The example song I mentioned seemed to be in sync on the next upbeat...so 1 and 2 and 3 and....audio beat would land on the following AND not the downbeat. I do not remember the BPM, that would provide a base for timing estimation.

On Wed, Jan 6, 2021 at 5:01 PM Rik Cabanier notifications@github.com wrote:

@danjferg https://github.com/danjferg do you remember where you saw the complaints about the delays? @NoSloppy https://github.com/NoSloppy how much would you say the delay is? 1/10 of a second? 1 second?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-755739637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I55J7H7ZZKFLPP7HC3SYTMSPANCNFSM4VPC3KTQ .

danjferg commented 3 years ago

@cabanier I think the bulk of what I saw was in the Oculus forums: quest 2 audio sync site:forums.oculusvr.com. Most of those seem related to casting, network and recording activities where lag could be expected. The other one I saw was a Reddit post, "Bug? Audio delay in Oculus Video / Oculus Gallery" which I now realize is very old.

As for the magnitude of the delay, it seems pretty consistent for me regardless of the song. It starts at 0.15 to 0.20 seconds and seems to slowly get better through as the song progresses. By the end of the song I can scoot back far enough to barely hit the blocks on time.

cabanier commented 3 years ago

FYI I found the issue on our side and it will be fixed in an upcoming release. Thanks you all for your help!

NoSloppy commented 3 years ago

Awesome, thanks for the update!

On Sat, Jan 9, 2021, 2:51 PM Rik Cabanier notifications@github.com wrote:

FYI I found the issue on our side and it will be fixed in an upcoming release. Thanks you all for your help!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/supermedium/moonrider/issues/125#issuecomment-757358188, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM3W3I4LCVK4WABYUIBUVEDSZCXS3ANCNFSM4VPC3KTQ .

danjferg commented 3 years ago

Great news! The kids and I are very excited!

dmarcos commented 3 years ago

@cabanier Woohoo! Thanks so much. This is going to make so many people happy.

soltrinox commented 3 years ago

Is there a recommendation to locally cache the Audio/Video files and then play the local SYNC files ? Possibly a WEB ASSEMBLY that handles the local stream HTTPS ?

GortiZ6 commented 2 years ago

@cabanier is there any news on this? Is it closed? Is there any browser suggestion for the Quest 2 apart from the default one? Sometimes I see the game not detecting my saber or exploding the arrows late and I was wondering if this could be the reason.

cabanier commented 2 years ago

I thought we fixed the audio issue. Are you saying there are other glitches during gameplay?

GortiZ6 commented 2 years ago

As I said I stumbled on this issue searching for someone with my same problem. Currently I would not define it really a "delay" issue since it's not "deterministic". It happens that sometime the "detection" of hits seems to be in pause or in delay, but it's not for the entire game. It seems to last for some portion of the song and not even on all songs. To me is more similar to a CPU too much loaded that create the problem (that's why I asked if there was a suggested browser, thinking it might be related to the oculus browser busy doing "stuff"). It's more noticeable in punch mode since I see that I punch on the symbols, but they do not explode (none of them for few seconds), in classic mode it's less frequent the "complete undetection", but it happens that for a portion of the song I can hit the arrows correctly and for few seconds I'm always in advance (I can hit the arrow, but roughly when they on my hands instead on the sword blade), and that's why I thought about a delay on the detection and posted on this issue (since it's still open I thought that it was not merged or something was still wrong)

Mike-B-22 commented 1 year ago

I can confirm this is still an issue, in Classic Mode. I discovered Moon Rider just yesterday, and had a great time playing Punch Mode, all the target-strike sounds were totally in-sync, they all matched the beats of the music precisely. But then I tried Classic Mode, and it was unfortunately very off-beat, none of the 'crash' sound effects from striking the targets (or the visuals) were even close to landing on the exact beats of the song, like in perfect rhythm. I was playing with a Meta Quest 2 headset, using their standard browser that came with the unit. I'll keep playing the Punch Mode though, that was a whole lot of fun and a great workout. I can't believe they had "On The Floor" by Jennifer Lopez :)