search

ppy / osu

rhythm is just a *click* away!
https://osu.ppy.sh
MIT License
15.17k stars 2.25k forks source link

Aspire beatmap video is not aligned correctly with gameplay #13696

Closed Moltanicaa closed 3 months ago

Moltanicaa commented 3 years ago

Describe the bug: This map is an aspire type of map that emphasizes the video in the background. But here, for some strange reason, the video is in a hurry for about 1.5-2 seconds. Screenshots or videos showing encountered issue: lazer gameplay: https://youtu.be/cJvUl7EoxAs

original: https://youtu.be/0ZXNxdxWZmo

map - https://osu.ppy.sh/beatmapsets/1503327#osu/3079907 osu!lazer version: 2021.623.0

Moltanicaa commented 3 years ago

@bdach I have information about another problematic aspire map, where there is a problem with music playback (assuming the map starts with negative time values), a storyboard problem, where storyboard objects are displayed much earlier than they should. I'm not sure if you know about this map now. Is it possible to create issue? This map - https://osu.ppy.sh/beatmapsets/1237363#osu/2571858

I will remove this comment after answering

bdach commented 3 years ago

@Moltanicaa separate issue for that please.

peppy commented 3 years ago

Can you confirm this only occurs for you on this one beatmap, not others with video offsets?

Moltanicaa commented 3 years ago

https://youtu.be/VlE4B8N2NUs

bdach commented 3 years ago

The map used for the comparison above (https://osu.ppy.sh/beatmapsets/1376486#osu/2869013) has video offset 0, but I've taken it and manually edited one of the beatmap .osu files to set offset to -1 and there seems to be no discernible difference. So likely it's somehow specific to this one broken beatmap from the OP.

Wieku commented 2 years ago

I experienced the same problem with this map in danser.

After digging into it, it turns out MP4/MOV container contains metadata called "edit list" which instructs parsers how to cut the video. Most players except QuickTime/VLC/FFPlay ignore it.

It looks like osu!stable ignores it as well, so videos with that data become misaligned in lazer. For now I'm unaware if are there other videos with similar problem.

ffmpeg CLI/ffplay have -ignore_editlist argument which fixes the problem but I don't know how to apply it with ffmpeg API.

bdach commented 2 years ago

Poking in ffmpeg source & docs it looks like the desired incantation should be something along the lines of av_opt_set_int(formatContext, "ignore_editlist", 1, 0) or similar? Have not tested, though.

Not sure how to jam that into framework, either... (do we just hardcode, or do we add some configuration primitive for it?)

peppy commented 2 years ago

Hardcoding sounds fine until it becomes an issue, I think.

Wieku commented 2 years ago

IIRC almost everyone re-encodes the video with ffmpeg, even cutting without re-encode actually removes frames, so the only source of editlist would be if someone used QuickTime to trim the video (which seems to be the case here as the video still has audio attached).

TechniKris commented 1 year ago

The issue seems to affect not just loved aspire maps, but also a ranked mapset (potentially multiple). I don't know if the technical reason behind the issue is the same, but there seems to be the same outcome.

Affected mapset: https://osu.ppy.sh/beatmapsets/1313664#osu/2722655 The topdiff [Repetition] visualizes it best, it's especially noticeable when the girl in the video repeatedly jumps up and down following sliders (it happens around every chorus). Eyeballing it, the offset in lazer is maybe a couple tenths of a second, but it's a large enough difference to completely break the experience of cursordancing together with the video.

The map is playable just fine with 100% dim or without video, but it somewhat affects playability with video playing and dim set to lower values.

I haven't tested them extensively since they don't do the same "dance" gimmick, but these two mapsets seem to use the exact same background video file:

I just wanted to bring this up since it's a ranked map that is affected.

bdach commented 3 months ago

I can't see the issue on beatmaps which @TechniKris mentioned locally. The beatmap in the OP looks clearly broken but not the purportedly "affected" mapset.

TechniKris commented 3 months ago

I can't see the issue on beatmaps which @TechniKris mentioned locally. The beatmap in the OP looks clearly broken but not the purportedly "affected" mapset.

Ah, I should've made a video showcasing the issue. Here, I made one now. I synced up gameplay, but the video on lazer is 8 frames (of this 60fps recording) ahead of stable

https://github.com/ppy/osu/assets/30842158/25a110eb-f6e2-4c49-aab8-f2814283ee52