input delay

[GoogleCodeExporter]

What steps will reproduce the problem?
1. using genplus via libretro and retroarch, load any game that polls input 
from the system during vblank (which afaict seems to be most games).  i'll be 
using sonic 1 as an example.
2. get to a point where you're able to control an onscreen character.
3. press the button retroarch has assigned to frame_advance once.
4. press and hold an input button that will produce a visual reaction on 
screen, such as a/b/c to make sonic jump while on the ground.
5. continue pressing the frame advance button until you see a reaction in the 
game, note the number of presses/frames needed to respond.

What is the expected output? What do you see instead?
for sonic 1 it should only require 2 frames to see a response in the game, with 
genplus via libretro and retroarch it takes 3. (mednafen for example only takes 
2 frames to respond)

What version of the emulator are you using (official, SVN revision,...)?
latest git versions of both retroarch and libretro-genplus as of this date

Please provide any additional information below (Emulator settings, Console
setting,...)

sonic 1, like many genesis games, polls input during vblank and uses the 
results when it steps the game logic for the next frame.  the system_frame step 
functions in genplus look like they're setup to emulate the active display 
portion of the frame followed by the vblank portion before returning.  unless 
it would adversely affect other areas of the program, to avoid what i feel is 
an unnecessary input delay (which could be up to 1 frame) perhaps these 
functions should be restructured in a way so that they return right after the 
active display portion of a frame is complete, then the next call to 
system_frame_* would begin by polling input (osd_input_update) and emulating 
the vblank portion of the frame it left off on.

if i'm not mistaken, many other emulators are setup like this.  gambatte will 
return from its main loop as soon as a frame is complete, and fceu is simply 
structured in the opposite way of genplus (ie vblank first (even on the first 
call to its step frame function), then the active display portion of the frame)

Original issue reported on code.google.com by eta...@gmail.com on 11 Sep 2012 at 2:49

[GoogleCodeExporter]

genesis plus gx already implements what you are describing: emulated inputs are 
updated (through osd_input_update, which is platform dependent) DURING the 
system_frame function, just BEFORE triggering the emulated vblank interrupt, so 
it is read immediaty by the emulated game polling routine and used in the next 
frame without any delay.

I am not 100% sure if the method you are using is valid (the core does not 
provide any frame_advance functionality) but what you see is most likely due to 
the way osd_inputs_update is implemented in the retroarch implementation you 
are using, how system_frame is synced with the host hardware controllers 
refresh and if specific GUI  keys are read at the same time as normal buttons 
or not. Anyway, it is related to libretro or retroarch, not genesis plus gx 
itself, so you might better report this on their side.

Original comment by ekeeke31@gmail.com on 11 Sep 2012 at 6:38

• Changed state: Invalid

[GoogleCodeExporter]

If you look at our implementation of osd_input_update, you can see that we do 
it almost exactly the same as you do in the standalone GX version - first we 
poll the frontend for input state, then this data is passed to the core and 
does something with it.

https://github.com/libretro/Genesis-Plus-GX/blob/master/libretro/libretro.c

Anyway, since RetroArch has been ported to so many systems, it might help if 
the OP would report on which specific system/console he is testing this against 
-ie. PC, Wii, PS3, Xbox 1, Xbox 360, or whatever. As it is, all these systems 
can produce varying amounts of input latency.

Original comment by libre...@gmail.com on 12 Sep 2012 at 4:22

[GoogleCodeExporter]

Well, i don't the think the method used in the first place can accurately claim 
if an emulator has more input latency than an other, for the main reason that 
it is tied to when retroarch polls and detect the frame_advance key as well.
 Since this is most likely done just before emulating the frame (system_frame call in our case), if emulated inputs are not updated at the same time (i.e before system_frame), there will always be one frame delay between the two events, even if vblank is emulated just after updating emulated inputs like i'm doing.

However, that does not necessarily mean the emulator has one frame latency 
compared to others because what matters is the delay between when host system 
inputs are being updated from hardware direct access (by the system itself, OS, 
driver, etc) and when the effect is seen on screen. 
Since most of the time this is done during host system VSYNC period and frame 
emulation is generally synced with beginning of VSYNC, this would generally 
happen DURING frame emulation, and, in the case emulated inputs are updated 
before frame emulation, AFTER inputs have been polled for that frame. So in any 
case, input change will not be reflected in upcoming frame but the next one. 

Similarely, if you use the frame_advance function, key press will be detected 
with one frame delay and system_frame not executed while it would have likely 
detected button change if it had been run, since input_update function is 
processed later in the frame.

That's said, after some thoughts, i agree it makes more sense to start frame 
emulation with VBLANK emulation and emulate the corresponding display in the 
same frame, not the next one, in the case where frame emulation is synced to 
host system so that hardware inputs have updated betwen the end of previous 
emulated frame and the beginning of the next one. I know this is actually not 
the case on Wii so it does not mattered (and requires some heavy core design 
changes) but it could be done differently on other systems...

I don't know if I was very clear, this is not necessarily easy-to-understand 
concepts, but what i am sure is that using advance_frame functionality, which 
will obviously be synced to key inputs itself and will affects the function 
where emulator inputs are being updated, in order to measure input latency, is 
not rigourous.

Original comment by ekeeke31@gmail.com on 12 Sep 2012 at 7:40

[GoogleCodeExporter]

ekeeke - On a somewhat related note to this -

[quote]
That's said, after some thoughts, i agree it makes more sense to start frame 
emulation with VBLANK emulation and emulate the corresponding display in the 
same frame, not the next one, in the case where frame emulation is synced to 
host system so that hardware inputs have updated betwen the end of previous 
emulated frame and the beginning of the next one. I know this is actually not 
the case on Wii so it does not mattered (and requires some heavy core design 
changes) but it could be done differently on other systems...
[/quote]

Perhaps the audio/video sync we arrived at for RetroArch Wii would appeal to 
you - we sync on both video and audio by way of a trick we call 'variable audio 
input rate control'. It has worked very well across nearly all emulators/games 
we've ported to RetroArch and it's audio pop-free/video-glitch free.

Genesis Plus GX on libretro RetroArch Wii BTW runs very well using this syncing 
method - I can't profess to know how exactly it performs on the standalone Wii 
port, but Virtua Racing runs in all three attract mode intros at 60fps - even 
at the bridge which you previously mentioned was a pretty intensive scene. This 
is all without any frameskipping.

Original comment by libre...@gmail.com on 13 Sep 2012 at 3:10

[GoogleCodeExporter]

As soon as you sync with audio, yes, it will be audio pop free. The same 
actually happens if you disable VSYNC in Genesis Plus GX and you would hardly 
notice any skipped or repeated frame either because the rates are pretty close 
enough. BUT that does not mean it does not happen once in a while, which is why 
i am asking how you can be 100% sure it NEVER happens with your implementation. 
Audio pops are easy to figure, a single repeated or skipped frame once in while 
(every 30s sometime if rates precision is good), much less. When you start 
syncing with both, from what i've observed, it will always derive and ends with 
a missing frame, depending on which one you synchronize first and how much 
precise your timings are.

 I couldn't actually understand how audio rate control exactly worked from looking at retroarch code, seemed like to me it was something related to resampling but couldn't figure how it would work in our case since the number of samples rendered per frame is fixed by the emulator and that wii output samplerate is also fixed.

Anyway, a good way to check if frames are never dropped (case where emulation 
renders two frames before video backend had time to display one) or repeated 
(case where emulation was too late at rendering next frame and the previous one 
ends being displayed twice) is with Sonic 2 during splitted screen VS mode. If 
you turn ON original render mode and VSYNC OFF in genplus-gx here, you will 
notice that every 10-15s, the screen got glitched. What happens is that this 
part of the game switches into interlaced video mode which doubles the 
resolution of the screen to 448 lines but halfs the refresh rate (odd field is 
refreshed at 60hz then even field and the whole frame is refreshed at 30hz). In 
original mode, this is emulated very accurately just like a real Mega Drive 
would do: the VDP still generate 224 lines on each system_frame call like usual 
so we still output a 224 lines-high frame buffer (GX single-field framebuffer 
mode) as in non-interlaced original mode but each field must be rendered 
sequentially and feeded to video backend in good time. If a SINGLE field is 
skipped or repeated because of brief desync, ALL the consecutive screens will 
have odd lines in place of even lines, until the next desync where they got 
replaced correctly.

Off course, this does not happen if you use Wii double-field framebuffer 
interlaced mode since in this case, the framebuffer is 448 lines and odd/even 
lines are always rendered then displayed to their proper location on screen. 
The result is less accurate however and very different from how a real Mega 
Drive would display (and flicker) in interlaced mode.

Since I think retroarch-wii only uses double-field framebuffer mode, it would 
be interesting to know how it handle this game part if you ever implement 
single-field rendering mode support (also referred as 240p or "original 
scanline" mode, though here it is more 240i in fact and there is no visible 
scanlines because screen is interlaced). This is actually the only way to be 
100% sure emulation is perfectly synced with video as well as audio, since the 
effect is quite visible and amnoying. I have personally tried many techniques 
(including syncing with both video and audio or adjusing input rates) but never 
succeeded in getting this one right unless emulation is synced with video only.

Original comment by ekeeke31@gmail.com on 13 Sep 2012 at 9:21

[GoogleCodeExporter]

i didn't explain myself very well either.  i understand the method i used isn't 
conclusive, the way system_frame_* is currently structured _may_ not be causing 
any added input delay.  however, afaict restructuring these functions couldn't 
hurt and may actually help, particularly on other platforms where the hardware 
might update the input faster / at a higher rate.

what i was trying to describe in the op i think is best explained with a 
figure/drawing, so i scribbled one out quickly showing what i believe the 
difference in approaches would look like:

http://i48.tinypic.com/2lvi984.png

in the middle would be the host system, small boxes being vblank period, larger 
ones being the active display.  the left and right columns would be the 
emulated system frames, left with system_frame_* the way it's currently 
structured and right with the way i was suggesting it be changed to.

Original comment by eta...@gmail.com on 13 Sep 2012 at 9:48

[GoogleCodeExporter]

Yes, as i said before, you would ideally start the frame with vblank so that 
end of frame immediately reflects input changes detected during vblank by the 
game but the problem is that the design of the emulator does not easily permits 
that. The main problem is that overscan areas, which normally appears at the 
start  (bottom border) and end (top border) of vblank are emulated and their 
height will change if active area height is modified during active area. This 
is actually why i made the frame starts with active area so i know the borders 
height to render in current frame when it ends (not sure if this is very 
clear). 

However, i am still not convinced it would really reduce input latency since it 
pretty much depends on how system_frame updates are synced with the video 
output anyway and having inputs polled then vblank near the end of the current 
frame is very similar to having them at the beginning of the next frame.

Another solution could be to get ride of the traditionnal use of system_frame 
call and use an infinite loop that call audio/video/inputs update function at 
specific points of the frame... but again , i am not sure if there is much to 
gain. What is sure is that i never got complaints from users experimenting 
input latency with this emulator (as opposed to other ones on wii) and we are 
speaking about AT MAX (probably much less) one frame (16.67 ms) delay that no 
human could realistically be able to detect.

Original comment by ekeeke31@gmail.com on 13 Sep 2012 at 12:03

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Guess it's never too late to change his mind so it's now 2 years later and I 
ended up implementing this in r927.

Also removed one frame of delay when vsync is used in r899.

Two frames delay might not be much and probably not noticeable by anyone on 
their own but it's still two additional frames of delay that would add 
themselves to any existing hardware latency and could make it become noticeable.

Original comment by ekeeke31@gmail.com on 14 Dec 2014 at 11:27

• Changed state: Verified
• Added labels: Type-Enhancement, Priority-Medium, Performance