flyinghead / flycast

Flycast is a multiplatform Sega Dreamcast, Naomi, Naomi 2 and Atomiswave emulator
GNU General Public License v2.0
1.36k stars 167 forks source link

TAS capabilities? #505

Open Loomeh opened 2 years ago

Loomeh commented 2 years ago

There are currently no Dreamcast emulators with TAS (Tool-assisted speedrun/superplay) abilities. This makes TASing Dreamcast games impossible.

I was hoping TAS tools like frame advance, storing inputs as files, rerecording, memory watch and avi frame dumping could be added.

I've tried to TAS PC HD ports of Dreamcast games (for example Jet Set Radio). However, PC TASing tools are extremely primative and so far it looks like TASing PC ports of Dreamcast games is also impossible.

More in-depth explanation of what a TAS is: https://youtu.be/Ietk1-Wb7oY

blueminder commented 2 years ago

FWIW, the latest Flycast Dojo pre-release has frame stepping and live input recording/playback in the Training Mode. The replay files in Dojo also act the same as a piano roll of inputs. I'm hoping to expand the features further from there.

https://github.com/blueminder/flycast-dojo/releases/tag/dojo-0.4.26

As far as video export goes, the last attempt at it was done by glisignoli. Progress got pretty far, with audio syncing being the main obstacle. Details on it can be found here, in case anyone else wants to take the challenge on:

https://github.com/glisignoli/flycast-dojo/blob/feature/recording/RECORDING.md

Loomeh commented 2 years ago

FWIW, the latest Flycast Dojo pre-release has frame stepping and live input recording/playback in the Training Mode. The replay files in Dojo also act the same as a piano roll of inputs. I'm hoping to expand the features further from there.

https://github.com/blueminder/flycast-dojo/releases/tag/dojo-0.4.26

As far as video export goes, the last attempt at it was done by glisignoli. Progress got pretty far, with audio syncing being the main obstacle. Details on it can be found here, in case anyone else wants to take the challenge on:

https://github.com/glisignoli/flycast-dojo/blob/feature/recording/RECORDING.md

I tried to TAS Sonic Adventure 2 but when trying to play back the replay it asked for Naomi BIOS. SA2 is not a Naomi game?

glisignoli commented 2 years ago

Yeah I tried to add avi recording. I'm not any sort of c programmer but I got pretty far.

What I was trying to do was modify gles.cpp to write out each frame to a video file. I found some code that already did this but it didn't have sound. So I added an audio backend that would write out to a wave file.

So funnily enough I thought the sound would be a problem, but it worked perfectly. The problem is the video was playing to fast. Instead of rendering 60fps (or whatever the refresh rate is) it would only render frames if they were drawn.

So for example: A static 'now loading' screen would only be drawn for a single frame while loading, even though it appears on screen much longer.

I tried doing something clever where I store the previous frame in memory and check the time difference between the last and current frame, if it's greater than 16.67ms (I think) I add the previous frame to the video file. I couldn't get this to work though.

If anyone can figure out how to solve this it would be pretty amazing.

vkedwardli commented 2 years ago

https://www.youtube.com/watch?v=0MKo1nD33yg

The above video is a PoC showing 4 Flycast instances replay + recording. However it is using macOS's video capture (so I don't have to tackle the "number of rendered frames" vs "actual vsync frames" problem).

If the screen recording API is not preferred (may be it is too CPU/GPU demanding?), may be you can use Syphon (macOS) or Spout (Windows) to pipe the GPU texture to a video encoder (a custom recorder or OBS)

This GPU piping method is particular useful for a slow machine like the first generation Retina iMac, which has a staggering 5120 × 2880 resolution: Since the GPU is not powerful enough for playing a 640x480 game + screen capturing at retina resolution.

Currently when using the first gen iMac 5k, the user have to set its desktop resolution to non retina e.g. 1280x720 and play at 960x720 (1.5x) to have a smooth live streaming experience. I have tested to integrate Syphon (OpenGL) on macOS, it can reduce a lot of CPU/GPU loading for live streaming purpose.