lightspark / lightspark

An open source flash player implementation
http://lightspark.github.io/
GNU Lesser General Public License v3.0
1.8k stars 221 forks source link

Lightspark for Artificial Intelligence Research #816

Open perara opened 3 years ago

perara commented 3 years ago

Hi there, I've had an increasing interest in your efforts toward a functional flash backend and I've started moving my efforts from Gnash to Lightspark. My efforts are located at https://github.com/cair/rl (https://github.com/cair/rl/tree/main/cpp/extern/flashrl-lightspark for lightspark "hooks")

Summary / TLDR:

  1. Is it possible to retrieve the raw memory state of a running game
  2. Is it possible to rewind game states. If so, how?

Description My project FlashRL is an effort to ease the use of Flash games reinforcement learning research and my initial implementation allows me to embed your Lightspark emulator and successfully run different games. With some modifications, primarily in exposing functions been able to extract the renderer image and are able to control the mouse and keyboard without any issues.

Digging through the code, I've found indications that the emulator store the memory structure of the flash runtime and I was wondering where and how I would be able to read the memory segment of a running game. Essentially, what I want is to read all of the memory bytes and present this to the RL algorithms as "raw information"

Second, I found some indications that it is possible to rewind the game state. Is this code functioning, and if so, could you direct me towards how this mechanism works?

Side Note On a side note to my efforts. It has come to my attention that you rely on gnash for older flash games. Currently you rely on the presence of the binary of the system. While I haven't looked to much at licencing terms etc, an alternative would be to allow builds with gnash "embedded": My work on this is located here if your interested: https://github.com/perara-libs/flashrl-gnash

dbluelle commented 3 years ago

1) I don't know if there is an easy way get the "memory state". The swf is more or less represented as a big list of objects derived from the ASObject class. We currently have the possibility to dump the count of currently allocated ActionScript objects when building in DEBUG mode (see line 1718ff in asobject.cpp), but I don't know how helpful that is in your case. Additionally we have several caches for bitmaps, shapes and other objects that are not neccessarily referenced in ASObject derived classes. 2) Hmm, I don't think such a thing currently exists. For rewinding you would have to somehow record all methods executed in ActionScript. It might be possible for simple MovieClips that consist of multiple frames that are presented to the user, but as soon as any actionscript is involved, this will not work properly.

embedding gnash is not neccessary any more. Lightspark is capable of displaying all versions of flash content for some time now and the gnash fallback is only used if you explicitely enable it by an environment variable