Discussion / Feedback: Issues running in Node / Server Side

[EuphoricPenguin]

C:\Users\Euphoric\Documents\wasmboytesting>node index.js
internal/modules/cjs/loader.js:596
    throw err;
    ^

Error: Cannot find module 'worker_threads'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (C:\Users\Euphoric\Documents\wasmboytesting\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:3379:5)
    at Module._compile (internal/modules/cjs/loader.js:702:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
    at Module.load (internal/modules/cjs/loader.js:612:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
    at Function.Module._load (internal/modules/cjs/loader.js:543:3)

C:\Users\Euphoric\Documents\wasmboytesting>

I've commented out everything in the code except for the call in for wasmboy, and this error still occurred.

[torch2424]

Hello!

This issue happens because we use web workers in Wasmboy, and this is done in node using worker threads: https://nodejs.org/api/worker_threads.html

Thus, could you add the flag --experimental-worker, or whatever it says in the docs? (Currently on my phone while my laptop updates)

Therefore, your command would be node --experimental-workers index js. And this would require the latest stable of node :)

Thanks for checking out Wasmboy! If you don't mind, may I ask what you are building / want to do. I would love to support it / check it out / add it to the readme! :)

[EuphoricPenguin]

Checked what I think was the docs, apparently, it is imported with a module require. I'm installing the current version of Node (12.1.0) right now. IDK if you have to use any flags, I'll check in a second. The project I'm working on right now doesn't use any of wasmboy yet, but hopefully, it will soon. Currently, it's just a test to get a basic code working. https://github.com/EuphoricPenguin/DiscordPlaysGameBoy - Here's where it will be eventually, in case you were wondering. It isn't part of it yet, but hopefully, it will be the GB/GBC emulator once it's complete.

[torch2424]

@EuphoricPenguin

Cool sounds good! Let me know if adding the flag works :) It's on my TODO list, to move the workers to use the node threads thing 👍

And Ohhhh that would be SOOOOO rad! I've been looking forward to seeing this run for an actual practical purpose on a server 😄 Rather than just my tests. Would love to help test this out in the future! 🎉

[EuphoricPenguin]

Yeah, the latest version of Node is running fine without issues. I didn't even have to use the flag, but I did call in var worker = require("worker_threads");.

[EuphoricPenguin]

Oh, @torch2424 , quick thing. It's saying all of the functions aren't functions.

var screen = WasmBoy.getCanvas();
                     ^

TypeError: WasmBoy.getCanvas is not a function
    at Object.<anonymous> (C:\Users\Euphoric\Documents\wasmboytesting\index.js:5:22)
    at Module._compile (internal/modules/cjs/loader.js:759:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
    at Module.load (internal/modules/cjs/loader.js:628:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:824:10)
    at internal/main/run_main_module.js:17:11

C:\Users\Euphoric\Documents\wasmboytesting>

[torch2424]

@EuphoricPenguin

Yeah, the latest version of Node is running fine without issues. I didn't even have to use the flag, but I did call in var worker = require("worker_threads");.

Ahhh hmm. That's quite odd. I don't need to do that in the tests. Maybe I'll make a quick server side demo project on glitch or something to figure this out...

Oh, @torch2424 , quick thing. It's saying all of the functions aren't functions.

Can you console.log the WasmBoy Object? E.g console.log(WasmBoy)?

I think I may have messed up my rollup config, and made WasmBoy a named export. Thus, you may have to do something like WasmBoy.WasmBoy.getCanvas() 😢

Which, yep, seems to be the case, my bad! https://github.com/torch2424/wasmboy/blob/master/test/accuracy/accuracy-test.js#L5

The tests will probably be another good place to figure things out for now. As the docs on the wiki are kinda out of date, and made with browsers in mind (mostly). But more than happy to answer questions here. I may be a bit delayed in response though 👍

[EuphoricPenguin]

C:\Users\Euphoric\Documents\wasmboytesting>node index.js
WasmBoy is configured!
Error Configuring WasmBoy...
(node:2616) UnhandledPromiseRejectionWarning: ReferenceError: fetch is not defined
    at fetchROMAsByteArrayTask (C:\Users\Euphoric\Documents\wasmboytesting\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:2418:19)
    at fetchROMAsByteArray (C:\Users\Euphoric\Documents\wasmboytesting\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:2445:10)
    at loadROMAndConfigTask (C:\Users\Euphoric\Documents\wasmboytesting\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:3611:34)
    at loadROMTask (C:\Users\Euphoric\Documents\wasmboytesting\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:3650:13)
(node:2616) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:2616) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

This seems to be an internal thing, but here's my crappy code (threw as much from the lib docs together as possible for testing) that is most definitely organzied wrong for testing just in case it happens to be the cause:

var WasmBoy = require("wasmboy");
var worker = require("worker_threads");
var png = require("pngjs");
var fs = require("fs");
var screen;
 const WasmBoyOptions = {
  headless: true,
  useGbcWhenOptional: true,
    isAudioEnabled: true,
    frameSkip: 1,
    audioBatchProcessing: true,
    timersBatchProcessing: false,
    audioAccumulateSamples: true,
    graphicsBatchProcessing: false,
    graphicsDisableScanlineRendering: false,
    tileRendering: true,
    tileCaching: true,
    gameboyFPSCap: 60,
  updateGraphicsCallback: false,
  updateAudioCallback: false,
  saveStateCallback: false
}

WasmBoy.WasmBoy.config(WasmBoyOptions, screen).then(() => {
   console.log('WasmBoy is configured!');
  screen = WasmBoy.WasmBoy.getCanvas(); 
  WasmBoy.WasmBoy.loadROM("./tobu.gb",);
 WasmBoy.WasmBoy.play();
 screen.pack().pipe(fs.createWriteStream("gameboy.png"));
}).catch(() => {
  console.error('Error Configuring WasmBoy...');
});

[torch2424]

@EuphoricPenguin

Hello!

So after looking at your code, it seems like you get to: console.log('WasmBoy is configured!');. Which means you get into the .then() block.

So it seems you reach the .catch() block from the error: Error Configuring WasmBoy....

So, the reason why you need to set headless: true is because there is no DOM in node. Meaning, you can't access a <canvas> element for output and things. What you will need to do, it run a single frame with: WasmBoy._runWasmExport('executeMultipleFrames', [NUMBER_OF_FRAMES]) Which returns a promise. And then take the graphics buffer and encode to a png for example:

• https://github.com/torch2424/wasmboy/blob/master/test/performance/performance-test.js#L185

• https://github.com/torch2424/wasmboy/blob/master/test/performance/performance-test.js#L189

• https://github.com/torch2424/wasmboy/blob/master/test/common-test.js#L17

• https://github.com/torch2424/wasmboy/blob/master/test/common-test.js#L55

Which you can then generate a png and then do whatever you want with it 😄

[EuphoricPenguin]

npm ERR! code ENOGIT
npm ERR! Error while executing:
npm ERR! undefined ls-remote -h -t https://github.com/torch2424/audiobuffer-to-wav.git
npm ERR!
npm ERR! undefined
npm ERR! No git binary found in $PATH
npm ERR!
npm ERR! Failed using git.
npm ERR! Please check if you have git installed and in your PATH.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Euphoric\AppData\Roaming\npm-cache\_logs\2019-05-07T00_42_07_550Z-debug.log

D:\Documents\jsgbc>

This error just popped up when I tried to npm install the package on my main PC. I didn't know if you knew what was the cause...

[torch2424]

@EuphoricPenguin Seems like you don't have git installed? 🤔

[EuphoricPenguin]

Yeah, sounds right. Probably need the latest version of Node too...

[EuphoricPenguin]

D:\Documents\dpptwo\colorsupport>node index.js
WasmBoy is configured!
Error Configuring WasmBoy...
(node:6356) UnhandledPromiseRejectionWarning: ReferenceError: fetch is not defined
    at fetchROMAsByteArrayTask (D:\Documents\dpptwo\colorsupport\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:2420:19)
    at fetchROMAsByteArray (D:\Documents\dpptwo\colorsupport\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:2447:10)
    at loadROMAndConfigTask (D:\Documents\dpptwo\colorsupport\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:3613:34)
    at loadROMTask (D:\Documents\dpptwo\colorsupport\node_modules\wasmboy\dist\wasmboy.wasm.cjs.js:3652:13)
(node:6356) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:6356) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Here's my current test code. I think it's still mostly incorrect, but I'm not sure if the error above is something I did or the module itself...

var WasmBoy = require("wasmboy");
var worker = require("worker_threads");
var png = require("pngjs");
var fs = require("fs");
var screen;
 const WasmBoyOptions = {
  headless: true,
  useGbcWhenOptional: true,
    isAudioEnabled: true,
    frameSkip: 1,
}

WasmBoy.WasmBoy.config(WasmBoyOptions, screen).then(() => {
   console.log('WasmBoy is configured!');
  screen = WasmBoy.WasmBoy._runWasmExport('executeMultipleFrames', 1);
  WasmBoy.WasmBoy.loadROM("./tcg.gbc");
 WasmBoy.WasmBoy.play();
 screen.pack().pipe(fs.createWriteStream("gameboy.png"));
}).catch(() => {
  console.error('Error Configuring WasmBoy...');
});

[torch2424]

So for your first error, I opened #291 , you are doing the right thing, and I totally made a bad assumption here 😂 (As you can tell, project is still a little rough around the edges for the headless case). But stoked that you brought this up! 😄

And your code looks good so far! But there are a few things:

screen = WasmBoy.WasmBoy._runWasmExport('executeMultipleFrames', 1);.

I think this line returns a promise, and not the actual "image of the screen". You will want to get the image rgb array, and then pass that array to build a png. That being said, I don't know how you plan to pass the image back to discord quite yet (as a file or link), but if you want to do a link, you may want to url encode it? or if you are sending a file, you may want to send a jpg to save some bytes. 😄

WasmBoy.WasmBoy.loadROM("./tcg.gbc");

This is what I opened an issue for. For now, load the rom file using fs, and pass that in 😄

screen.pack().pipe(fs.createWriteStream("gameboy.png"));

I am not entirely sure what you are trying to do here. I may be just lacking in my node knowledge though. What do pack() and pipe() do? Do you have docs? And are you trying to continually output to a file?

Thanks! 😄

[EuphoricPenguin]

Nah, that's some code I'm recycling from gbajs. Probably doesn't work here. I was just looking for errors, but I was stopped by the internal module file reading stuff before that became an issue. I'm trying to get the image as something I can pass to gif-encoder, which I think is identical to what you mention above.

[torch2424]

@EuphoricPenguin Oh rad, that makes sense than :smile:

And yeah, let me know if you need anymore help extracting the image. I think you would honestly be fine just copy pasting my image functions, and then going from there. Every imageDataArray is just an RGB array. So It "should" be flexible :+1:

[EuphoricPenguin]

I think I'm a little confused on how to get the RGB array. In the link you sent above, it looks like you're passing it to another function to do something...

[EuphoricPenguin]

const getImageDataFromFrame = async () => {
  // Get our output frame
  const frameInProgressVideoOutputLocation = await WasmBoy._getWasmConstant('FRAME_LOCATION');
  const frameInProgressMemory = await WasmBoy._getWasmMemorySection(
    frameInProgressVideoOutputLocation,
    frameInProgressVideoOutputLocation + GAMEBOY_CAMERA_HEIGHT * GAMEBOY_CAMERA_WIDTH * 3 + 1
  );

Yeah, the last part. What is that for? You're doing math on the whole array? Sorry, this is new to me. I'm a tad confused lol.

[EuphoricPenguin]

Oh, separate question: When you execute frames, it's just grabbing them while the emulator continues to run, correct? Basically, I don't have to worry about advancing the emulator, will I?

[torch2424]

@EuphoricPenguin

Yeah, the last part. What is that for? You're doing math on the whole array? Sorry, this is new to me. I'm a tad confused lol.

So what that does is slice out (as in make a copy of) a section of the WASM linear memory. The Core (Wasm Module) write out [r, g, b, r, g, b, ....] values in memory, where each r, g, b collectively represents one pixel.

Then after that, I convert the r, g ,b we got from WASM memory into an r, g, b, a array (this is because HTML canvas only accepts r, g, b, a, and I think pngs need this as well).

And yes, this result, imageDataArray is then passed to createImageFromFrame() 😄

Does that help?

When you execute frames, it's just grabbing them while the emulator continues to run, correct? Basically, I don't have to worry about advancing the emulator, will I?

So kind of. It depends on how you want your thing to run. I'll take a look at the other project and try and see what they are doing.

But for now, essentially you will want to do something like (psuedo code incoming):

const runWasmBoyFrame = () => {
  // Runs a single frame of the game. You can set 1 to whatever number
 // you like and it will run that make frames. It does not return the frame
 // only sets the emulator state so you can grab the frame from its memory.
  WasmBoy.WasmBoy._runWasmExport('executeMultipleFrames', 1);

  // Grab the image and stuff using the code I sent you
  const myFrameImage = someFunction();

  // Save it to disk, to discord, or whatever you want here.
}

[EuphoricPenguin]

So... yeah, that first part helped explain, thanks. So, for the second part, my understanding is that it just gets the frame for whenever you run the export function, correct? Some emulators actually require you to manually deal with keeping the clock running so lag would be significant if it couldn't advance itself. Here's an example of an emulator that doesn't "auto advance" frames

[torch2424]

@EuphoricPenguin

So... yeah, that first part helped explain, thanks

You are welcome! 👍

So, for the second part, my understanding is that it just gets the frame for whenever you run the export function, correct?

So it does "get the frame", by running WasmBoy.WasmBoy._runWasmExport('executeMultipleFrames', 1). But since WasmBoy is written in Wasm, the frame is output to wasm memory. All the code I sent you is essentially grabbing the "frame" out of wasm memory, into your JS code being run. 😄

Some emulators actually require you to manually deal with keeping the clock running so lag would be significant if it couldn't advance itself.

So yeah that makes sense though? The reason being, on the server at least, I don't have as much context on what you are doing (running tests, extracting audio, streaming a game, etc...). So the standard 60 frames per second doesn't really make us much sense? But if it does for you, here is a snippet on running code at 60fps. Just run the WasmBoy.WasmBoy._runWasmExport('executeMultipleFrames', 1); and const myFrameImage = someFunction(); in there. 😄 🚀

As a side note, for the browser, WasmBoy has a function called play(). What it does is run frames at 60fps and outputs using DOM Apis. In the browser, we know our output target / DOM API is a <canvas> element, and you are essentially telling WasmBoy, "Hey, feel free to just dump your graphics here on the empty square on my webpage". But on the server there is no "default graphical output". That, or there are multiple options depending on what you want (terminal, streaming, etc...). 🤔

Could you share more of like your end to end goal, and perhaps I can help that way? E.g A user pings your bot on discord, emulator starts up, etc....

And thanks for sharing server boy, that was super rad to see 😄

[EuphoricPenguin]

Basically, with gbajs, I can grab the screen at any point, and the emulator will continue to run. I don't need every frame, just ones when people are actually using the bot. The GIF setup will basically still be the same, just I'll get a few more frames at once.

var screenshotdelay = config.miscsettings.scrnshtdelay / 2;
function  MakeScreen() {
    waitSync(screenshotdelay);
    png = gba.screenshot();
    png.pack().pipe(fs.createWriteStream("GBA.png"));
    waitSync(screenshotdelay);
  }

Here's my function to grab the screen like mentioned above.

I think I understand what you're trying to say, but yeah, I don't need to export every frame if that helps, but I still need the emulator to compute every frame, since I don't know which frames I will end up needing.

[torch2424]

@EuphoricPenguin

If you don't mind me asking, where can I try out your current bot? That way I can understand as an end user what should be happening? 😄

Basically, with gbajs, I can grab the screen at any point, and the emulator will continue to run. I don't need every frame, just ones when people are actually using the bot.

Cool, so yeah, You can wrap the execute frame in that 60fps. And then outside of the 60fps, use the image code I sent you?

That or try something like:

await WasmBoy.play();

// You need a frame
await WasmBoy.pause();
thatImageCodeISentYou();

await WasmBoy.play();

Whenever you need images? 🤔

I don't need to export every frame if that helps, but I still need the emulator to compute every frame, since I don't know which frames I will end up needing.

So you want the emulator to continue playing on the server, while waiting from a response from your client on what to do next? Is that what you mean? 🤔

If so, either try the 60fps code, and the execture frame code (what I reccomend, and probably easiest to maintin), or the WasmBoy.play() idea I just posted above. 😄 👍

Let me know if this helps. If you want the exact smae functionality as GBAJs, with that "screenshot" function, I'm open to open a new issue, or you can feel free to submit a PR?

[EuphoricPenguin]

https://discord.gg/mVpKRwg - Our current code is running here. Not the final result I'm looking for, but it'll give you a better idea of the hopeful end result.

Thanks for being this active/helpful to a newbie btw, very nice of you.

I don't really need an exact replacement, since I'm just working with what I have. I'll worry about making the functions similar on my own later on (building functions I can use with simple logic stitches that on their own work out the differences...)

[torch2424]

@EuphoricPenguin

https://discord.gg/mVpKRwg - Our current code is running here. Not the final result I'm looking for, but it'll give you a better idea of the hopeful end result.

Thanks for the invite! I tried it out, and my mind was blown, this is soooooo rad 😂

Thanks for being this active/helpful to a newbie btw, very nice of you.

Anytime! I'm honestly super stoked on your project (since you first told me a little while ago), and it'll be the first major thing built with the emulator so I am super excited. 😄

Also, I am glad that you are a little new. This type of feedback will definitely help once I start rewriting the docs. Plus, it only strengthens my "how to explain this over text" skills. That being said, since you've been really nice, and you're project is really cool, it definitely helps me wanting to help haha 😂

I don't really need an exact replacement, since I'm just working with what I have. I'll worry about making the functions similar on my own later on (building functions I can use with simple logic stitches that on their own work out the differences...)

Awesome glad to hear! That all sounds like a good idea to me. 👍

Now on the the good parts

So one last question I have is, are you constantly running the emulator because you want that real time clock support? Currently, WasmBoy does not support real time clock unfortunately (it will eventually), but what I'm about to suggest I think is worth it.

So for a simple example, let's say I enter in discord 'up*3'.

Then in my server I would do this:

const discordMessageHandler = (message) => {

  // Parse our message into number of presses, and the button
   const numberOfPresses = pressesParseThingy(message);
  const button = pressesParseThingy(message);

  // Set our joypad to the button pressed.
  // https://github.com/torch2424/wasmboy/wiki/Lib-API#setjoypadstate
  const WasmBoyJoypadParams = [0,0,0,0,0,0 ...];
  if (button === 'up') {
    WasmBoyJoypadParams[someIndex] = 1;
  }
 // and so on and so forth
  // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/apply
  WasmBoy.setJoyPadState.apply(WasmBoyJoypadParams);

  // Run the number of frames we want for that button.
  // Decide how many frames per button press you want
  // Lets just say, a button is press once per second.
  for (let i = 0; i < numberOfPresses; i++) {
    WasmBoy._runWasmExport('executeMultipleFrames', 60);
  }

  // TODO: If it was a hold, just call WasmBoy.play().
  // Then call WamBoy.pause() whenever a new command comes in, or `letgo`

  // TODO: Grab the screenshot here with all the image memory grabbing code stuff
  // I sent you

  sendImageToDiscordOrSomething();

  // BONUS: Take a save state here, and save it as a disk or something.
  // https://github.com/torch2424/wasmboy/wiki/Lib-API#savestate
  const saveState = await WasBoy.saveState();

  // Done!

}

With this approach, you'll also not be pegging your server constantly running an emulator (not a huge burden on CPU I know, but why not save some), and you could do this serverless if you wanted? 🤔 Which is why I suggest I think this is worth sacrificing real time clock support, because your server wont be constantly running the emulator.

This all being said, while writing this, the // BONUS thing I wrote made me realize how I could run WasmBoy Serverless, as long as I write the current saveState to disk 🤔 Which is another good demo to write up once I get around to writing up some server side demos.

You going to make your project open source by the way? I'd love to feature it on the README once it's getting close to done (no worries if not!).

[torch2424]

Also, I thought more About what I said about WasmBoy.play(), It should totally work without <canvas>, and if you use that image code I sent you about grabbing the memory, You should be good to go.

The only reason why I was skeptical, was you may get some vsync issues, or like half drawn frames. But if the gbajs isn't waiting for the frame to fully drawn, then you may have already noticied it, and been okay with it 😄

So yeah, let me know if you liked the method I sent you. If not, I can give you a snipper on how I'd do it with it always running 👍

[EuphoricPenguin]

Yeah, the style of the emulator is to run nonstop. I guess pausing might be a decent option, but since I don't think there's some kind of substitute in GBAjs, idk if it would be really nessecasry. What I meant by clock is the idea that frames are run at the CPU clock speed without me having to worry about keeping some resemblance of that correct.

[EuphoricPenguin]

RTC is underutilized in both games, and in the GBA titles the one feature it is used for (growing berries) breaks after 100 hours anyways.

[torch2424]

Yeah, the style of the emulator is to run nonstop. I guess pausing might be a decent option, but since I don't think there's some kind of substitute in GBAjs, idk if it would be really necessary. What I meant by clock is the idea that frames are run at the CPU clock speed without me having to worry about keeping some resemblance of that correct.

Well if that is the case than, and you really want it constantly running, then I think the following code should work :smile: :

// Start the emulator
WasmBoy.play();

// Some time later

// Take a screenshot
await WasmBoy.pause()
const image = thatImageCodeISentYou();
await WasmBoy.play();

I opened #293 for a direct screenshot function. But then a lot of questins come to mind, like should it export png vs. jpg, and all of that. :thinking:

RTC is underutilized in both games, and in the GBA titles the one feature it is used for (growing berries) breaks after 100 hours anyways.

Definitely, agreed :smile:

[alex-red]

Hey guys, sorry to bump this old discussion, but did anyone figure out a way to properly input a command, wait for it to fully finish, then pause for screenshot?

My code:

    // actual input
    WasmBoy.setJoypadState(newState);
    await WasmBoy._runWasmExport('executeMultipleFrames', [60]);
    await WasmBoy.play();
    // reset input - looks like it holds the button down otherwise
    WasmBoy.setJoypadState(WasmBoyJoypadState);

    await WasmBoy.pause();
    const image = await getScreenshot();
    await saveState();

My problem is that it doesnt seem to fully finish rendering the button input so we have to do it multiple times, I've also messed around with frameskip and the frame number but that makes it skip ahead.

@torch2424 awesome work btw, and thanks for the headless support

[torch2424]

@alex-red

Hey guys, sorry to bump this old discussion, but did anyone figure out a way to properly input a command, wait for it to fully finish, then pause for screenshot?

No worries on the bump! :smile:

My code: ... My problem is that it doesnt seem to fully finish rendering the button input so we have to do it multiple times, I've also messed around with frameskip and the frame number but that makes it skip ahead.

Oh! Looking at your code, yeah, it's because the API is confusing, my apologies on that. I haven't had much time to work on WasmBoy, as I shifted a lot of my time to working on AssemblyScript itself :upside_down_face: That being said, I do have a personal Code TODO list. and I reeaaalllyyyy want to pick this project back up. As I actually use WasmBoy to play some games for fun from time to time :joy:

But, essentially, you don't want to run .play() as that is essentially just going to just play the game in a setInterval at a normal 60fps. What you want to do is:

// Set the input, yes, this will be held down until you set the new input state
WasmBoy.setJoypadState(newState);

// Run the WasmBoy for 60 frames. This does the actual exectuion of the frames
await WasmBoy._runWasmExport('executeMultipleFrames', [60]);

// No need to play or pause here, see my explanation above
// This would just keep the emulator running at 60fps, rather than running a specific number of frames
// await WasmBoy.play();
// await WasmBoy.pause();

// Reset the input
WasmBoy.setJoypadState(WasmBoyJoypadState);

// Get the screenshot
const image = await getScreenshot();

// Save state returns an object if I'm not mistaken, and then you can save it however you like somewhere
const saveState = await saveState();

awesome work btw, and thanks for the headless support

Thank you! Glad that you like it! And I'm stoked the headless support works for ya! :smile: :+1:

---

Let me know if this works for you! Thank you! :smile:

[alex-red]

@torch2424 Thanks for the help and quick reply :smile:

Unfortunately it seems like the same issue, the game does run for 60 frames but its not enough to process the entire action. (trying to run a pokemon rom btw):

So sending "UP" only works after multiple tries, so I'm thinking it is not running enough frames. Maybe thats not the issue though because even if i change # of frames to 300 or put it in a for-loop it doesn't always seem to run all the way through.

[torch2424]

@alex-red You are welcome! :smile:

So sending "UP" only works after multiple tries, so I'm thinking it is not running enough frames. Maybe thats not the issue though because even if i change # of frames to 300 or put it in a for-loop it doesn't always seem to run all the way through.

Hmmmm :thinking: Can you log out the object you are sending? This is the function that handles the object and sends along the appropriate array to the webworker that then talks to the wasm module: https://github.com/torch2424/wasmboy/blob/master/lib/controller/controller.js#L49

Also, I noticed you are doing a save state, are you trying to do this all in one go? Or are you saving and reloading the state? :thinking:

[alex-red]

Sure, I'm just sending this:

const WasmBoyJoypadState = {
  UP: false,
  RIGHT: false,
  DOWN: false,
  LEFT: false,
  A: false,
  B: false,
  SELECT: false,
  START: false,
}

With the corresponding input set to true.

And yea I'm saving it at the same time as sending the input and getting the screenshot.

[torch2424]

Sure, I'm just sending this:

Hmmm that seems correct :thinking:

And yea I'm saving it at the same time as sending the input and getting the screenshot.

Oh I meant are you doing something like:

// Set the input, yes, this will be held down until you set the new input state
// Set the input to down
WasmBoy.setJoypadState(downState);

// Run the WasmBoy for 60 frames. This does the actual exectuion of the frames
await WasmBoy._runWasmExport('executeMultipleFrames', [60]);

// Reset the input
WasmBoy.setJoypadState(resetState);

// Get the screenshot
const image = await getScreenshot();

// Save state returns an object if I'm not mistaken, and then you can save it however you like somewhere
const saveState = await saveState();

// .... Some time passes ....

await loadState(saveState);

// Set the input, yes, this will be held down until you set the new input state
// Set the input to up
WasmBoy.setJoypadState(upState);

// Run the WasmBoy for 60 frames. This does the actual exectuion of the frames
await WasmBoy._runWasmExport('executeMultipleFrames', [60]);

// Reset the input
WasmBoy.setJoypadState(resetState);

// Get the screenshot
const image = await getScreenshot();

Like, are you reloading the save state? Or are you running it all at once:

// Set the input, yes, this will be held down until you set the new input state
// Set the input to down
WasmBoy.setJoypadState(downState);

// Run the WasmBoy for 60 frames. This does the actual exectuion of the frames
await WasmBoy._runWasmExport('executeMultipleFrames', [60]);

// Set the input to up
WasmBoy.setJoypadState(upState);

// Run the WasmBoy for 60 frames. This does the actual exectuion of the frames
await WasmBoy._runWasmExport('executeMultipleFrames', [60]);

// Reset the input
WasmBoy.setJoypadState(WasmBoyJoypadState);

// Get the screenshot
const image = await getScreenshot();

// Save state returns an object if I'm not mistaken, and then you can save it however you like somewhere
const saveState = await saveState();

Or better yet. Is your code on Github? I think I could understand better if it is, or if you can post a gist of everything or something?

Thanks! :smile: :+1:

[alex-red]

It is the latter :) I've put a gist here with the relevant calls: https://gist.github.com/alex-red/1221a55e79eeea9762695d0639c7493b

The server call is at the bottom, let me know if you need anything else

[torch2424]

@alex-red Ah! I bet I found it! :smile:

So it seems like you are running .play() in your init function: https://gist.github.com/alex-red/1221a55e79eeea9762695d0639c7493b#file-work-js-L175

This will start running the emulator in a loop, and continually updating the joypad (thus, cancelling out something you may be setting).

Can you try commenting out that .play() call? :smile: I think it's a race condition between when you are handling the call, and the running wasmboy. Which would make sense with the behavior you described.

Again, my apologies for the API being so confusing. API design was on my roadmap, just haven't had the time haha! :joy:

Let me know if it works! Thanks! :smile: :+1:

[alex-red]

@torch2424

That makes sense, unfortunately no luck though -- it looks like it isn't even advancing the frame without .play().

No worries though, I'll tinker around and see, otherwise I'll move on to just using it non-headless :)

[torch2424]

@alex-red So I went ahead and added a test, and I totally got it working :smile:

https://github.com/torch2424/wasmboy/tree/save-state-per-frame/test/integration

See the headless-simple.js test. Pretty much you can copy paste the stuff in the it() block and it should work?

Also, see the .png files, as they react to the input to what I am doing headless.

And I never call .play() :smile:

I hope that helps! :smile: :+1:

[torch2424]

Oops! I just noticed though. If you are trying to execute frame by frame, and save / load states. Because we're using web workers, I'll need to add something for a headless save state. As the current state wouldn't have been passed back yet by the worker without running play().

I really gotta clean up this API :joy:

[alex-red]

@torch2424 Wow thanks so much, that worked!

No idea why I didnt think to try:

    WasmBoy.setJoypadState(newState);
    await WasmBoy._runWasmExport('executeMultipleFrames', [1]);
    WasmBoy.setJoypadState(WasmBoyJoypadState);

I also realized that the sharp lib i was using to double the image size was caching aggressively :sweat_smile:, so disabling the cache removed any of the "stuttering". And yes I just realized the saving stopped working haha

[torch2424]

@alex-red You are welcome! :smile: I am glad that it works!

I also realized that the sharp lib i was using to double the image size was caching aggressively sweat_smile, so disabling the cache removed any of the "stuttering".

Oh nice! I am glad the stuttering got fixed on your end as well! :smile:

And yes I just realized the saving stopped working haha

So! I actually just fixed it! I happen to be working on a demo for work actually haha! And I also ran into this. So I'll publish a new version tommorrow-ish, and it's on the branch save-state-per-frame. Unfortunately, it'll be breaking for older save state versions, but, it works now haha! :joy: