Idea - integrate in gdevelop

[blurymind]

gdevelop is an excellent open source game engine for 2d games. It has a great community and developers and shares a lot of traits with piskel:

• The new Ide is written in javascrit and nodejs html5
• multiplatform and runs on the same platforms piskel runs on

While gdevelop can be used to set animation frames on a sprite, it can not create or edit frames. This is where this idea comes in - package gdevelop with piskel included in it's own independent subfolder - with some intermediate code to get gdevelop's newIde to communicate with piskel - use it to create new frames and edit the pixels of existing frames.

I wonder how feasible this is.

In any case I posted this idea at gdevelop's tracker too. https://github.com/4ian/GD/issues/470

These two projects can really benefit from a partnership imo 👍

Piskel can that way also expand its userbase by expanding to another complimentary platform - a game engine.

[blurymind]

Hi, I might need some help here.

Does piskel have a DOM function that lets us open multiple file paths as frames of an animation? I have the image paths. https://github.com/piskelapp/piskel/pull/620/files#diff-fdf1dc8439b891625c765d6d0370f9c0 I am aware of the drag and drop functionality in piskel, but not sute how to simulate it artificially

[blurymind]

How about:

 pskl.service.FileDropperService.prototype.onFileDrop (
 {preventDefault: function () {} ,stopPropagation: function () {} ,clientX:5,clientY:5,dataTransfer:{files:[img,img]}}
)

piskel-packaged-min-2018-04-27-08-20.js:5 Uncaught TypeError: Failed to execute 'readAsDataURL' on 'FileReader': parameter 1 is not of type 'Blob'.
    at Object.readFile (piskel-packaged-min-2018-04-27-08-20.js:5)
    at Object.readImageFile (piskel-packaged-min-2018-04-27-08-20.js:5)
    at Object.a.FileDropperService.onFileDrop (piskel-packaged-min-2018-04-27-08-20.js:17)
    at <anonymous>:1:102 

I seem to be doing something wrong here :( How do you use this function? What image format is that function taking? Its not image path, so what is it

[juliandescottes]

The APIs you are looking at rely on the File API: https://developer.mozilla.org/en-US/docs/Web/API/File

You can get File objects from the clipboard, from a drag and drop, or from a file input.

I don't know how you plan to integrate Piskel with gdevelop and how gdevelop can communicate with Piskel? If Piskel is supposed to be loaded in an iframe in the same domain as gdevelop, then you can try to forward the images directly as Image objects or using the image data and rely on pskl.utils.FrameUtils.createFromImage.

[blurymind]

@juliandescottes I have it now embedded into the gdevelop elecron app - keeping the vanilla version from this repository. I can send signals to its window, but not yet sure what the best way to load my image paths into piskel as animation. Basically I have a sequence of images and I have put them inside an array.

What function can I use to load it?

Trying this: pskl.service.ImportService.prototype.createPiskelFromImages_(imageFrames,"foo",500,500)

gives me the following error:

Uncaught TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'
    at Object.resize (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:5:23911)
    at http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:12005
    at Array.map (native)
    at Object.a.ImportService.createFramesFromImages_ (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:11959)
    at Object.a.ImportService.createPiskelFromImages_ (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:11699)
    at EventEmitter.ipcRenderer.on (http://localhost:3000/External/Piskel/index.html:24:44)
    at emitTwo (events.js:106:13)
    at EventEmitter.emit (events.js:194:7)
resize @ piskel-packaged-min-2018-04-27-08-20.js:5
(anonymous) @ piskel-packaged-min-2018-04-27-08-20.js:17
a.ImportService.createFramesFromImages_ @ piskel-packaged-min-2018-04-27-08-20.js:17
a.ImportService.createPiskelFromImages_ @ piskel-packaged-min-2018-04-27-08-20.js:17
ipcRenderer.on @ index.html:24
emitTwo @ events.js:106
emit @ events.js:194
piskel-packaged-min-2018-04-27-08-20.js:5 Uncaught TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'
    at Object.resize (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:5:23911)
    at http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:12005
    at Array.map (native)
    at Object.a.ImportService.createFramesFromImages_ (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:11959)
    at Object.a.ImportService.createPiskelFromImages_ (http://localhost:3000/External/Piskel/editor/js/piskel-packaged-min-2018-04-27-08-20.js:17:11699)
    at EventEmitter.ipcRenderer.on (http://localhost:3000/External/Piskel/index.html:24:44)
    at emitTwo (events.js:106:13)
    at EventEmitter.emit (events.js:194:7)

This is the function I use to construct the image array:

    var imageFrames = []; /// first collect the images to edit
    for (var i = 0; i < direction.getSpritesCount(); i++) {
      var spriteImagePath = resourcesLoader.getResourceFullUrl(project, direction.getSprite(i).getImageName());
      var importedImage = new Image();
      importedImage.src = spriteImagePath; 
      imageFrames.push(importedImage);
    }

Will commit the changes to my fork of Gdevelop and share a link later today :) Any tips are much appreciated. I think that this will lead to more people using piskel and gdevelop - as the two seem to be a really good fit and I am keeping piskel vanilla

[blurymind]

@juliandescottes I think I am getting closer. can you help me give piskel the correct image data please.

Here are the relevant parts sending the data to it https://github.com/blurymind/GD/blob/master/newIDE/app/public/External/Piskel/index.html#L40 and https://github.com/blurymind/GD/blob/master/newIDE/app/src/ObjectEditor/Editors/SpriteEditor/SpritesList.js#L168

I get an error about the file not being the correct format :(

[juliandescottes]

The function you are using, createPiskelFromImages_, takes an array of Image objects. Image objects (https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/Image) or Canvas objects can work. In fact anything that can be used with CanvasRenderingContext2D/drawImage is ok.

In your current implementation imageFrames contains objects returned by localFileSystem.readFile(). Looking at the code it returns:

      var contents = fs.readFileSync(file);
      return contents.toString();

which is your image file as a String, and won't help here.

Some other things you could try:

1 - I see you tried creating Images and setting the src to local file path for the image. Seems like this could be a good idea, but I don't know how the security model works in electron apps. Maybe the image can't be used with the canvas for cross origin reasons. You should also normally wait until the load event on the image before proceeding.

2 - Try to read the content of the image file as base 64 (found https://stackoverflow.com/questions/24523532/how-do-i-convert-an-image-to-a-base64-encoded-data-url-in-sails-js-or-generally which seems to answer that but haven't looked into this) and then create an Image with a src using this base64 string. That's basically what we do in Piskel when the user imports Images as files, look in FileUtils::readImageFile for the general idea. Maybe you can even use readImageFile() here?

And generally, if you are unsure about how something works, you should have the development version of Piskel running (grunt play) and then use your browser's devtools to set breakpoints in the functions that interest you. See what kind of arguments are passed, how they are called etc ...

[blurymind]

@juliandescottes Thank you for the help. I think that I am finally getting closer! I get a base64 string that is actually valid if tested with: https://www.base64decode.org/ inside an array, so at least the image data should be correct now

Piskel gives me this error now: Uncaught Piskel.fromLayers expects array of non empty pskl.model.Layer as first argument

It seems to be missing an argument

Btw I did a commit here: https://github.com/blurymind/GD/commit/b60d470a76d163dc236e49cc2cc28a217c338f75

[juliandescottes]

The onload event on images is asynchronous, so the array you pass to to createPiskelFromImages_ is empty. You need to wait until all the images are loaded before you call this method.

[4ian]

(GDevelop developer here :))

@blurymind The index.html that is public/External/Piskel/index.html should have access to npm packages installed for Electron (https://github.com/4ian/GD/blob/master/newIDE/electron-app/app/package.json), in particular there is async.

Here I think you could use async, in particular each method: https://caolan.github.io/async/docs.html#each to loop over the images and call pskl.service.ImportService.prototype.createPiskelFromImages_ only when all images have been loaded properly and pushed to imageData array.

[4ian]

@blurymind Something like this (not tested, just wrote this in 2 minutes based on what you did, also could be improved by using async.map but that should be enough to get you running):

const async = require('async');

async.each(imageFrames, function(imagePath, callback) {
    label.innerHTML = "loop "+ String(i);
    var image64 = new Image();
    image64.onload = function () {
      image64.src = base64_encode(imagePath);
      console.log(image64.src);
      imageData.push(image64);
      callback();
    }
}, function(err) {
    console.log(imageData);
    pskl.service.ImportService.prototype.createPiskelFromImages_(imageData, "foo",55, 55, false);
});

[blurymind]

Thank you both for the help! :D Getting help from both software authors is great honor for me - as I love both projects!

@4ian for some odd reason image64.onload never actually fires and the array stays empty. Removing it leads the array filling up So far I have this:

    var imageData = [];      
    async.each(imageFrames, function(file, callback) {
    // Perform operation on file here.
    console.log('Processing file ' + file);
    var image64 = new Image();
    image64.src = base64_encode(file);
    imageData.push(image64);
    console.log(String(imageData.length));
    callback();
    }, function(err) {
  // if any of the file processing produced an error, err would equal that error
    if( err ) {
      // One of the iterations produced an error.All processing will now stop.
      console.log('A file failed to process');
    } else {
      console.log('All files have been processed successfully');
      console.log(imageData);
      pskl.service.ImportService.prototype.createPiskelFromImages_(imageData, "foo",55, 55, false);
    }
  });

It prints out the filled array, then it should trigger piskel to load the images- but piskel loads nothing. @juliandescottes This time around though piskel does not spill any error message, but nothing happens. No images get loaded, even though the printed array contains them and has been populated the way @4ian suggested

Btw I made a commit here: https://github.com/blurymind/GD/commit/41b00dcc3595585d1bd3a1a01a732bd2f6b1c775

I think we are getting closer! :D

[blurymind]

I noticed that Electron is appending to the start of image src the localhost path: Could that be the reason nothing gets loaded? If so, I need to figure out how to make it not do that.

Edit, Appending "data:image/png;base64," to it solves the url having localhost there issue, but piskel still loads nothing without making a peep as to why

made commit https://github.com/blurymind/GD/commit/03a08297ab84b7af36e3c28e40375b6b2c35b70f

[4ian]

Can you copy/paste the whole src of the image (from data:image/png;base64, until the end without forgetting any character) in this jsfiddle: https://jsfiddle.net/casiano/Xadvz/

And see if the image is displayed? If no, then it means that the base64 image is malformed for some reason. Otherwise, there must be a problem while loading the image in Piskel.

[blurymind]

@4ian I tried it, the string is valid

Piskel just doesnt seem to do anything after I load the array into createPiskelFromImages_ now. There are no errors, so its hard to determine what to do next

[4ian]

Not sure what is wrong. Did you manage to use createPiskelFromImages_ successfully at one time, for example using an hardcoded base64 image? Do you have a working example somewhere?

[juliandescottes]

@blurymind the string really needs to start with data:image/png;base64, otherwise the browser/electron will think it is a relative path.

Try to prefix your data urls as follows image64.src = 'data:image/png;base64,' + base64_encode(file);

[4ian]

I think he already did, according to his latest commit: https://github.com/blurymind/GD/commit/03a08297ab84b7af36e3c28e40375b6b2c35b70f#diff-bf9a6bff2e41cedcd1b65b9fd5849fe4R21

@blurymind Can you console.log and paste there (or on some website like pastebin.com) the entire parameters passed to createPiskelFromImages_? (without changing or omitting anything)

[juliandescottes]

Ah thanks I missed the update at the end of the comment.

So the issue is that this is still not waiting for the onload. Here is the basic pattern you need to do:

img.onload = function () { ... some code you want to execute }
img.src = base64src;

The onload event will be triggered by setting src on the image, so you cannot have it inside of the onload callback.

Based on the example from @4ian with the fix mentioned above:

const async = require('async');

async.each(imageFrames, function(imagePath, callback) {
    var image64 = new Image();
    image64.onload = function () {
      imageData.push(image64);
      callback();
    }
    // This is the important part! Set src, otherwise the `onload` will not fire
    image64.src = base64_encode(imagePath);
    console.log(image64.src);
}, function(err) {
    console.log(imageData);
    pskl.service.ImportService.prototype.createPiskelFromImages_(imageData, "foo",55, 55, false);
});

[blurymind]

Thank you both :) I am going to try this tomorrow... couldnt get to my laptop today, and now I am super eager to get the images in piskel

On Fri, 4 May 2018 20:31 Julian Descottes, notifications@github.com wrote:

Ah thanks I missed the update at the end of the comment.

So the issue is that this is still not waiting for the onload. Here is the basic pattern you need to do:

img.onload = function () { ... some code you want to execute } img.src = base64src;

The onload event will be triggered by setting src on the image, so you cannot have it inside of the onload callback.

Based on the example from @4ian https://github.com/4ian with the fix mentioned above:

const async = require('async');

async.each(imageFrames, function(imagePath, callback) { label.innerHTML = "loop "+ String(i); var image64 = new Image(); image64.onload = function () { console.log(image64.src); imageData.push(image64); callback(); } // This is the important part! Set src, otherwise the onload will not fire image64.src = base64encode(imagePath); }, function(err) { console.log(imageData); pskl.service.ImportService.prototype.createPiskelFromImages(imageData, "foo",55, 55, false); });

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/piskelapp/piskel/issues/805#issuecomment-386709701, or mute the thread https://github.com/notifications/unsubscribe-auth/AGMbVdyqci-O8GaQHkVPOhYLtqjfe6dFks5tvKyrgaJpZM4TX-YP .

[blurymind]

@juliandescottes Thank you for the code snippet. Unfortunately it still doesn't do anything. Basically Piskel's window shows up, all of the data collected from gdevelop is sent to it and printed in the console, there are no errors- but nothing happens

I made a git commit here: https://github.com/blurymind/GD/commit/c2fba30ad4b7c75720adc7262e21bf3ddeb0f008

Are we missing something? For a bit I thought that the function createPiskelFromImages_ constructs a piskel object and another one should be used to load it? Is this the best function to use for loading an image sequence? Maybe some other part of my code needs work?

Btw I am pre-loading piskel at the moment, and when called simply show it. But I can change that if its causing possible issues. I create the electron window with piskel in main.js and give it security clearance to have access to the local files - right when the user starts gdevelop. https://github.com/blurymind/GD/blob/master/newIDE/electron-app/app/main.js#L67 Piskel is then hidden until initiated by the user. When its window is closed- it doesnt get destroyed- just hidden again

[juliandescottes]

Hey! That's better, but we now need to load the Piskel instance created by this method.

If you search for createPiskelFromImages_, you can see we get a return value from the method and use it. The method just builds an instance of Piskel that can be used in the application but it doesn't replace the current animation.

You need to call piskelController.setPiskel. Have a look at how this is already used in Piskel https://github.com/piskelapp/piskel/search?utf8=%E2%9C%93&q=piskelController.setPiskel&type=

You can get the current piskelController with window.pskl.app.piskelController.

Is this the best function to use for loading an image sequence?

With pskl.service.ImportService.prototype.createPiskelFromImages_, we are reaching into the prototype of a class to get a method that is normally supposed to be called on an instance of ImportService. It still happens to work when called like this, but this would be better extracted in a util (going to src/utils)

[blurymind]

@juliandescottes this function is gold! :D Thanks to you and @4ian we are finally in business now. Piskel is loading gdevelop's sprites!

commit here: https://github.com/blurymind/GD/commit/8d0122441afcbd5430505602b07e66f1e840a656

I now need to create a button that will save the changes back to gdevelop and hide Piskel. If there are any new frames made in piskel, gdevelop will need to load them automatically too.

I need to store their original paths to the frames metadata. So when exporting I can overwrite the original images.

In any case, we are getting closer now to full integration. Both apps will work as one 👍

[4ian]

@blurymind Good job :) If think you might be able to listen for the closing of the Piskel window (on('close', ...) and at this moment send an event to the window to dump the content back to GDevelop (as base64 I guess?). Then in GDevelop save from base64 to original files (or new files if needed) (a few hints here: https://github.com/4ian/GD/issues/470#issuecomment-385392169 - might not be perfectly accurate so feel free to experiment)

[blurymind]

@juliandescottes I need some way of storing the image's original path inside the frame created from it. This is something that is somewhat related to #733

The idea is to be able to overwrite any files that were imported from gdevelop when saving and create new files for frames created in piskel. To do that- I need to be able to store the paths somewhere in the frame metadata and retrieve it when saving the image changes. Changing the order of frames in piskel should also reflect changing the order of paths I am retrieving

I am still figuring out how to itterate through frames data. Perhaps try the export controller? https://github.com/piskelapp/piskel/blob/d1156954ca8a929be51ff69a88d6d70e25648d47/src/js/controller/settings/exportimage/PngExportController.js

and/or getFrameCount + getFrameAt to loop through them? https://github.com/piskelapp/piskel/blob/d1156954ca8a929be51ff69a88d6d70e25648d47/src/js/controller/piskel/PiskelController.js#L113 https://github.com/piskelapp/piskel/blob/d1156954ca8a929be51ff69a88d6d70e25648d47/src/js/controller/piskel/PiskelController.js#L83

I wonder how to best approach this

[blurymind]

Ok, a little update: I wrote some javascript that attaches a new 'originalPath' string to each frame object when piskel opens. This way I can keep track of which frame overwrites which file - even if you change their order. Any undefined frames will be automatically detected too. This was easier than I thought and it doesnt make any changes to piskel's vanilla source code- as it all happens in the intermediate index file

Next step now is to write the files back to their original paths when the save button is pressed or window closed!

EDIT: I am now writing the files to their original paths!

Will do a comit later today :D

All thats left now is tell gdevelop reload the frames in the new order they might be in - which is required on the gdevelop side of the code!

[4ian]

This was easier than I thought and it doesnt make any changes to piskel's vanilla source code- as it all happens in the intermediate index file

Ah that's perfect! Good job, look forward to see the commit :)

[blurymind]

commit here https://github.com/blurymind/GD/commit/fe261af9abebdcc5a33bfe54bc60b358191bf92b

Its still WIP :) Need to come up with a better function to create unique file names for frames created in piskel and not imported from gdevelop

[blurymind]

@juliandescottes and @4ian Thank you immensely for the help! After a bit more than 11 days of studying both Piskel and Gdevelop's code - piskel can now be bundled with gdevelop's newIde -enabling all gdevelop users to employ it for their pixel art needs.

Pull request is here: https://github.com/4ian/GD/pull/493

It needs some cleaning up and bug testing of course

[juliandescottes]

Wow that's great @blurymind ! Really impressed with what you achieved here!

[blurymind]

@juliandescottes thank you. I wouldnt have done it without your help :) @4ian did some refactoring on some of the parts of the code- but its now fully functional and ready for gdevelop's next release. It warms my heart to see two great open source projects come together

[blurymind]

Piskel Is now a part of Gdevelop 👍 I can close this now.

You can cross promote the two projects if you want to. Piskel's website can put gdevelop as one of the engines that get shipped with piskel included - and Gdevelop's website could likewise put a link to Piskel and say that its pixel editing capabilities are powered by Piskel.

It would be worth noting that because Piskel is designed so well - it wasn't that difficult to bundle it with gdevelop. I could easily inject new buttons into its header without touching the source code for example. It allowed me to attach new variables to frames without touching anything inside src. It's really good for integration