Help with registering a new type of a resource (DEV question)

[blurymind]

Description

I need to figure out how to allow gdevelop to add other filetypes than png as a resource to any project.

This is needed for two things:

• One is for pskl files - so we can preserve layer data of animations automatically when using piskel to create/edit them when editing images with piskel - also store a pskl file
• Another is for Yarn story files - In which case I want to be able to store these text files as a project resource that the newIde can create and edit (via Yarn) through a specialized yarn story game object or behavior

The part that is confusing to me is- can one register the new resource file type with the newIDE or does it also need to be registered with the runtime (gdevelop.js) ?

What are the files in src that I need to be looking at in order to make this work

Construct2 for example allows to add json files as resources and later on read from them. In gdevelop - it looks like you have to manually type the filepath of any structured file as a string in order to load it via the event sheet

My plan is to first try to get this working for piskel files - and if succesful - use the knowledge to apply it to yarn files (a more complex task). So before working on Yarn story parsing - improve the current piskel workflow

[4ian]

Resources are part of GDCore. Resource: https://github.com/4ian/GD/blob/master/Core/GDCore/Project/ResourcesManager.h#L33 ImageResource: https://github.com/4ian/GD/blob/master/Core/GDCore/Project/ResourcesManager.h#L165 AudioResource: https://github.com/4ian/GD/blob/master/Core/GDCore/Project/ResourcesManager.h#L236

I'm unsure if adding specific resources for Piskel/Yarn is the good way to go. I would prefer to see if we can add support for "FileResource" or "JsonResource" or anything like this.

In gdevelop - it looks like you have to manually type the filepath of any structured file

"Files" in storage actions are a term coming from the "past", i.e: the XML files that were used in GD4 for native games. In GDJS, these "files" are stored as JSON in browser local storage.

To dig a bit more, what is exactly stored in Piskel file? Is that a JSON file?

[blurymind]

@4ian FileResource should be sufficient for both of these things imo. All I really need to keep a track of is file paths relative to project root- regardless of their extension.

In query to your question about piskel- its just a json file- so is the yarn file. They are both ordinary json text files that have specific structures.

All I really want to do here is simplify workflow in the newIde - alleviating the user from the need to manually type paths in the event sheet in order to set up yarn data. Also allowing the ide to create/edit it easily in aid of the project without leaving gdevelop or clumsily navigating to files every time they need to be edited. The idea is -similarly to piskel - be able to create a yarn document from within gdevelop and edit it inside gdevelop (via yarn). Having it stored in the project folder as a resource is beneficial to future game localization, but also allows better tracking in github.

For piskel - I need a way to associate pskl files with animations, so when the user saves an animation or a resource in piskel, the layer data is preserved in a file that is kept somewhere within the project folder - along with the flattened frames

[blurymind]

Would be quite nice if we could add json files as resources in the editor. Then simply select them from a searchable drop down list in the event sheet - where applicable. That would make them much more user friendly to use for development or as data files.

The "FileResource" type in the newIde could also get us closer to https://github.com/4ian/GD/issues/454 -for attaching .js files to gdevelop projects

[4ian]

I've created a card here: https://trello.com/c/dT5esfRc/122-add-supporting-for-embedding-json-files-in-a-games-and-reading-them-from-events to track this feature.

JSON files would be useful if they can be read from events (useful for quiz games, I got another user asking me about this - or translations for games supporting mulitple languages).

[blurymind]

Thank you @4ian , it will greatly simplify adding both of the two things :) Piskel is probably the more important one really, although the Yarn integration would also be awesome to have. I noticed that gdevelop has a way to make it easier to track ingame items- adding a dialogue engine and dialogue editor would make it a killer adventure game engine imo.

Btw I am attaching an example layered piskel file. As you can see, the extension is .piskel - but when opened with a text editor- it is a json file really. The same is with yarn files.

New Piskel-20180814-002115.zip

I need them to keep their extensions, but to allow the newIde to keep track of them as project resources(it does it for png images) and parse them as json files when required

[blurymind]

Another addon I want to make that uses json is for this library: http://depts.washington.edu/madlab/proj/dollar/pdollar.html Json is required to store pre-recorded gestures. The goal is to some day recreate this game mechanic in gdevelop: https://www.google.com/doodles/halloween-2016 :} I have made this as an addon for godot btw, but it's implementation is in 1$ not p$

I exposed some of the remaining yarn data in bondage.js now, which will come in useful later for check tag/title conditions when its used as an addon

[4ian]

I'm not sure what's your usecase of JSON here? Could you explain how they will be useful, how they will be used and how the user of GDevelop would use them with your addon?

[blurymind]

In the case of a gesture extension - for the capturing of gestures and storing them. A gesture is a point cloud that forms a stroke

In this case I guess that the only reason is make the extension easier to use. I could technically just have the point data in the js file. Some is already is included in the js implementation http://depts.washington.edu/madlab/proj/dollar/pdollar.js

It makes more sense for the Yarn dialogue addon that would use bondage.js - where the json file contains all the game dialogue

[zatsme]

$p support would be amazing for gdevelop, for mobile games gesture support is essential 👍🏻

[blurymind]

@zatsme basic swipe direction detection can probably already be implemented via the event sheet, but for more complicated shapes this is an awesome library :)

[zatsme]

Yes, but using the library will be easier and probably more accurate. 😁

[4ian]

Do you have an idea of how to expose the library to the users in a way that would make it easy to use? :) Like, should there be some kind of interface to author the gestures, or even a list of predefined gestures?

[blurymind]

@4ian in the case of how I made it in godot - I included a gesture recorder with the addon. https://blurymind.itch.io/1-gesture-recognition-addon-for-godot

This can be done for gdevelop in the form of the example project for the extension. The gesture recorder can save/load strokes from/to a json file- which the addon can be set to load in the game :)

But json files can not be used as a resource yet - so that feature and a gesture recorder will not be included

[4ian]

I see! It's an interesting and really good idea to have the gesture recorder as part of the example. (btw, your addon for godot looks nice, good job! :))

I think it's still possible to start with a set of predefined set of gestures and by then I'll maybe have the time to see if we can include external JSON files.

It seems that your gesture recorder would also need to save local files - which is not possible for now. So this is a different addition to do too (i.e: dealing with local files for storage) that would work only for games exported on Windows/mac/Linux (i.e: with Electron). Not sure exactly what's the best way to integrate this to the existing storage actions. 🤔

[blurymind]

Ah thank you :)

can we not save game states as json files on android? Saving to mobile is something to tackle with image resources too - the web version too. It's further down

An alternative solution is to display the result as a long string that the developer can copy - if it fails to save

In any case, a custom gesture editor would be nice, and shouldnt be too hard to implement - even as a web version. But its not as important as being able to use json files as a resource on any platform.

This affects saving piskel layers and dealing with dialogue data - which are more important to tackle atm imo

[blurymind]

Btw if we use piskel's json data to store image data, instead of actual bitmaps we use now- we can technically store it in the json project file of the game instead of separate json file resources. That way when the user downloads their project file from the web version - it will contain all of their animations with layers and everything! That would solve that.

Downside to this is that it will theoretically increase the size of project files. You are also trying to move away from having the entire project on a single file. It would be a bad idea imo to put all images as a string in a single file xD hence why I prefer to have piskel keep each layered animation in a separate piskel(json) file

[4ian]

can we not save game states as json files on android?

Right now everything is saved in browser local storage (or emulated local storage in case of Cocos2D-JS). Interacting with file is a large amount of work that should be done on everything platform, on using files on mobile is massively different than on desktop.

This affects saving piskel layers and dealing with dialogue data - which are more important to tackle atm imo

I'm still not sure if all of these are solved by the same thing. At least for the end users, piskel layers should not even be visible to him. It's an implementation detail, not necessarily something that should go in the resources. It's a solution possible, but there must be others :) Speaking of this, as you just said:

Btw if we use piskel's json data to store image data, instead of actual bitmaps we use now- we can technically store it in the json project file. That way when the user downloads their project file from the web version - it will contain all of their animations with layers and everything!

Exactly, that's another solution 😄

Downside to this is that it will theoretically increase the size of project files. You are also trying to move away from having the entire project on a single file

Size is not a problem, we're embedding JSON and data that are lightweight :) Also, splitting the project in different files can be done in an "agnostic" way of what is inside. (This means that the code responsible of the splitting is just given the JSON of the project, a few keys that should be used to split, and then take care of saving the project in the different files, or on the contrary reconstructing the original JSON. At the end, the project is totally unaware if it is splitted into multiple files or not. And that's exactly what we want :))

I'm trying to think about alternative solutions because I know that importing JSON files could be useful, but it's something not to be taken lightly - and it's much too easy to cripple GDevelop with tons of features that are half bugged (that was me years ago, so better avoid it again and take the time to properly assess new things :)).

In the end, I'd like to add this, but I want to double check what the requirements will be. Do you plan to access to the JSON resources from the events ? If yes, with which action/condition? If not, is it ok if you can only use JSON resources from the extensions?

[blurymind]

For piskel - I just need a way to save the piskel object - either to an external json file- stored in the project folder, or as part of the actual project json file. Its only goal is to preserve the layer data of the bitmaps piskel created for gdjs. In theory we can even stringify any imported images that are not made in piskel and store them in the project json. But my goal really is to solve the layers problem atm.

For Yarn - I have the yarn editor- which is very good for authoring non linear dialogue and even setting up cut scene events while the dialogue is running (you can insert tags, check title, metadata that can be used to trigger logic). It saves the same way as piskel - I can either send the whole dialogue back to gdevelop's project json or to a json file resource. In this case I think that it has to be outside of the project json - as it will make it much easier to deal with when translating a game or collaborating on a story with non programmers. There is also the runtime part of it- for which I want to use bondage.js to parse the data via a js extension. The js extension will require to access json resource files that were set to be used in the newIde. That is the biggest roadblock atm. From there on actually parsing them should be easy. Bondage.js now gives access to everything I need to set up the actions/conditions to be used in gdevelop, but I must first initiate it by giving it the yarn story object. Before even thinking of embedding yarn with the newIde, I want to make sure that yarn json files can be parsed by gdjs - that we have an awesome extension for them

So it would be good to be able to implement an action to load a json file from the events, yes

[Lizard-13]

Not sure if it was suggested already for this very specific feature, but if it just needs to store the path points you could save the points in the project file as any object data, this way you can let the user draw the gestures in the object editor, get the points (saved in the project with the object data) and load it directly with the data passed to the runtime object. I say it because unlike piskel and yarn it doesn't even need external files, because there is no editor for it at all.

[blurymind]

@Lizard-13 That is a cool idea that I want to try with piskel objects, but I have no idea how to write new custom variables to the project.json file. Can you show me any examples? :}

[Lizard-13]

In the old-extensions system it was made adding attributes to the object un/serializer functions: https://github.com/4ian/GD/blob/8cad5d76463eb059994d5bf68a726b6f4826c9a6/Extensions/TiledSpriteObject/TiledSpriteObject.cpp#L38-L50

And then the data is passed to the runtime object: https://github.com/4ian/GD/blob/8cad5d76463eb059994d5bf68a726b6f4826c9a6/Extensions/TiledSpriteObject/tiledspriteruntimeobject.js#L19-L25

But no idea how it is done in the new pure js extensions, surely @4ian knows if/how it can be down :)

[4ian]

What @Lizard-13 mentioned works but it's for objects or behaviors only :)

@blurymind For Piskel:

I have being thinking about it, and one thing to consider is that resources are resources for the game, not for the editor. So I think that for Piskel, using "JSON" resources is not appropriate: we don't want to clutter the final games about information about piskel layers. Instead, maybe we could add to gd.ImageResource some kind of arbitrary extra "data" that could be attached to it. This would allow you to have a piskel property where you can store the json of the layers. Would that work for you? As a reminder, gd.ImageResource is the class used to store images in the list of resources. It's used there when Piskel is closed:

https://github.com/4ian/GD/blob/6cdee3b9773045ad2daa645a8c47aa71878fe83c/newIDE/app/src/ResourcesList/LocalPiskelBridge.js#L48-L61

If we image that we have something like imageResource.setMetadata("piskel", thejsonofpiskel), would that work? (You would have to load it to when opening Piskel).

For yarn: Is having an object something that is possible? In this case, the JSON of the dialog would be stored inside the object. If that's not the case, let's image we have a JSON resource:

• When would you save them? What would be the name of the resource?
• In the game engine, how would you load it?

For the last question, here is how the gdjs.ImageManager and gdjs.SoundManager are constructed: https://github.com/4ian/GD/blob/e9bf8ff30963d25b107c45e5e527c8f6a869068d/GDJS/Runtime/runtimegame.js#L20-L25

They take a reference to the resources and then iterate over them to load (https://github.com/4ian/GD/blob/e9bf8ff30963d25b107c45e5e527c8f6a869068d/GDJS/Runtime/pixi-renderers/pixi-image-manager.js).

Do you have an idea of what kind of API/function/method you'd like to have?

[blurymind]

@4ian this is a great idea! For the piskel you are right. It would make alot of sense to store the data in the actual imageresource object - as metadata of the image sequence. If you add the ability to set metadata like that- I will put it in use when piskel loads/saves in gd5 if you want to - will do a pull request on it :) 👍 Giving all resources ability to have metadata would also be very valuable to the engine. Godot does a similar thing in some aspects btw.

For yarn - as a resource having them would be very helpful- as it will enable people who only have yarn to open the files, edit them and send them over to the gdevelop dev. It will make story translation much easier. My vision for this is to have Yarn editor itself embeded with gdevelop - very much like piskel and monaco. In yarn's case, it will treat json files the same way piskel treats image sequences- as resources. If there is no yarn (json) file loaded- open yarn and when saving-create one. The name of the file can correspond to the name of the game object it is being attached to, but can also be specified by the user. After saved (to project folder) and attached to a game object - to edit it, the user doesnt need to navigate to it.Its a single click open and single click save and close. On the runtime's side- the json file gets loaded into a dictionary{} when the game loads. It is initiated by the dev via the event sheet. The dev can setup in the event sheet and yarn additional triggers while a dialogue is running. For example, tags that were set in yarn can be used as conditions in the event sheet. My implementation also listens for when a text card has changed- what choices were made by the player, what hidden tags were set in that dialogue card and even commands.

Basically I want to load the json files that are attached as a resource - as an ordinary object{} on init, so my extension wrapper of bondage.js can use its data to do things. Json files are tiny, even if you have a lot of data in them. The concept is the same as how image resource and sound resource is loaded and used when triggered from whats in the event sheet :)

[KinkGD]

@blurymind , any news from "the front" ?

[blurymind]

I will revisit this once @4ian adds support for metadata in objects like this imageResource.setMetadata("piskel", thejsonofpiskel) When we have that method merged, I will address saving piskel data and storing the layers :)

[4ian]

Support added :) This interface is resource.setMetadata(string) but for now please store JSON inside, inside an object containing the name of the editor. This way we could later change the interface to be resource.setMetadata(editorName, string)

[blurymind]

@4ian thank you! 😄

[blurymind]

I am currently refactoring the external editor handling code to check if metadata exists and if so, to pass it accordingly to the editor

[4ian]

Do you think we can close this? I think metadata was the proper way of solving this :)

[blurymind]

Yes i think we can close this now. I need to figure out how to add yarn support at some point. Its a json type data again, but i am not sure how i would set and get metadata on a js extension. Does the extension api allow to set metadata to a new type of a resource or game object? Can I declare an object in the js extension, store values to it and just write them it ro project.json? :D

On Mon, 12 Nov 2018 23:58 Florian Rival <notifications@github.com wrote:

Do you think we can close this? I think metadata was the proper way of solving this :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/4ian/GDevelop/issues/569#issuecomment-438071869, or mute the thread https://github.com/notifications/unsubscribe-auth/AGMbVXV6gaE0YEfgopa8MrBoQSQOHsBcks5uugcSgaJpZM4Vn7uI .

[4ian]

Let's talk about it back when you want to work on this if that's ok? Might be metadata or some other way to store things yeah.

[blurymind]

Yes ok 😄 Closing this now