[Proposal] Implement Database from Json files using Data Driven Programing

[MarianoGnu]

About a month ago i did this script for a personal project, i think this can match any case where you make a data driven approach, using this approach, creating a new item, skill, or even maybe enemy could be as easy as adding it's data structure to a Json file. How it works: there's a const value in the script currently pointing to res://game_data/ folder, this could be changed. but inside this folder you place subfolders with the names of different categories (in my case these was: items, perks, fighter_classes, etc) and inside every folder you place whatever quantity of json files with any name you like, these files have only 2 requeriments:

1. the file must be an array, even if it only has 1 or 0 objects
2. every object inside the array must have an id property, if this id is repeated for every category, the last one readen will override the previous one. This is an example of how an Item structure looks for my game

After the files are loaded, the Node having this script attacked to will have all this information on a Dictionary called data. this variable has a property member with the name of every folder on the main directory, and from there you can access every object on the json file with the id provided on the json file, so for example to access to the "Photon Gun2" weapon you should use the path data.items[2]

The filter_data method: the script also has a filter_data(source, criteria) tool that will gather from the objects on the database those who matches a certain criteria. for example this line of code: filter_data(data.items, {type="BARRIER"}) will return all the items that can be equiped on the BARRIER slot.

[NathanLovato]

Which problem does this solve in this project? We already use data and declarative code where we can.

We already use Godot's Resources system to store data, which is built just for that, used throughout the engine, is well-tested, and optimized.

I'll need some really good arguments to maintain our own system and to lose the ability to edit data in the inspector.

[willnationsdev]

I definitely agree that Resources are a far better approach for Godot projects (especially demos like this one).

[MarianoGnu]

Mainly, that json files are faster to read and edit, you dont have to lookup for nodes and/or resources on the file system or opening an scecific scene to find and change what you want to change. I agree this is not the most user friendly, but it's workflow friendly, you dont have to screate an specific scene for every item, effect, job, enemy or whatever, you just create template scenes that can work on any behaviour and then it reads it's properties from the json files, that for example how champions and heroes on league of legends are done.

[willnationsdev]

@MarianoGnu The latter point is a quality of off-loading data to data files rather than sub-resources within a scene file. It is applicable to any data format.

JSON I'll agree is faster to edit sometimes, but it also has no data integrity protections and is more complicated for non-programmers who may want to tweak things since there is no generated GUI for them. Resources in comparison can generate EditorInspector content easily, setup setters/getters for properties, and use sub-Resources and sub-Inspectors to create the same nested structures that a JSON file supports, all while still offering drag and drop support and FileDialogs.

[NathanLovato]

that for example how champions and heroes on league of legends are done

They probably have some tools that serialize the data. I'd be really surprised if the designers balanced the characters from JSON instead of using some UI while the game is running.

json files are faster to read and edit

Maybe for a developer like you who's used to working with JSON. I don't think so in general: a folder with a flat structure of text files, which is exactly what items are in the game, is just as easy to edit.

And contrary to JSON Godot recognizes the format and lets you edit files from the inspector as well, from nested inspectors...

As you say yourself:

I agree this is not the most user friendly

So it's not workflow-friendly either.

This is an open source project, that we're building so the Godot community can use it and developers of all horizons can join and contribute. We should avoid adding dependencies on external tools and making design tasks less accessible to designers.

As I said above, you need strong arguments for us to ditch a well-tested, built-in, optimized system in favor of doing our own from scratch. For now, I'm not convinced at all. I only see drawbacks.

[MarianoGnu]

Well, i have done a lot of editors and resources, and what i can say is

1. It takes a lot of time to code a custom editor that matches database data (for example selecting initial equipment from a list of items currently existing)
2. It a lot of time to use a GUI to edit data compared to wrigint plain text (this is the same reason why most of people prefer GdScript over VisualScripting.

That's all i have to say, maybe you can figure out some way that solves my problem and improve my own workflow.

[salvob41]

I understand that this is an educational projects for Godot but when developing for a game that want to be released, usually (often) you need support from backend, especially if you want some online feature ( multiplayer? buy items? score ladder? ). For this you will connect to external services, and JSON is a pretty viable way (the best if online).

Plus, (but this is my opinion) Resources are not user-friendly and/or workflow-friendly at all, at least as they are right now: It might be hard backtracking resources attached to .tscn file.

Example for this project regarding battle_template Resource:

• Party member has a Resource export var (that can be ANY Resource for Godot editor).
• Resources like Godette.tres and Robi.tres have a PackedScene in it. In order to find out which PackedScene they have you cannot click the scene on the editor of resource but you have to click the arrow next to the editor and click Show in FileSystem.
• Accessing to the <nameCharacter>Anim.tscn will show the scene of a Position2D with Sprites and AnimationPlayers (Where will be added in the main actor scene? With the information we have from this process, we cannot know).
• Battler.tscn has another resource called template and it is a Resource as well. What kind of resource? ....

A JSON has key->value data. it might be verbose but tells you exactly what you need and it is understandable by other services that you (most likely) will end up using

[NathanLovato]

It's late here, I'll answer in greater details later, but don't forget that:

1. You can edit and browse resources directly from the file system
2. You can edit a resource attached to an existing one as a sub-resource inline
3. Godot updates paths and dependencies for you, this is a big thing for refactoring

You can also get any object as json in Godot.

There's one question that's still left without an answer: which problem does it solve in this project? Is there an issue with content editing considering the small scope that we have? Is it worth the cost of making content creation less accessible to designers? And if editing resources feels slow right now, can't we improve the file and data structures to solve that directly?

NB I do understand that json is an efficient format, I use and used it myself. I'd recommend a good old table instead for e.g. items, and anything that doesn't need a nested structure, like items. It's quite common in games, it maps 1:1 to arrays...

Having worked as a game designer, with a variety of tools and plain data... Which is faster depends a lot on the context. Again, resources are also editable as text files, so if you use a flat structure it's about the same as working with a list of json objects.

As for people not using visual script it's unrelated imo, it's just that the node editor's ux needs a lot more work.

[MarianoGnu]

yeah, all these features you mentioned are new in 3.1 The issues i mentioned where working with 2.x and 3.0

[NathanLovato]

And this is a Godot 3.1 project

[MarianoGnu]

All right, we could reopen this discusion if we ever find a problem with the workflow and need to figure out another easier way of doing things.