Replace built-in scripts with "discrete scripts"

[marcospb19]

This proposal was originally made here.

Describe the project you are working on:

Currently I'm not working on any project (?).

Describe the problem or limitation you are having in your project:

Currently I'm not working on a project.

Built-In Scripts are very problematic and we have seen requests to remove it from the engine, however people also argue that it is very useful and ask to not remove them, here are some quotes:

• @Calinou

  I also think we should remove built-in scripts in 4.0 due to the various limitations and their worse VCS-ability.

Built-In Scripts lacks VCS-ability because the .gd content is inserted inside a .tscn file, this makes diffs a nightmare.

• @magpie514

  Please don't remove built-in scripts. The project folder is already a mess with metadata for every image and there are times there is just not sane to add new files to it when a 4-line builtin script does the job.

A lot of people rely on Built-In Scripts on their projects, they claim that the feature is very useful on managing the file mess from small scripts (this feature is a plus for Godot over other engines).

• @Calinou

  Making them work seamlessly with external editors is arguably a difficult task.

This is very important, as far as I know, the majority of game developers use VSCode as their main text editor, external editor support is a must for the future of Godot.

VSCode extensions grant incredible support for Godot, including integration with the GDScript language server, also worth mentioning that if someone has a background on Unity, Unreal Engine or Javascript/CSS/HTML, they are probably used with using editos like VSCode.

Most of the issues are caused because Built-In Scripts aren't .gd files, they are .gd code inside of .tscn files, so we need to change it.

Every time a new feature is added to scripts, we suppose that they will work with Built-In Scripts, but this isn't always possible, corner case code (that can add complexity) is necessary to support Built-In Scripts functionalities, thus development time is wasted unnecessarily, and some of the issues remain unsolved.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

No

Can this be an add-on in the asset library?:

No

Describe the feature:

See next comment

[marcospb19]

Before suggesting a replacement we need to ask: Why are Built-in Scripts great? Because They hide the complexity of having to deal with various .gd files when building a scene.

Proposal: Discrete Scripts

Discrete Scripts will behave almost exactly like Built-in Scripts to the user eyes, but the better design solves a lot of issues.

Discrete Scripts have their own .gd files that is stored at the same folder of the scene it's being applied.

(I suggest changing the name of the feature because the script isn't "Built-In" anymore, maintaining this name would lead to confusion)

Example:

Imagine I'm developing a game that contains a car, and I'm editing the scene at res://Vehicles/Car.tscn, here is the structure of the Scene:

In, Car scene, I'll attach a Discrete Scripts to the node called Engine, instead of putting it's content inside of res://Vehicles/Car.tscn, a new .gd file is created at res://Vehicles/Car::Engine.gd.

This script is hidden by default in the FileSystem Dock, but we can toggle it using a button that will be added to the FileSystem Dock, at the side of the Toggle Split Mode button.

This trivially solves problems with VCS-friendliness problems and external editors.

Synergizing this with the change of the look of the scripts on issue #847 (I'm willing to help!, please review it), I also suggest new icons for Discrete Scripts on the FileSystem Dock and Scene Dock (SCENE TREE).

---

@TicklesTheBrain

I don't think game developers should expect to be able to port their complex projects painlessly to a new version of the engine every time a major version comes out.

User intervention won't be necessary for updating from older versions of Godot.

[sonic2kk]

This sounds absolutely fantastic and as a user of Godot I hope to see something like this get implemented!

[KoBeWi]

This proposal assumes that you never manage files outside Godot editor, which is untrue. I'm already triggered by the .import files everywhere, adding built-in scripts, even if they were hidden in the editor, will make managing files even worse. Maybe there could be a dedicated directory for these scripts, so they are at least all kept in one place.

Also, built-in scripts despite all their special treatment, are still built-in resource, so embedding them works like embedding for any other resource. Not sure if there is easy way to change this.

btw, one big advantage of built-in scripts that you didn't mention is that they make the scene self-contained. You can easily move it between projects and all dependencies will be inside. "Discrete scripts" would make it much less convenient.

[marcospb19]

@KoBeWi

This proposal assumes that you never manage files outside Godot editor, which is untrue. I'm already triggered by the .import files everywhere

Glad you pointed this out. This is a relevant point

1. I'm also hoping that .import files change location in the future, like you mentioned at #1060, to a .import folder.
2. There are priorities we should take into account, users spend more time inside of the engine than outside.

You can easily move built-in scripts between projects and all dependencies will be inside, "Discrete scripts" would make it much less convenient.

3. Supporting sharing files to different projects in Godot isn't a priority (compared to VCS-ability), it almost always do break a lot of things, but with Discrete Scripts this would be a 2-step move.

Maybe there could be a dedicated directory for these scripts.

4. This was considered in the original propose, I suggested that these scripts should go into a folder called discrete_scripts/Vehicles/Car::Engine.gd, maybe this could be added as an option later.

[dalexeev]

Advantages of built-in scripts (for me):

1. they close automatically when I close the corresponding scene (can be implemented for regular scripts);
2. they are removed along with their scenes;
3. they don't clutter the file system.

I partially solved (2) and (3) by the fact that for each scene I create a folder in which I store the scene resources, including scripts. But I am lazy to do this for temporary/debug scenes.

[aaronfranke]

I assigned the 4.0 milestone because a decision should be made on this before 4.0 is out, but I don't really like the idea and I think the chances of this being implemented are very low, since @reduz does not like to hide functionality from the user. EDIT: We just need to be very explicit about what's going on and then I think it should be fine.

I think it's weird to pretend that an external file is a built-in file, and it would make the file system in Godot look different than in file browsers, Git clients, and external editors like VS Code. Aside from that, we can't use the :: naming scheme, since : is not a valid character to use in a file name in Windows, so a different naming scheme would need to be chosen. EDIT: An alternative is to use . characters since they are already banned as node names.

I'm 100% in favor of removing built-in scripts entirely (EDIT: or replacing them works too). What some users cite as benefits of built-in scripts (mostly seems to be the ability to have simple scripts without their own file), to me are 1) really minor things and 2) things that annoy me (when I'm reviewing diffs on Git, I absolutely do not want GDScript files mixed in with tscn files).

[h0lley]

It probably no longer bothers seasoned Godot devs because they're just so used to it, but I think that for people starting out, having two files of the same name side by side feels wrong and messy.

I partially solved (2) and (3) by the fact that for each scene I create a folder in which I store the scene resources, including scripts. But I am lazy to do this for temporary/debug scenes.

That is a logical approach for people valuing a clean folder & file hirarchy, but it is silly that we have to do this arranging manually, since in Godot it is very common - or rather, the norm - that for every scene file, there's one script file belonging to it.

Whoever originally implemented built-in scripts must have also understood this and imho, did so with good reason.

If it means that I can practiaclly cut in half the amount of files I have to browse through in my FileSystem panel, I will very gladly make the one extra click of accessing the script file through its scene file. This option should not be forsaken.

I think it's weird to pretend that an external file is a built-in file,

Which is why OP suggested to stop advertising it as build-in but rather call it discrete or something. This really does not have to be a problem.

since @reduz does not like to hide functionality from the user. and it would make the file system in Godot look different than in file browsers, Git clients, and external editors like VS Code.

This is also a non-issue when users still consciously opt into this functionality like it currently is the case for built-in scripts, knowing the implications.

Here's one more proposition: Users can in fact already easily hide any files from the FileSystem Panel by prefixing them with a ..

I could do that to get rid of the "duplicates", however, what is missing then is the ability to easily rename and move, as well as to search the contents of those files. Now, if whenever one renames or moves a .tscn file, Godot would check if there's a .gd file of the same name (optionally prefixed with a .) alongside it and then also move and/or rename that one accordingly, that would be a solution - and one that can be implemented with very little effort I presume.

Next we'd need to add an exception for the Find in Files function so that .gd files are searched no matter whether or not they are prefixed with a ..

With those changes I think we'd truly get most of the benefits of built-in scripts with the full feature-set of stand-alone .gd files. If that were to be implemented, I'd be all for removing or hiding the traditional functionality for built-in scripts.

[marcospb19]

however, what is missing then is the ability to easily rename and move, as well as to search the contents of those files. Now, if whenever one renames or moves a .tscn file, Godot would check if there's a .gd file of the same name (optionally prefixed with a .) alongside it and then also move and/or rename that one accordingly, that would be a solution - and one that can be implemented with very little effort I presume

I apologize for not mentioning this in my original suggestion, but this was also what I intended for discrete scripts (and I thought that I had made this clear), when renaming and moving, the scripts attached should also be renamed and moved. Not only that, but when duplicating (also duplicate all scripts) and when making node root of the tree (moving but we need to assure this operation isn't going to behave differently).

Do you agree that there should be a button in FileSystem Dock to show this hidden files? I would love to hear feedback on my idea.

[h0lley]

@aaronfranke in reply to this thanks for your analysis. :) I think the first approach has the problem of potentially unwieldy filenames, as readability still plays a role when navigating opened scripts in this panel:

And also when browsing files outside of Godot.

There's no neccesity for those file names to reflect the full node path, is there? After all, it's not like the scene is iterating over the directory to then attach scripts based on their filenames. The information which script is to be attached to what node is already stored in the scene file.

So I'd say one descriptive bit separated by .s after the scene name, e.g. .player.camera.gd, should be sufficient. Godot would set and perhaps update that bit based on the name of the node the script is attached to. It would also sometimes have to append numbers as it can occur that the filename is already taken. For that, I am not sure if .car.wheel2.gd or .car.wheel.2.gd would make more sense.

And yes, it's probably a good idea to keep the possibility to use this system to also bundle resource files other than .gd to the scene file should that become relevant.

@marcospb19 in reply to this yea, I don't think I should post a new proposal on godot-proposals as yours exist already. I have re-read your propostion and you are right, mine really is almost identical, though I guess the part about how Godot syncs changes of the scene file to the script files belonging to it is important to outline after all.

As for the ability to toggle visibility of those files in the FileSystem panel, I am not sure if that's important. After all, nobody is currently complaining that built-in scripts cannot be made visible in the FileSystem panel, right? What might be an use-case for that?

[KoBeWi]

So I am entirely against removal of built-in scripts. I already commented why 3 months ago, but I was asked on IRC to write my opinions/use-cases, so

First of all, the assumption that you have one script per scene is wrong. I often have multiple scripts per scene, each of them are built-in. I also have a directory containing all my maps, literally hundred of scenes. Now imagine that each of their built-in scripts became a new file... The built-in resources are one of the best selling-points of Godot and scripts are part of that too. I don't want my project to became a mess pile of useless files like Unity tends to do. I like to keep things tidy. (btw the .import file proposal was rejected)

Another thing is moving scenes. As I already mentioned before, you can move around scenes with built-in scripts as much as you like. But what's also important - you can easily duplicate them. I often have two similar scenes. So I have one scene with a built-in script, I duplicate it, then I just make minor changes to the script in the new scene. This is less convenient with file scripts, because I have to remember to copy them too to make a new one. Also for the "discrete scripts", if one scene has multiple of them and they are hidden, you have to make sure that they are moved/duplicated along with the scene file if we want to replicate functionality. This will effectively replace current built-in script hacks with filesystem hacks. Meh.

IMO the biggest problem with built-in scripts is that the editor has a special logic for handling them, which means we need to predict every thing you can do with scripts and make sure built-in scripts work too. I wonder if it would be possible to move the logic out of the editor and into the Script resource. Scripts themselves should recognize if they are built-in or file and handle this logic internally, which would make editor simpler and maybe allow to fix some problems, dunno.

Also I totally don't see how not being able to edit built-in scripts with external editor is a problem that can be solved by removing them xd If you use an external editor, using "external scripts" shouldn't be a problem either.

things that annoy me (when I'm reviewing diffs on Git, I absolutely do not want GDScript files mixed in with tscn files).

They are both text files though, I don't see any problem with that.

[dalexeev]

So I am entirely against removal of built-in scripts.

I agree with you. At least I think "discrete scripts" are not a good idea. We need to either improve the built-in scripts or just remove them rather than add a new option with confusing behavior.

IMO the biggest problem with built-in scripts is that the editor has a special logic for handling them, which means we need to predict every thing you can do with scripts and make sure built-in scripts work too. I wonder if it would be possible to move the logic out of the editor and into the Script resource. Scripts themselves should recognize if they are built-in or file and handle this logic internally, which would make editor simpler and maybe allow to fix some problems, dunno.

I wonder why built-in scripts have so many problems. After all, their only difference is that they are not stored in a separate file, but in a sub-resource. In addition, resources of other types (non-scripts) work equally well, whether they are stored in a separate file or not.

I experimented a bit and found out the following things:

; test.tscn
[gd_scene load_steps=3 format=2]

[sub_resource type="GDScript" id=1]
script/source = "extends Node

func _ready():
    print(\"test\")
"

[sub_resource type="StyleBoxFlat" id=2]
bg_color = Color( 1, 1, 1, 1 )

[node name="Node" type="Node"]
script = SubResource( 1 )

[node name="Panel" type="Panel" parent="."]
margin_right = 40.0
margin_bottom = 40.0
custom_styles/panel = SubResource( 2 )

var res = load("res://test.tscn::1") # OK: [GDScript:39839]

var res = load("res://test.tscn::2") # Error: Resource file not found: res://test.tscn::2.

var res = preload("res://test.tscn::1") # Error: Can't preload resource at path: res://test.tscn::1

If I manually create a file like this

; test2.tscn
[gd_scene load_steps=2 format=2]

[ext_resource path="res://test.tscn::2" type="StyleBoxFlat" id=1]

[node name="Node" type="Node"]

[node name="Panel" type="Panel" parent="."]
margin_right = 40.0
margin_bottom = 40.0
custom_styles/panel = ExtResource( 1 )

and open it in the editor, it works, but when I save it:

; test2.tscn
[gd_scene load_steps=2 format=2]

[sub_resource type="StyleBoxFlat" id=1]
bg_color = Color( 1, 1, 1, 1 )

[node name="Node" type="Node"]

[node name="Panel" type="Panel" parent="."]
margin_right = 40.0
margin_bottom = 40.0
custom_styles/panel = SubResource( 1 )

I don't quite understand how this works, but it seems to me that if the core can correctly load and save resources with :: in the path (as sub-resources), and then simply pass it, then the logic of normal and built-in scripts will not differ.

It is clear that some file types cannot store sub-resources, but this should not be a problem for the sub-resources themselves. ResourceLoader/ResourceSaver should check if a given file type can store subresources or not. If yes, then the core loads/saves the resource at the specified path, if not, then an error occurs. Also, subresources should be included in the dependency graph.

Also, this would fix issue godotengine/godot#22226. It will be more convenient to give meaningful names to subresources (this is possible even now, but very few people know about it, and this makes little sense now), rename them, move them into separate files.

[h0lley]

@KoBeWi

Now imagine that each of their built-in scripts became a new file...

Would that be an issue if they are out of sight though? Hm, since you mentioned how you are annoyed by the .import files as well, I figure you like to browse your project files outside of Godot, where .-prefixed files are not hidden. Sadly you are not on a Linux OS where .-prefixed files are generally regarded as hidden by the default file explorer.

There is a way to hide those files on Windows, too: attrib +H J:\.* /S /D (J being a drive letter in this example), but I believe this only applies to all current files and would need to be repeated for files created afterwards. :/

you have to make sure that they are moved/duplicated along with the scene file if we want to replicate functionality.

If Godot had the proposed logic for bundling files by name, it would be entirely feasible for that to happen automatically (move/duplicate/delete). The user would only need to manually take care of it when copying a scene file with a file explorer other than Godot's... which really should be quite a rare occasion, no?

@dalexeev

rather than add a new option with confusing behavior.

What's so confusing about it though? The most recent iteration of what's been proposed would be for almost all intents and purposes identical in behavior to the current built-in scripts, but would also allow editors (both Godot and external ones) to deal with them as .gd files, which would resolve pretty much all issues.

To me the source of the issues seems to be the fact that gd code is embedded and escaped within .tscn files, and I don't think there's an easy/feasible solution while keeping that arrangement.

They are difficult to use with external editors, Editing during test play does not update the remote scene, They don't open properly when using Find in Files, due to them being escaped and wrapped in the scene file syntax, they have poor readability outside of Godot, e.g. in VCS, they go haywire as tool scripts (it somewhat works, but I have to restart Godot each time I open them), class_name is not available, etc.

All of this could be instantly resolved by giving up on having GDScript stored in .tscn files, while the desired functionality of built-in scripts could be preserved with the "discrete scripts" or "files bundled by name" propositions.

[dalexeev]

Editing during test play does not update the remote scene, They don't open properly when using Find in Files, they go haywire as tool scripts (it somewhat works, but I have to restart Godot each time I open them), class_name is not available,

This is not a fundamental limitation, but only the nuances of the current implementation. It is quite possible to fix this by revising the logic for saving/loading subresources.

They are difficult to use with external editors,

External editors are not designed to edit subresources. You either need to use separate files or create a plugin for the external editor. It would be weird if a separate file would appear as non-separate in Godot editor.

due to them being escaped and wrapped in the scene file syntax, they have poor readability outside of Godot, e.g. in VCS,

Only double quotes and backslashes are escaped. Built-in scripts are usually only used for utilitarian scene-specific functionality, they should not be too large, overloaded with logic and architecture. Otherwise, it is better to keep the script in a separate file.

---

In my file manager, the display of hidden files is always enabled, because I want to completely control the contents of directories. Hidden files and folders for me are just special objects, not entities that do not seem to exist, but in fact they are.

I understand why built-in scripts are needed and why separate files, but I don't understand the "discrete files" approach. I don't understand why scene.tscn + .scene.tscn.1.gd is better than scene.tscn + scene.gd.

[h0lley]

External editors are not designed to edit subresources.

Then have it no longer be a subresource. At the end of day we are talking about GDScript here. External editors are meant to edit GDScript. No point in being hung up on it being a subresource or not.

I don't understand why scene.tscn + .scene.tscn.1.gd is better than scene.tscn + scene.gd.

So the idea is that when attaching a new script to a node, instead of being able to tick "built-in script", you'll be able to tick "bundled script" or similar. When going for bundled, you do not choose a name for the script yourself, but let Godot do that for you. The script is then saved alongside the scene file, but won't be displayed in the FileSystem panel - just like currently, build-in scripts are also not displayed as separate entries in the FIleSystem panel. When renaming, moving, or duplicating the scene file, Godot will check for the presence of bundled scripts (which is entirely derived from their filenames) and apply the modifications to them also.

From that we get the benefits of built-in scripts (on the surface it should pretty much be an identical workflow) with all the functionality of separate script files. And any headaches about implementing functionality for two kinds script storing methods are history.

You may call it weird, but I think that's largely because you have an understanding of how built-in scripts are implemented as subresources. Do you think that the average Godot user cares about how their built-in script is being saved? I think when someone starts out with Godot and ticks that "build-in script" box, all they think is "oh cool, so I can add scripts that don't clutter my FileSystem panel for no good reason". That Godot stores the script as subresource to that scene is just a technical detail and of no bearing to the user experience. Godot could store that script just as well as a hidden file - most users won't even notice.

It is quite possible to fix this by revising the logic for saving/loading subresources.

I mean, yea, that would be nice. But you are aware that we are stuck with most of the issues listed for quite a many years now, yes? I just don't see it happening if it's a larger undertaking. Adding a couple of file checks seems much more realistic.

[marcospb19]

@Serenitor

As for the ability to toggle visibility of those files in the FileSystem panel, I am not sure if that's important. After all, nobody is currently complaining that built-in scripts cannot be made visible in the FileSystem panel, right? What might be an use-case for that?

I was brainstorming about it, by allowing it to toggle show/hide on those scripts in the FileSystem Dock you could make a very fast search for a discrete script that you know the name of some node (or parent node) it is attached to, because the name of the script would contain a node path (assuming the naming you suggested).

you have to make sure that they are moved/duplicated along with the scene file if we want to replicate functionality.

The user would only need to manually take care of it when copying a scene file with a file explorer other than Godot's... which really should be quite a rare occasion, no?

Not only rare, but stuff in Godot will break if you move files around randomly, so that argument against .

---

@KoBeWi

things that annoy me (when I'm reviewing diffs on Git, I absolutely do not want GDScript files mixed in with tscn files).

They are both text files though, I don't see any problem with that.

The VCS problems of built-in scripts may be something that doesn't annoy you, but it's a big deal that core devs have already addressed, also, at https://godotengine.org/ there is a dedicated image in the A different way to make games just to advertise Godot's VCS-ability. So that's a big problem with built-in scripts.

IMO the biggest problem with built-in scripts is that the editor has a special logic for handling them, which means we need to predict everything you can do with scripts and make sure built-in scripts work too ... Scripts themselves should recognize if they are built-in or file and handle this logic internally, which would make editor simpler and maybe allow to fix some problems, dunno.

I recognize that some of those problems are solvable, but because of the current design of built-in scripts, solving others becomes unreliable, that's why several core devs have suggested removing them since 2016 (we are about to celebrate other anniversary of those problems lol).

---

@dalexeev

I agree with you. At least I think "discrete scripts" are not a good idea. We need to either improve the built-in scripts...

We cannot improve built-in scripts to solve some problems, like external editors integration.

...rather than add a new option with confusing behavior.

Why is the behavior confusing? Maybe you think this because we haven't yet come to a final decision, but what about this: "Discrete scripts are like built-in scripts but have their own file, with unique path.". Done, this is not confusing.

This is not a fundamental limitation, but only the nuances of the current implementation. It is quite possible to fix this by revising the logic for saving/loading subresources.

Here is the thing, are devs supposed to fix built-in scripts problems forever? Or is it better to switch to another way of doing things that is less error prone?

They are difficult to use with external editors,

External editors are not designed to edit subresources. You either need to use separate files or create a plugin for the external editor. It would be weird if a separate file would appear as non-separate in Godot editor.

We have argued a lot about the removal of built-in scripts, you and me (and much others) agree that they shouldn't be removed, on the other side, external tools integration is a HUGE DEAL for Godot, and those things do not fit together at all.

I believe that the removal of built-in scripts will happen for one reason of another, but as people like us love the feature, we have the option to fulfill those needs with something else, instead of leaving a open hole. With such useful stuff like VSCode and Godot Language Server laying there (we can't ignore it), this rework is a must.

---

I don't want my project to become a mess pile of useless files like Unity tends to do.

Nobody wants to push that mess into into Godot users, right?

Because of this I highly suggest that (at least as an OPTION) the scripts should go to a folder called discrete_scripts at the root of the project instead of inside of each folder (because Windows users hate this, . isn't invisible).

Imagine you're reviewing a git commit diff, and you see that there are additions on the file discrete_scripts/worlds/street/traffic_lights.red.gd, you know exactly the location of the scene this script refers to, even with is being in another directory, this would also make possible for you to intuitively access all discrete scripts in worlds/street in your external text editor without having to load all scenes and check manually for each of them.

It's the best of both worlds, having all those scripts managed automatically by the engine, while also having the opportunity to search for them in a folder. Theoretically this should be optional, Windows users hate the mess, linuxbsd users don't really care about . files merged into the original file structure (IMHO that's actually very nice :D).

---

This will effectively replace current built-in script hacks with filesystem hacks

Built-in scripts abstractions are so useful and important for Godot that I wouldn't call this a hack, but a feature, and a widely used one. Also, creating a script file in a specific and unique path is a very normal operation, I don't know what are the hacks you're referring to.

[KoBeWi]

I don't know what are the hacks you're referring to.

The discrete scripts would need special treating when moving/duplicating scenes, just like built-in scripts have special treating for editing. Especially duplicating might be problematic, because it doesn't have a special behavior for anything yet. I can already imagine how many problems it can bring. Not to mention that the whole discrete script idea is completely against how any sub-resource work in Godot. Built-in scripts are actually just a byproduct of how built-in resources work, so to change it we'd have to introduce a completely new system.

Because of this I highly suggest that (at least as an OPTION) the scripts should go to a folder called discrete_scripts at the root of the project instead of inside of each folder

Put *.import files there and I'm in, heh.

[marcospb19]

Especially duplicating might be problematic, because it doesn't have a special behavior for anything yet. I can already imagine how many problems it can bring.

It's like copying a folder, you copy what's inside of it too, no underlying obscure hacks.

Not to mention that the whole discrete script idea is completely against how any sub-resource work in Godot.

It is indeed different, I think that's worth it, programmers spend a lot of time... programming... messing with scripts, and we need those external tools to work... so yeah, pretty much it.

"But what about shaders? are you suggesting that they should be able to be stored in dedicated files too so people can edit then with external...?"

Well... maybe? That's clearly utopia, I'm trying to attack here the bigger problems with scripts where there's room for improvement, I think that this is the most important point here.

Put *.import files there and I'm in, heh.

xD what a subtle complaint about *.import, I agree with you https://github.com/godotengine/godot-proposals/issues/1060#issuecomment-697713656

[KoBeWi]

It's like copying a folder, you copy what's inside of it too, no underlying obscure hacks.

Except duplicating a directory is handled by OS, there's no special code for that.

[h0lley]

Not to mention that the whole discrete script idea is completely against how any sub-resource work in Godot. Built-in scripts are actually just a byproduct of how built-in resources work, so to change it we'd have to introduce a completely new system.

Absolutely see the point here. Godot has this elegant synergy between Node and Resource types where pretty much everything is build upon, and then here we come an suggest something entirely different.

However, built-in scripts are already a little different from other subresources. Other than GDScripts, there's no resource where you are prompted to create an external file by default, is there? There's no resource that has this "built-in" checkbox. There's buttons to create new GDScript files all over Godot, where's other resources are generally only created from the inspector. And that's fine - not everything can be without exceptions. Unique problems require unique solutions. Ultimately we want this stuff to work - how elegant the implementation is is secondary, albeit not unimportant.

I think embedding GDScript into scene files simply wasn't a good idea to begin with.

[KoBeWi]

However, built-in scripts are already a little different from other subresources. Other than GDScripts, there's no resource where you are prompted to create an external file by default, is there? There's no resource that has this "built-in" checkbox. There's buttons to create new GDScript files all over Godot, where's other resources are generally only created from the inspector.

They are not really different, there are just helpers to create them. You can create any Resource by using New Resource option in the filesystem dock or the inspector, that includes scripts. But as scripts are the most important and even have a dedicated editor for them, it makes sense that there are also tools that make managing them easier. The underlying logic is mostly the same though.

[dalexeev]

@Serenitor Check out my mock-up above. Tomorrow I will open a proposal in which I will try to explain this more clearly. I think this is a fairly transparent solution to the problem, the resource can be made inline/external with one mouse movement, simply by dragging it into/from the file.


The only two more or less worthy arguments against built-in scripts are difficulties with VCS and external editors:

> > They are difficult to use with external editors, > > External editors are not designed to edit subresources. You either need to use separate files or create a plugin for the external editor. It would be weird if a separate file would appear as non-separate in Godot editor. > > > due to them being escaped and wrapped in the scene file syntax, they have poor readability outside of Godot, e.g. in VCS, > > Only double quotes and backslashes are escaped. Built-in scripts are usually only used for utilitarian scene-specific functionality, they should not be too large, overloaded with logic and architecture. Otherwise, it is better to keep the script in a separate file.

As for the .import files, the problem is not with them, the problem is that they are created even for the default parameters, and should be created only for files with changed import parameters. There is already an option to create an import preset, but .import files will continue to be created for all files.


[h0lley]

I think your mockup where a scene file can be opened like a directory to reveal its subresources is amazing. It's intuitive and tells the user what's going on at a glance. I'd love to see that implemented.

Though I don't quite see how that resolves the issues that arise from build-in scripts being stored differently than external scripts.

To be honest, if built-in scripts wouldn't be so ridden with issues, I would probably never use external script files when attaching scripts to nodes. At least I'd go for the subresource option by default.

It is quite possible to fix this by revising the logic for saving/loading subresources.

Here's where I'd like to believe, but cannot, since I'd think this would have happened sometime within the last five years.

[marcospb19]

As for the .import files, the problem is not with them, the problem is that they are created even for the default parameters, and should be created only for files with changed import parameters.

No. "The default values are arbitrary and may indeed change from one version to another. It's not safe to count on them staying the same."

---

Except duplicating a directory is handled by OS, there's no special code for that.

@KoBeWi you're arguing against creating discrete scripts because we would need "specific code" to copy and move files, and this would leave us with bugs.

That is not true, Godot deals with a lot of really complicated stuff, this is very simple, I mean really, if a scene moves, you move the script files attached inside of it. Like... this is literally the thing that you should be least worried about here, we are talking about code that would potentially take hours to write.

If you call this hacky, you're calling Godot code hacky itself.

---

They are not really different, there are just helpers to create them. You can create any Resource by using New Resource option in the filesystem dock or the inspector, that includes scripts. But as scripts are the most important and even have a dedicated editor for them, it makes sense that there are also tools that make managing them easier. The underlying logic is mostly the same though.

On the other side, here, you're right, we are proposing something more elaborated, different than "how built-in resources work", as you explained. Me and @Serenitor agree that the built-in resource isn't enough for scripts, Godot needs something else, reasons.

---

The only two more or less worthy arguments against built-in scripts are difficulties with VCS and external editors.

I like the "two more or less" here, like if VCS friendliness wasn't advertised in the main Godot website in a top 6 feature list for the engine, and external editors aren't used by like 90% of game devs.

[dalexeev]

I like the "two more or less" here, like if VCS friendliness wasn't advertised in the main Godot website in a top 6 feature list for the engine, and external editors aren't used by like 90% of game devs.

Quite a bold statement. Do you have statistics or do you judge 90% of users by yourself? But this, apparently, is not needed by anyone?:

btw, one big advantage of built-in scripts that you didn't mention is that they make the scene self-contained. You can easily move it between projects and all dependencies will be inside. "Discrete scripts" would make it much less convenient.

I can embed the theme and styles inside the scene file, I can embed the texture in the .font file (BitmapFont resource), etc., but I can't embed GDScript in the scene file? Really? GDScript is a resource like all other resources, therefore it should be possible to embed it as a subresource in those file types (.tscn, .tres, etc.) that support embedding of resources.

VCS friendliness

If Godot was converting your scripts to base64, then it would be really VCS unfriendly. :smiley: And the problem with escaping double quotes and backslashes can be fixed by adding a property_hint that tells Godot to save the string like this:

[gd_scene load_steps=2 format=2]

[sub_resource type="GDScript" id=1]
script/source = <<<EOL
extends Node

func _ready():
    print("Hello!")
EOL>>>

[node name="Node" type="Node"]
script = SubResource( 1 )

The problem with external editors is also not insurmountable. For example, use temporary files when opening from the editor or create plugins for external editors, since they

used by like 90% of game devs

[Calinou]

The problem with external editors is also not insurmountable. For example, use temporary files when opening from the editor or create plugins for external editors, since they

Requiring all external editor communities to write and maintain an add-on that could cause loss of data if done wrong doesn't sound like a good idea to me.

[dalexeev]

Allowing an external editor to manipulate the .tscn file - yes, it may not be a good idea. But we can use the same approach as visudo, git commit, etc. An external editor modifies the intermediate temporary file, the Godot editor monitors the changes. In this case, the risks of data loss are extremely small.

[h0lley]

GDScript is a resource like all other resources, therefore it should be possible to embed it as a subresource in those file types (.tscn, .tres, etc.) that support embedding of resources.

Technically it is, practically it isn't. It contains script logic that we want to search, parse, edit outside of the inspector or even outside of Godot, have the editor watch for changes during test play, etc. So as a subresource, it's simply unique in that it contains GDScript. It almost seems like typing "extends Resource" is not sufficient to solve complex problems. :P

[marcospb19]

@dalexeev, temporary files might work perfectly for git commit and visudo because the files are opened once in a blocking text editor that only exits the program after closing, or when the editor implements a --wait flag that makes it blocking instead of forking the process.

Do you have any other examples that apply to the GDScript context for us to compare? Where the file is updated several times and the communication needs to be tracked down on Ctrl + S presses?

This would require Godot to watch the directory where the temporary file is stored for any changes, this can be done without polling in Unix, I don't know about Windows, but I also know that Godot already does some checks when it loses and regain window focus.

Indeed, I already suggested temporary files here as a valid alternative to solve external editor, but as for now, I see that this is an over complicated solution for a problem that can be solved in a more intuitive, and a clearer way.

Why do you think that this solution is better than what's proposed here?

---

If Godot was converting your scripts to base64, then it would be really VCS unfriendly.

Let's say you changed 2 built-in scripts, where do their diff appear? in the middle of the same file, different positions, among other subresources that have nothing to do with it, that's a very bad diff.

Quite a bold statement. Do you have statistics or do you judge 90% of users by yourself?

By myself, considering the popularity of Unity, Unreal and VSCode, i'd say it's a reasonable number, and the Godot community already uses VSCode a lot, together with the integrated editor. IMHO the Godot Editor is great for beginners to start, but more advanced users usually opt for external tools that are bloated with infinite features.

---

GDScript is a resource like all other resources, therefore it should be possible to embed it as a subresource in those file types (.tscn, .tres, etc.) that support embedding of resources.

With discrete scripts you wouldn't need this embedding feature anymore, the engine will deal automatically with your scripts in a different-structured way, that is if built-in scripts are 100% removed for the engine, which may or may not happen (I think they should, there's no need to keep it).

[bzdghf]

The biggest advantage of built in scripts for me is that it allows me to think about scenes as unit of abstraction instead of scripts. Not doing this breaks the idea of scenes in my opinion making using godot feel similar to ECSs in other engines. Having a script shared by more than one scene also doesnt make sense to me as I become unsure which directory to put the shared script into. Managing script files is also tedious/unnecessary if they are always going to be used in a single scene. If not because of bugs (especially the tool keyword, and scripts not being saved) I'd love to continue using built in scripts.

Bundled scripts sound super neat for their git and vsc usability though. Id much rather have these and put up with a cluttered file system than nothing at all. If this is implemented we can also add a feature to let the script icon open it in any external editor of choice as there is no special builtin scripts to deal with.

[name-here]

If you make a folder for each scene which contains the .tscn file and any scripts you want to pack in with that scene, you can just move/copy the folder to move/copy the scene, and all the scripts will come along with it. If Godot did this automatically for "discrete scripts", or whatever they'll be called, I think it would solve a lot of the issues mentioned above. To make sure you don't have to open a folder to open scenes from the File System panel, the folders could be named something like [scene name].scene, and Godot could have some special behavior, like opening the scene file when you double click the folder, and giving them a special icon, something like the folder icon with the scene icon on top of it. You could still look inside it like you do with folders, of course. This is inspired by MacOS's folders with extensions that it mostly treats like files for ease of management.

[Xrayez]

Not being able to properly identify built-in scripts during debugging is one of the major reason why I'm not using built-in scripts for the most part:

If there are several built-in scripts in the same scene, then it's quite difficult to identify which sub-resource corresponds to which node. Perhaps there's a way to encode node name in the stack trace information?

Clicking on the item in stack frames makes editor to switch to that script, but I'm using GDScript get_stack() function as well for my custom unit testing framework, so in either case it would be nice to encode node name when built-in scripts are used.

[dalexeev]

The name can be specified, but it does not appear on the call stack, really.

[YuriSizov]

@KoBeWi may be interested in addressing this 👀

[YuriSizov]

Et voi la! It's been fixed by KoBeWi.

[pedronasser]

Comments saying that people rely on builtin scripts and therefor it should not be removed from the editor IMO are misleading and illogical. There is no really benefit for using builtin besides making it easier to organize your scripts and scenes. People who "depend" on it are just used to it, but they are definitely not depending on this feature. I think the editor should add a big WARNING saying it's not recommended to use builtin scripts and that by using it can lead to problems.

[h0lley]

yes, built-in scripts is purely a QoL feature. who said something else though?

organization of scripts and scenes is a biggie though. in a large number of cases - probably vast majority for most projects - a script only ever pertains to one specific node of one specific scene, and that's when built-in scripts are simply the cleaner, more maintainable choice.

perhaps in the future built-in script could be used to couple scene files to classes, allowing things like custom nodes that are instantiated with their corresponding template scene when picked from the create node prompt. then, built-in scripts would be more than just QoL.

I think the editor should add a big WARNING saying it's not recommended to use builtin scripts and that by using it can lead to problems.

It already has that.

[marcospb19]

There is no really benefit for using builtin besides making it easier to organize your scripts and scenes.

:+1: well, that's it yeah, QoL is QoL, I disagree that this is illogical.

[magpie514]

Wow, this is still ongoing.

organization of scripts and scenes is a biggie though. in a large number of cases - probably vast majority for most projects - a script only ever pertains to one specific node of one specific scene, and that's when built-in scripts are simply the cleaner, more maintainable choice.

This is the entire point really. At times all your nodes need is 2-3 lines of code or a simple "change color when this value is less than this value", and built-in scripts make those a lot more manageable. Specially UI/HUD elements that are very simple individually. And if it grows up too complex or needs to be reused elsewhere, it can still be separated into a file at your convenience. Of course, my larger classes and more complex/reusable nodes are isolated as files, but for something simple like a bar or gauge or some flashing icons, or simple FX sprites that are like "linear ramp modulate past this point" that are like two lines, it's not worth polluting the folder (and vscode workspace) with individual scripts.

Anyway, for a while I thought built-in scripts were going to be removed at any time and shied away from using them, but once I needed to commit to 3.5 to finish my largest project (I'm too far in to upgrade it to 4 now) there's no risk of that happening anymore, so I started using them again and for projects this large it's definitely the way to go for very node-specific scripting. My UI nodes folder in particular got so much easier to handle and that saves me a lot of headaches.