Changing Hierarchy of or renaming Scene Nodes in a complex inherited scenes setup causes a 60+ second freeze

robcbryant commented 3 months ago

Tested versions

Reproducible in: v4.3.beta1.official [a4f2ea91a], 4.3-dev6

System information

Windows 10.0.19041 - Vulkan (Mobile) - dedicated AMD Radeon RX 5600 XT (Advanced Micro Devices, Inc.; 31.0.21029.6002) - AMD Ryzen 7 3700X 8-Core Processor (16 Threads)

Issue description

I use a lot of inherited scenes from a base scene that contains a lot of complex tilemap settings (so I don't have to redo the settings every new scene). With a loaded inherited scene, if I change the node hierarchy(e.g. reparent a node to a new node) or rename any node, the editor will freeze for roughly 60+ seconds and then be fine again. Not sure what's causing it.

A non-inherited scene doesn't seem to be affected by this 60 second hangup behavior and behaves as expected when renaming nodes or changing the node hierarchy.

Currently using v4.3.dev4.official [df78c0636] build because updating to any newer 4.3 dev or beta builds causes this functionality to break. Issue does not exist in dev4 or previous versions.

One possible contributing factor is that I have roughly 60 scenes all inheriting from a single base scene template. This number of inherited scenes may be causing some yet undefined slowdown behind the scenes with dev6 and beta1 engine builds.

Only tested with 2D Node scenes--not 3D, if that's helpful

Thanks!

Steps to reproduce

Make a scene, then make a new inherited scene, save it and then edit the hierarchy or rename child nodes.

Minimal reproduction project (MRP)

N/A

akien-mga commented 1 month ago

Can you still reproduce the issue in 4.3.rc1?

If so, could you make a minimal reproduction project that exhibits the issue?

robcbryant commented 1 month ago

Let me download and test it in a minute!

robcbryant commented 1 month ago

Can confirm it's still the same issue in 4.3.rc1

Non inherited scenes work fine, but changing the hierarchy of an inherited scene does not. I attempted to replicate the issue with a new project by making ~20 inherited scenes from a base scene, and they're all fine--no hangups or lag. One issue with making a new project though is that I can't simply recreate the complexity of my scenes for a test project. It's far too much work. Each inherited scene might have 100 nodes or so in different arrangements and the tilemap nodes in them will have a lot of painted data associated with them (1000s of tiles).

My only gut reaction to this is that because it's from an inherited scene and not a new scene, there's some upper limit to the engine's ability to parse all the scene files when trying to manipulate complex scene that inherits from a complex scene.

Some new tests I did:

Opening a new fresh inherited scene in my project before it's saved (when it says unsaved ) -- has the same issue. If I try and rename the top node in the hierarchy it will freeze for a minute or so.

Opening the original scene the others inherit from and changing the node structure or renaming a node also hangs up the editor for around a minute.

The one thing that can be done is to add a new node. Adding any new node doesn't effect the editor--only when that new node is moved or renamed.

akien-mga commented 1 month ago

Thanks for testing. Without a reproduction project it's harder for us to debug, but I fully understand that in this case it seems difficult to make one, as the slowdown seems tied to the complexity of the project. So we might need more work from you to identify what causes the slowdown, if you're willing.

Some things which could be useful:

Can you confirm that 4.3.dev5 is the first release that causes the slowdown? You mentioned staying on dev4 and having the issue in dev6 but didn't mention dev5.
If you have a setup for compiling C++ (Visual Studio or MinGW), you could try to compile Godot from source to bisect the regression between the commits of the last good (dev4, df78c0636) and first bad (dev5, c9c17d6ca) snapshot, to find the exact commit which caused the issue. The problem is likely a performance bottleneck which might be easy to identify if we can review the commit which caused it.
Alternatively, if you're familiar with a C++ debugger, you could compiling Godot from source (latest master) with debug symbols (scons debug_symbols=yes), and when the editor seems frozen, break into the process with the debugger to identify where it's spending all this time, which might also help narrow down which commit caused the issue and which section of the code is problematic.

CC @KoBeWi

robcbryant commented 1 month ago

I did not test dev 5--Just did now--interesting, so with 4.3 dev 5 it still stalls---but for only about 15-20 seconds rather than like 45+ seconds.

I don't have a setup for visual studio right now, but I can look into it. I haven't done c++ compiling in ages.

It might take me a few days though.

KoBeWi commented 1 month ago

Can't reproduce it. Some test project, with clear instructions on how to cause the bug, would be helpful.

robcbryant commented 1 month ago

capture.csv Very Sleepy Monitor save file AND callgrind format.zip

Here's the debug monitor from the latest Engine source I built with debugging on. I used Very Sleepy because I'm on windows. Not sure what format works best (I'm not great at reading these things) so I just included them all. Sleepy's native format and callgrind format are in the zip file.

The first few seconds are the last bit of it loading the project, then I wait a few seconds of idling, and then try renaming a node in an inherited scene and it immediately halts and freezes the editor. While monitoring it seems to double the time it takes for the editor to unfreeze.

I'm also currently building a series of different commits to try and figure out the specific commit culprit. Might take a few hours or more

EDIT: If it's helpful here are the profile statistics: Duration: 177.906 Date: Thu Aug 01 09:53:47 2024 Samples: 489744

robcbryant commented 1 month ago

My project is currently stuck on rc1.

I moved to a laptop with a fresh git clone of my project for dev4 and none of the source builds I tried making in the bisect searching of the bug will even open my project because it can't import any of the resources--it just gives an error to the resources not existing and therefore cannot load in the console. It will do this indefinitely and then eventually crash out.

Unfortunately I have no way of testing any of my builds because my project just won't open with them.

I looked for documentation on what may be the case for this but I couldn't find anything.

rc1 did something to my core project files that seems to completely ignore my git versioning. I'm having trouble reopening my project in dev4 now. On my laptop--the same project experiences the same issue in the just released rc2. I'm currently trying to get my project to work on dev 4 again.

Willing to hear any advice on why Godot seems to be ignoring any github reversions/resets of my project to commits made on dev4.

akien-mga commented 1 month ago

Make sure to delete the .godot folder after resetting to the desired Git commit.

robcbryant commented 1 month ago

I've tried that method multiple times--sometimes it works, and sometimes it absolutely will not work and just gives an assortment of errors. the .godot folder isn't part of my GitHub project repository so none of the commits I manually download to test will load/import reliably if at all.

EDIT 1:

I get a neverending stream of errors of missing files, sometimes saying it needs to be opened in the editor before it runs (which it is being opened in the editor) and then often it reaches a point where it's trying to import tilesets and gives the nonsense below:

`ERROR: The TileSetAtlasSource atlas has no tile at (29, 30). at: (scene/resources/2d/tile_set.cpp:5400) ERROR: TileSetAtlasSource has no tile at (29, 30). at: (scene/resources/2d/tile_set.cpp:5348) ERROR: Cannot create tile. The tile is outside the texture or tiles are already present in the space the tile would cover. at: (scene/resources/2d/tile_set.cpp:4963) ERROR: The TileSetAtlasSource atlas has no tile at (30, 30). at: (scene/resources/2d/tile_set.cpp:5400) ERROR: TileSetAtlasSource has no tile at (30, 30). at: (scene/resources/2d/tile_set.cpp:5348)

EDIT 2: If I remove all of my scenes with tilemaps then open it as a new project in the editor, it will go through the errors of loading resources for all the objects but then open. If I then add the scenes with tilemaps it will have no issues. Something about the UIDs seems broken on loading a project without a .godot folder.

akien-mga commented 1 month ago

Could you check 4.3.rc3 to see if things improved by any chance?

Given your problem seems linked to TileMaps, I'm not too hopeful as there haven't been recent changes there, but it's worth testing anyway.

robcbryant commented 1 month ago

Sorry--I've been super busy and distracted so haven't had time to continue investigating.

I'm thinking it might be linked to how massive and complex my tilemaps are--thousands of tiles spread across multiple layers--I use multiple ones as well: Tilemap for Collision Tilemap for static art layers 1-2 tilemaps for background parallax 1 tilemap for foreground parallax

They all share the same core tileset defined in the original parent inherited scene that has about 10 different imported sprite maps that are each 1024x1024 in size sliced into 16x16 pixel tiles (It's a LOT of art). Looking at the raw scene in notepad+ is staggering lol.

Last thing I checked on my laptop--I got my build to work finally on it using the same godot version (dev4) and it seemed to be experiencing the same issue that doesn't happen on my main desktop using dev4 so I'm at an absolute loss

Only thing I can think to do when I have time is to continue trying to understand the profiler on the debug build I made on y'alls recommendation.

akien-mga commented 1 month ago

I'm bumping this to 4.4 as the 4.3 release is imminent, and I don't think we'll have the opportunity to fix this in time, as this is difficult to reproduce. It's still a nasty performance regression we should investigate and, once fix, we can backport the fix for a 4.3.x release.

robcbryant commented 1 month ago

Totally Fair! Thanks for trying at least on 4.3. I'll see what I can do when I have time to work on it again using the debug profiler.

Praytic commented 2 weeks ago

issue-92612.zip I have prepared my MRP for this issue. It contains a single tileset which was broken after auto-migration from 4.1 to 4.3 release. When you run the project you should see the following errors:

...
ERROR: Cannot create tile. The tile is outside the texture or tiles are already present in the space the tile would cover.
   at: create_tile (scene/resources/2d/tile_set.cpp:4963)
2024-09-01 19:40:27.029062-0700 godot.macos.editor.arm64[42807:6147515] ERROR: The TileSetAtlasSource atlas has no tile at (2, 1).
at: has_alternative_tile (scene/resources/2d/tile_set.cpp:5415)
ERROR: The TileSetAtlasSource atlas has no tile at (2, 1).
   at: has_alternative_tile (scene/resources/2d/tile_set.cpp:5415)
2024-09-01 19:40:27.029082-0700 godot.macos.editor.arm64[42807:6147515] ERROR: TileSetAtlasSource has no tile at (2, 1).
at: create_alternative_tile (scene/resources/2d/tile_set.cpp:5363)
...

Zireael07 commented 2 weeks ago

@Praytic I think you mixed up the issues - this is about freezes while renaming, not about a tileset being broken

godotengine / godot