godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.25k stars 21.22k forks source link

Text resource files saved with too long lines, VCS treat files as binary #16783

Closed nadnam closed 3 weeks ago

nadnam commented 6 years ago

Godot version: 3.0

Issue description: When saving text resource, eg. scene (.tscn file) with exported var of eg. type array that contains large number of elements, resulting text file will end up with too long lines as array is stored as single line, regardless of number of elements. Linux file command would recognize them as "ASCII text, with very long lines".

Problem is two fold with common VCS like Git, Fossil-SCM etc.

  1. They will treat these files as binary (even though they are text) so diff will fail to show changes
  2. VCS will store changes between files not as textual delta but as binary change, which is orders of magnitude larger and usually equals compressed size of file itself. This means that developer will unknowingly and unexpectedly grow repository size orders of magnitudes more with each commit.

Eg. existing committed scene file of 500Kb size (example), with one word or value changed, will take up with next commit not ~50bytes as text delta but whole 500Kb (or compressed ~100Kb, depending on VCS, compression and ways it stores deltas).

Suggested solution: To break up lines to reasonable length when possible. It would solve problem with exported vars of type array, dict. etc.

Calinou commented 6 years ago

Note that large scene and resource files (typically > 500 KB) should be saved as binary instead of text, which helps decrease saving and loading times. Scenes can be converted to a binary format when exporting, but you still want your project to open and save quickly while developing.

nadnam commented 6 years ago

Thanks for info, but I was mentioning 500KB file size only as an example. In practice I encountered this problem with .tscn files with size less than 10KB that contain array of array few basic types ([[Vector3, Vector3, int], ...]), with 50 or 100 elements. These are extra info set on scene using custom editor plugin (few waypoints for AI), and in no way huge amount of data nor time critical for loading speed.

Not being able to see changes in VCS diff is main issue here, as changing anything in scene makes whole file useless for version control history.

eon-s commented 6 years ago

Some resources should be better structured in the resource files (tscn and tres), not only for this but also for readability (curves and meshes are too hard to read).

bojidar-bg commented 6 years ago

Related #12925.

KoBeWi commented 3 weeks ago

This sounds more like Git issue than Godot one. I never encountered this issue myself, even though I often get super long lines due to TileMaps, so maybe it's no longer relevant for some reason. In any case, I think this should be a proposal.