godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.12k stars 69 forks source link

Add capability to share imported resources #9031

Closed dhoverml closed 7 months ago

dhoverml commented 7 months ago

Describe the project you are working on

Several different projects that use the same addons and resources. The projects use symlinks to reference the shared directories so that they appear local to each project within Godot.

Describe the problem or limitation you are having in your project

Since Godot treats each resource as its own copy, it creates a local imported copy of that resource. N projects means N copies, when only 1 copy would achieve the same result. This is benign until there are thousands of shared resources.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Let Godot check the path found in .import file. If the path exists, then use the files there, rather than re-import and overwrite the .import file. Even though there's currently no way to specify this path in the editor, it will at minimum allow that particular feature to work correctly when/if it is added. At minimum, manually setting shared resource paths will work.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

When determining the path to store the imported resources, check if path is valid and, if it is, use that instead. https://github.com/godotengine/godot/compare/master...dhoverml:godot:keep-import-path

If this enhancement will not be used often, can it be worked around with a few lines of script?

Almost. Manually editing the .import file to point to the shared resource works, however Godot overwrites the .import file if it cannot find the import under .godot/imported. So the workaround for that is to set the importer to keep, which works great until I try to export my project (the exporter ignores the import file when it sees keep)

Is there a reason why this should be core and not an add-on in the asset library?

The editor-importer is part of core.

AThousandShips commented 7 months ago

Are we talking sharing files from outside the project directory? As in the same files imported into multiple projects? Or are we talking about the resulting import data being shared?

What copies do you mean exist?

dhoverml commented 7 months ago

Are we talking sharing files from outside the project directory? As in the same files imported into multiple projects?

Yes

Or are we talking about the resulting import data being shared?

Yes, but only between projects locally. Exported apps can use their own copy.

What copies do you mean exist?

The imported files that you'd find in .godot/imported. As in, every project that uses the shared resource will generate the same copy of the imported file.

Here's an example of how to setup shared resources that use exactly one copy of the imported file. Also, this is not as tedious as it seems with some bash scripting.

  1. Create a png, and put it here: ~/my_shared_resources/my_png.png
  2. Then reference the shared resource in a project with a symlink:
    ln -s -r ~/my_shared_resources ~/myproject/

    And so this path now exists, through the symlink:

    ~/myproject/my_shared_resources/my_png.png
  3. Open the project:
    ~/path-to-godot/godot --editor --path ~/myproject
  4. Notice that the editor imports the png, which generates
    
    ~/myproject/.godot/imported/my_png.png-GENERATEDSHA.ctex
    ~/myproject/my_shared_resources/my_png.png.import

this is the same .import file:

~/my_shared_resources/my_png.png.import

6. If you open `my_png.png.import`, you'll see this (there are a few occurrences of `res://.godot/imported`, this is one of them).

path=res://.godot/imported/my_png.png-GENERATEDSHA.ctex

7. Maybe we want to share this png, in which case we don't want a separate `.godot/imported/my_png.png-GENERATEDSHA.ctex` for each project.
So, move it to the shared directory instead:
``` bash
mkdir ~/my_shared_resources/imported
mv ~/myproject/.godot/imported/my_png.png-GENERATEDSHA.* ~/my_shared_resources/imported/
  1. Rename the path in the import file to point to the imported file in my_shared_resources
    sed -i 's|res://.godot/imported/|res://my_shared_resources/imported/|g' ~/my_shared_resources/my_png.png.import
  2. At this point, we can point other projects to the same shared resources, and they (if this proposal is implemented) will not import their own copy. Instead they will re-use the already-imported resource(s) in res://my_shared_resources/imported/.... For example:
    ln -s -r ~/my_shared_resources ~/my_other_project/
    ln -s -r ~/my_shared_resources ~/and_another_project/
    # then open any of these projects, and they'll use the imported file(s) under 
    # ~/my_shared_resources/imported instead of generating their own copies
AThousandShips commented 7 months ago

That's going to be very error prone and breaks the basic concept of the project being self contained

This seems extremely niche and specific and means a lot of possible error cases

Also this assumes all the import details are equal, also what happens when the import details on the other project change? How do you detect that they match? Because the imported result differs

Also there's imports that depend on other imports and they can be deeply nested, how would you handle connecting those?

Calinou commented 7 months ago

As mentioned above, the issue is that you could only share imported resources if both the source resource and all of its import options are the exact same. This is bound to be very unlikely for more than two projects on the same PC.

If you are low on disk space, consider removing the .godot/ folder of projects you haven't edited for a while.

dhoverml commented 7 months ago

This seems extremely niche and specific and means a lot of possible error cases

I had a feeling that would be the case, though no worries!

Also this assumes all the import details are equal, also what happens when the import details on the other project change? How do you detect that they match? Because the imported result differs

So far every project I've done this with doesn't need different imports.

Also there's imports that depend on other imports and they can be deeply nested, how would you handle connecting those?

I haven't ran into those; do you know which import types would do this?

Also, just curious, did you take a look at the potential PR (this one)? As in, does this PR make sense as a standalone change, or do we always want to enforce that the import directory is ~/.godot/imported?

AThousandShips commented 7 months ago

I see no reason to change the import path, it needlessly complicates things

For nested imports just look at atlas textures, the imported texture is put inside an atlas file and the resulting imported texture points to the atlas, not the original texture, the atlas is a generated file and would be in each project

dhoverml commented 7 months ago

Okay, thank you for the input. I appreciate your time; I'll close this one.