ContentProvider Support for Resource Packages

[ilexp]

As mentioned by @SirePi here, it would be nice to have a way of packaging together Resources in a container single file. This could be implemented easily in the ContentProvider class.

• Introduce a Resource Package file format (file extension .rpk maybe?), which is essentially a disguised .zip as usable by System.IO.Compression.
• Any package file is treated as a virtual directory, which can be addressed as usual with path strings.
  • The file Data\SomeDir.rpk would be treated the same as Data\SomeDir.
  • The Resource path Data\SomeDir\SomeTex.Texture.res would work all the same, regardless of whether SomeDir is a directory, or a SomeDir.rpk package file.
• Provide a LoadPackage method that allows to load all Resources inside a Resource path.
  • It shouldn't matter whether Data\SomeDir actually resolved to a package file or a directory.

Reflect on this and its implications a bit more until implementing. This is a non-breaking addition to the framework. No need for pre-v2.0 implementation.

[hsnabn]

For file extension, how about .res.pak? It sounds like a better fit for Duality.
EDIT: Also, how about putting multiple Prefabs into a singular Prefab/Package? Sorry if this is the wrong place for this.

[ilexp]

For file extension, how about .res.pak? It sounds like a better fit for Duality.

To avoid confusion, I explicitly want to exclude any .res file endings - because it's not really a Resource itself / .res.

EDIT: Also, how about putting multiple prefabs into a singular prefab/package? Sorry if this is the wrong place for this.

A Resource package is a mostly virtual construct that can contain any amount of Resources. However, if you want a Prefab to contain multiple objects, you'll need to group them using a shared parent object.

[hsnabn]

To avoid confusion, I explicitly want to exclude any .res file endings - because it's not really a Resource itself / .res.

I think it clears up confusion more, actually! .res.pak immediately translates to Resource Package for me, at least. .rpk however, does not have the same "describe-ability", in my opinion.

A Resource package is a mostly virtual construct that can contain any amount of Resources. However, if you want a Prefab to contain multiple objects, you'll need to group them using a shared parent object.

Oh not like that, I was thinking of more like a group of Prefabs in the same "package". But I guess you can put Prefabs into a Resource Package, since they are Resources. Never mind.

[ilexp]

I think it clears up confusion more, actually! .res.pak immediately translates to Resource Package for me, at least. .rpk however, does not have the same "describe-ability", in my opinion.

I think .rpk does a better job in distinguishing it as a completely different file format, especially because of the difference to .res files you mentioned: .res should probably be reserved to Resources, and a Resource Package is - despite containing Resources - not a Resource itself.

Wouldn't rule out .res.pak yet, but am not entirely convinced here. I'd prefer something easily distinguishable.

But I guess you can put Prefabs into a Resource Package, since they are Resources.

Yup :)

[hsnabn]

No problem, since this is not a breaking change, we can sit on it for a while.

I noticed something else, though:

• Any package file is treated as a virtual directory, which can be addressed as usual with path strings.
  • The file Data\SomeDir.rpk would be treated the same as Data\SomeDir.
  • The Resource path Data\SomeDir\SomeTex.Texture.res would work all the same, regardless of whether SomeDir is a directory, or a SomeDir.rpk package file.

Also this:

• It shouldn't matter whether Data\SomeDir actually resolved to a package file or a directory.

Say, for example, there were both a directory and Resource Package with the same name. What would happen then?

[ilexp]

Say, for example, there were both a directory and Resource Package with the same name. What would happen then?

Good question! I'd say the directory is preferred and the package ignored, as this would allow devs to quickly "extract" a Resource package and work with the directory - without having to delete the package in order to do so.

[hsnabn]

Either way, shouldn't there be a way to specify package or directory? Yes, I understand, we should probably refrain from using the same name, but it wouldn't hurt to have a way to specify exactly what we should be accessing, yes?

[ilexp]

Either way, shouldn't there be a way to specify package or directory? Yes, I understand, we should probably refrain from using the same name, but it wouldn't hurt to have a way to specify exactly what we should be accessing, yes?

As the packaging mechanism is designed to be transparent, there actually shouldn't be a way to specify this, as this would suddenly make this implementation detail of the content backend visible to the average game plugin programmer.

Different environments may require different project structures in Duality, some of which may support packaging, others might use it as the only way for content access, and a third one might do something entirely different. The trick to keep them all together is by adding an abstraction layer - such as the virtual content paths and the ContentProvider API. Letting implementation details that may vary in different environments "seep through" has the potential to introduce problems when switching from one to the other.

[hsnabn]

Ah OK, I understand. It's not a very important point, anyway.

[Xinayder]

It would be nice if we could use Resource Packages to pack our game files, instead of having everything under the Data directory. EDIT: just realized it would be possible :P.

Introduce a Resource Package file format , which is essentially a disguised .zip as usable by System.IO.Compression.

What about using zlib or GZip to compress these files?

[ilexp]

What about using zlib or GZip to compress these files?

.Net has a System.IO.Compression library, which features Zip compression. This will likely be the default choice here :)

[hsnabn]

.Net has a System.IO.Compression library, which features Zip compression.

I think this is the best bet as well. You might have to introduce external libraries to use zlib or gzip (I've said this many times before, but don't take my word on it. I'm not very versed in stuff like this (yet)).