Flesh out plans for ResourceManager

[DanRStevens]

First up, I think ResourceManager needs to be rename. As per the Clean Code Collection, "Manager" is a suspect name. Considering what the class does, perhaps VirtualFilesystem might be a better name. The class's responsibilities are similar in some ways to what the PhysFS library does, which is basically a virtual filesystem, so some inspiration may come from there.

A virtual filesystem could support mounting of various sources into an integrated view. That may include archives, such as VOL and CLM, or a subtree from the real filesystem. The result would appear as if it was a single folder subtree. Mount points appear as folders in the virtual filesystem.

Example view of VirtualFilesystem folder tree:

realFile1.txt
realFolder/realFile2.txt
archive1File.txt
archive2File.txt
archive3Folder/archive3File.txt  (optional (done), folders within archives)
mountPoint4/archive4File.txt  (optional, archive mounted at non-root folder, with no contained subfolder)
mountPoint5/realFile5.txt  (optional, real folder mounted at non-root folder)

Side note: Outpost 2 does not have non-root mount points. Hence why I've marked that part as optional. Still, it may be worthwhile to consider the more general case.

Mount points should be allowed to overlap, with identically named files hiding others of lower priority. It is not an error if two files mounted from different sources end up at the same virtual path. Only one will be visible. That would be consistent with how Outpost 2 handles accessing files from VOL archives. All archives are mounted to the root of the virtual filesystem. During search it returns the first file it finds when searching all archives. In Outpost 2, the folder tree representing the filesystem has the highest priority, with VOL files coming next, depending on the order in which they were loaded.

Example: In priority order, a real filesystem subtree, archive1.vol, archive2.vol, are all mounted to root. Contents of filsystem subtree:

• File1.txt
• File2.txt

Contents of archive1.vol:

• File2.txt
• File3.txt

Contents of archive2.vol:

• File2.txt
• File3.txt
• File4.txt

This would result in the following virtual filesystem view:

• File1.txt (from real filesystem folder)
• File2.txt (from real filesystem folder, hides "File2.txt" files in archive1 and archive2)
• File3.txt (from archive1.vol, hides "File3.txt" from archive2)
• File4.txt (from archive2.vol)

---

Access to resources is done using relative filenames, which are relative from the root of the virtual filesystem. If the file corresponds to a real disk file, the real filesystem path can be obtained by stripping the virtual mount path and then prepending the real filesystem path of the subtree mount point.

Example: The real filesystem contains a file: resources/data/image.bmp. The folder resources/data/ is mounted into the virtual filesystem as imageData/. Access to the file is through imageData/image.bmp. To convert to a real filesystem path, first strip the mount point imageData/: imageData/image.bmp -> image.bmp Then prepend the real filesystem path of the mounted subtree resources/data/: image.bmp -> resources/data/image.bmp

---

Related: Issue #232.

---

Responsibilities of VirtualFilesystem are being able to view, search, and open streams to the contained files, using a single unified virtual filesystem path. This corresponds quite well with the current methods offered by ResourceManager, which are able to search paths and open streams.

In terms of future direction, we can rename the class, generalize how files are mapped in from the mount sources (perhaps defining a FolderSubtree interface), perhaps expand methods to support viewing of the virtual filesystem, and at some point down the road, if we feel like it, or want to reuse the class in another project, we could add non-root mount points. That last part is not a priority.

[Brett208]

Thanks for writing up your thoughts. Two questions:

When you say: The folder resources/data/ is mounted into the virtual filesystem as imageData/

Does this mean you can somehow rename the mounted directory? Would this be an override to the constructor that allows specifying the real directory mounted and the new name?

If we implement multiple mount points, what takes priority? I think the answer is the most recently mounted directory working backwards to the original mount used in the constructor. This would best mimic how Outpost 2 does it when we invoke it with the ConsoleModLoader.

Thanks, Brett

[DanRStevens]

Yes, the names in the virtual filesystem don't need to correspond to names on disk, making it possible to rename things.

Keep in mind, the example was for the general case with non-root mount points. I don't propose implementing that at this time. Rather, I'd like to stick with a design that allows that to be implemented in the future.

Modifying the above example so it mounts the real folder resources/data/ to the root of the virtual filesystem, the real filesystem file resources/data/image.bmp would be seen within the virtual filesystem as image.bmp. To translate this to a real filesystem name, we don't need to worry about stripping the mount point prefix (since it is root), and can instead just append the real filesystem folder: resources/data/ + image.bmp => resources/data/image.bmp

This is the simple case, which is sufficient to support working with Outpost 2.

---

An additional example. We could also mount resources/ at the root of the virtual filesystem. Then we have access to data/image.bmp. Converting back to a real filesystem name, using the real mount folder name: resources/ + data/image.bmp => resources/data/image.bmp

---

Priority is based on order in which the mount points are added to the virtual filesystem. The virtual filesystem would scan it's list of sources, in order, and find the first matching name, or return an error if none was found.

We may want to provide methods to insert new sources in a particular position. PhysFS has such a mechanism. An example is methods to add first (push_front), add last (push_back), or insert somewhere in the middle at position i (or at the position given by an iterator).

[DanRStevens]

Another side idea, we could have a filtered folder tree which gets mounted. Maybe you only want access to files of a specific type. Example (with non-root mount points):

class FilteredSubTree {
  FilteredSubTree(const std::string& basePath, const std::string& fileGlob);
};

class VirtualFilesystem {
  Mount(SubTree& subTree, const std::string& mountPoint);
};

virtualFilesystem.Mount(FilteredSubTree("data/", "*.bmp"), "images/");
virtualFilesystem.Mount(FilteredSubTree("data/", "*.map"), "maps/");

In the above, the real filesystem might mix *.bmp and *.map files in the same data/ folder. The virtual filesystem would separate out these two file types into subfolders images/ and maps/.