libprojfs
A Linux projected filesystem library, similar in concept to the Windows Projected File System and developed in conjunction with the VFSForGit project.
While the libprojfs C library may also be used independently of VFSForGit, our primary goal is to enable users to run the VFSForGit client on a Linux system.
PLEASE NOTE: At present libprojfs is undergoing rapid development and its design and APIs may change significantly; some features are also incomplete, and so we do not recommend libprojfs for any production environments.
However, we wanted to share our progress to date, and we hope others are as excited as we are at the prospect of running the VFSForGit client on Linux in the not-too-distant future!
Current Status
| Branch | Functional Tests |
|---|---|
| master |
The library is under active development and supports basic directory projection, with additional features added regularly.
We will only make a first, official versioned release once initial development is complete.
A libprojfs filesystem can currently be used to mount a VFSForGit
MirrorProvider test client; we expect to continue our
pre-release development until the VFSForGit client proper is also
functional with libprojfs, at which point we may make an initial
tagged release.
Design
The libprojfs library is designed to function as a stackable Linux filesystem supporting a "provider" process which implements custom callbacks to populate files and directories on demand. This is illustrated in the context of a VFSForGit provider below:
Actual file storage is delegated to a "lower" storage filesystem, which may be any Linux filesystem, e.g., ext4. At the time when a libprojfs filesystem mount is created, there may be no files or directories in the lower filesystem yet.
As normal filesystem requests are made within the libprojfs mount
(e.g., by ls or cat), libprojfs intercepts the requests
and queries the provider process for the actual contents of the
file or directory. The provider's response is then written to
the lower filesystem so future accesses may be satisfied without
querying the provider again.
See our design document for more details.
We anticipate that whether libprojfs remains a FUSE-based library, or becomes a libfuse-like interface to a Linux kernel module, it may be useful for purposes other than running a VFSForGit client.
For this reason, we have tried to ensure that our native event notification API is aligned closely with the Linux kernel's fanotify/inotify/fsnotify APIs.
Getting Started
Please see our detailed Build and Installation
page for step-by-step instructions on how to build and install
libprojfs and its dependencies on Linux, and (optionally) also build and
run the VFSForGit MirrorProvider test utility with libprojfs.
So long as libprojfs remains based on FUSE, the primary
dependency for libprojfs is the user-space libfuse library,
as well as having the Linux fuse kernel module installed.
Support for user.* extended attributes must also be enabled
on the "lower" storage filesystem. Your Linux kernel should have
been compiled with the necessary configuration options (e.g.,
CONFIG_EXT2_FS_XATTR for ext2/3/4) and the underlying filesystem
you use for libprojfs should be mounted with appropriate settings
(e.g., user_xattr for ext2/3/4).
If you are using a Docker container, you
may need to ensure the fuse kernel module is installed on your
host OS, and user.* extended attributes are supported on your host's
filesystem. See the Using Docker containers section for
more details.
To test libprojfs with the Microsoft VFSForGit MirrorProvider
(as we do not support the primary VFSForGit GVFS provider yet), .NET Core
must be installed and parts of the VFSForGit project built. See the
Building and Running MirrorProvider section for details.
Contributing
Thank you for your interest in libprojfs!
We welcome contributions; please see our CONTRIBUTING guidelines and our contributor CODE_OF_CONDUCT.
Licensing
The libprojfs library is licensed under the LGPL v2.1.
See the NOTICE file for a list of other licenses used in the project, and the comments in each file for the licenses applicable to them.
Development Roadmap
We are developing the libprojfs library first, using FUSE to prototype and test its performance, before migrating functionality into a Linux kernel module (assuming that proves to be necessary to meet our performance criteria).
For more details on the planned development phases, see our design document.
Authors
The libprojfs library is currently maintained and developed by several members of GitHub's Engineering organization, including:
You can also contact the GitHub project team at opensource+libprojfs@github.com.