Provides a POSIX compliant file system mount of an OpenStack Swift Object Storage Container
ProxyFS is a disributed read/write hierarchical POSIX file system layered on top of Swift object storage. File system accesses map to Objects inside a Swift Container.
All code contributions for ProxyFS go through GitHub.
https://github.com/NVIDIA/proxyfs
Please feel free to contribute by opening a pull request to the
development
branch. If you see an open pull request, feel free to
review it and leave comments on it. ProxyFS follows a
git flow
development model.
If you'd like to ask questions, discuss ideas, or otherwise communicate with other ProxyFS contributors, please register for and participate in the ProxyFS dev mailing list or the ProxyFS Slack group, which you can join through this inivitation.
ProxyFS develoment/building leverages Docker Containers and Docker Compose.
A Multi-Stage Dockerfile
is provided that builds the following three images:
dev
- used to build, debug, and unit test a simple ProxyFS deploymentbuild
- used to provide a clean build environment for constructing deploy
deploy
- a space/dependency efficient deployable Docker Container holding:
ickpt
- an optional, replicable, strictly consistent service to store ProxyFS checkpoints that are, otherwise, the only element of a ProxyFS file system subject to the Eventual Consistency risk of using OpenStack Swift as a storage back-endimgr
- a service capable of managing the metadata for a collection of ProxyFS file systemsiclient
- the FUSE service deployed to support the mounting of a ProxyFS file systemTo build the images:
To instantiate either a development or test environment, a docker-compose.yml
is provided that instantiates the following Docker Containers:
swift
- a deployment of a Swift "All-in-One" Container instancedev
- an instance of the dev
image with your ProxyFS repo dir mapped to /src
ickpt
- an instance of the deploy
image that launches the ickpt
serviceimgr
- an instance of the deploy
image that launches the imgr
serviceiclient
- an instance of the deploy
image that launches the iclient
service setup to present its FUSE mount point at /mnt
To kick off development activities:
To build all the images:
dev
/src#] makeTo clear out prior deamon runs and run the daemons in the background:
dev
/src#] rm -rf /tmp/ickptDBdev
/src#] ickpt/ickpt ickpt/dev.conf &dev
/src#] imgr/imgr imgr/dev.conf &dev
/src#] idestroy/idestroy iclient/dev.confdev
/src#] imgr/mkmount.sh -fsdev
/src#] iclient/iclient iclient/dev.conf &Notes:
idestroy
step will fail with httpGETResponse.Status unexpected: 404 Not Found
if this is the fist iteration through rm -rf /tmp/ickptDB
and idestroy...
steps-s
rather than -fs
to imgr/mkmount.sh
For a more appropriate environment in which to perform functional testing, the docker-compose.yml
file my also be used to launch the suite of Docker Containers:
At this point, you have a separate Docker Container running each of the ProxyFS services. Inside the iclient
Docker Container, the /mnt
directory will be the FUSE mount point to your ProxyFS file system.
For either the up cases (i.e. dev
or iclient
), both the imgr
and iclient
services provide an HTTP endpoint that, due to the port mapping specified in docker-compose.yml
, should be reachable from a browser in the host:
imgr
- http://localhost:15346/iclient
- http://localhost:15347/See LICENSE file