microsoft / ProjFS-Managed-API

A managed-code API for the Windows Projected File System
Other
142 stars 34 forks source link

ProjFS + Copy-on-Write #96

Open forrestthewoods opened 2 months ago

forrestthewoods commented 2 months ago

Microsoft DevDrive supports copy-on-write semantics which is super cool. I'd like combine ProjFS + CoW so that large binary assets can be shared.

Currently the way ProjFS works is you call PrjWriteFileData. What I'd like to do instead is perform a block clone.

Is there any way to do this today? I believe the answer is no. But I wanted to check.

In case it helps, my intended use case is an experimental version control system. I'd like to support monorepos with very large files which are deduplicated via block cloning.

Thanks!

cgallred commented 1 month ago

It isn't quite clear to me what you're doing.

forrestthewoods commented 1 month ago

It isn't quite clear to me what you're doing.

Imagine a version control system with a CAS cache. The cache may contain a very large, multi-tens of gigabyte file containing AI model weights. I don't want to make a copy of those bytes on the projected client. Similarly, consider a cache containing large files like LLVM toolchains. I want multiples repos to be deduplicated and share the same bytes from the CAS cache. And I want a virtual file system so that large repos only need to remotely fetch files for the cache if and as they are needed.

Does that help?