socketfuse

[bAndie91]

Hi Vitaly, i like execfuse and used it for many purposes already. I'd like to address the performance concern which is in execfuse's design. I imagine a tool like execfuse, let's call is socketfuse, because it'd relay fuse calls not by executing commands but by talking to a long-running process via socket (or on other type of IPC if there is more suitable). protocol would be line-based to ease users to write simple scripts. the long running programm would be managed by socketfuse: run it when socketfuse starts, optionally restart when exists abnormally, and terminate on unmount. it reduces the cost of process spawning and save/restore internal state, while keep full control over the filesystem logic. it could leverage the linux feature with which a process can pass an open file handle to an other via unix domain socket. what's your opinion?

[vi]

Related: http://ccgfs.sourceforge.net/

My own contribution: https://github.com/vi/ccgfs-win-storage

[vi]

If you want to contribute to that "socketfs" or "ipcfs" or whatever, you can design a protocol using some IPC-generator (based on protobuf or whatever) that generates client and server stubs for various programming languages (notably, C or Rust). Maybe I'll link it together with FUSE.

Note that it may be worth to design it around low-level FUSE (where inodes are exposed), not where filenames are transferred all the time.

[bAndie91]

noted. ccgfs is really a flexible tool. would not you find a high-level fuse middleware with line-based protocol to be useful? IMO with a low-level fuse interface users (who implement their filesystem logic in whatever language) have to implement more than it's usually neccessary for a "quick sketch" fs which they can write even in bash. for me it's handy that fuse lib cumputes inodes for me.

[vi]

Maybe.

Do you have some specific design of such line-based protocol in mind? For example, how non-UTF8 filenames that contain newlines and other control characters inside would be handled?

Shall it cut corners to be easier for quick prototypes at expence of correctness; or the protocol should emphasise safety and be ready to support remote running of any possible FUSE filesystem?

[vi]

Maybe related: https://github.com/LK4D4/grfuse

[bAndie91]

design of such line-based protocol

i'd pass parameters on stdin just like exec arguments, '\0'-separated strings, first byte indicating how many arguments follow. this allows fast prototyping, say your fs driver written in bash, then instead of newline-based read arg, call read -d '' arg to read the next argument; and naughty filenames also work.

i'm fascinated on remove FS. the only thing bothers me is that fuse_context->uid loss its meaning.

[vi]

fast prototyping driver written in bash

So is your requirements is being able to write crude filesystem driver as a long-running Bash script (as opposed to multiple mini-scripts per each function)?

Would bashfs - a Bash library + C-based driver, without line-based protocol be better for this? You write a Bash script with a set of functions, then source a bash library - it executes all necessary parts and calls your Bash functions.

remove FS

What is "remove FS"? Maybe "remote FS"?