search

threefoldtech / 0-fs

A new filesystem for zero-os that leverage on unionfs and a thin fuse layer to get the files on demand
Apache License 2.0
0 stars 1 forks source link

Performance bottleneck #18

Open maxux opened 5 years ago

maxux commented 5 years ago

After lot of usecase discovered and tested, we have now a better idea on how 0-fs is used and the high load it cas receive.

Some usecase showed that 0-fs is using lot of system resource to deal with simple workload sometime. It's a well-known fact from the beginning that using FUSE and overlay, etc. on top of each other, would not be the best way to reach good performance, but it could at least be used to proof that this system works great !

We need to keep in mind, that a system needs to be able to runs thousands of flist, without consuming all the system resources (spending most of the cpu time in the filesystem is not acceptable).

We already saw that, on a workflow like running a browser on top of 0-fs, the 0-fs process is using half of a single core, mostly all the time.

I think, in a near futur, we will need to move to something else. This needs to be discussed.

In my opinion, the best way (performance wise) is doing this fully in kernel space, but of course this is the most difficult and longer way to do it. Any other idea are welcome.

Ping @zaibon, @delandtj, @despiegk and @muhamadazmy for your input.

This issue is a step further than https://github.com/threefoldtech/0-fs/issues/1

muhamadazmy commented 5 years ago

I brought up and idea to @despiegk today that I want to rewrite the entire 0-fs in rust, for the same reasons u mentioned above.

Also we can try to do some performance tweeking on the current implementation, like drop the uninon fs mount and have everything done internally (not sure if this is will increase or decrease the performance)

zaibon commented 5 years ago

@muhamadazmy that sounds great. What would be the approach, still use fuse or do you have another idea ?

muhamadazmy commented 5 years ago

I think we will still use FUSE for this, it will take much longer (and trickier) to implement directly as a kernel module. Plus not all libraries are available inside the kernel (sqlite for exmpale, etc ...)

But for now I have a good idea that we should give a try. Currently for each flist you mount, the fuse is mounted ro (to access and download flist files), and then we have an overlay to a rw location where container, or a vm can modify the read-only layer. But this mean we only need a single ro layer per flist. In that case, if we run 10 zdb flists for instance we will have only one fuse mount point, with just 10 overlays layers.

This can save loads of memory, but extra care should be paid to when an flist is no longer needed, and can be unmounted.

This unfortunately will prevent us from doing some flist tricks like merging multiple flists in runtime... :disappointed:

zaibon commented 5 years ago

I don't think at the moment we have memory issue with 0-fs. So I don't think the trade off of losing the ability to merge flist at runtime is worth the gain for now.

maxux commented 5 years ago

[...] it will take much longer (and trickier) to implement directly as a kernel module. Plus not all libraries are available inside the kernel (sqlite for exmpale, etc ...)

Obviously some rethink would be needed. It's not acceptable IMO to have sqlite/capnp/etc. in kernel space, the tree should be sent from userspace and keep lower level structure in kernel.