Open a-prokopyev-resume opened 4 days ago
Hi, thanks for submitting this issue/request.
I am investigating MooseFS and HAST as potential hyper-converged options for the hoster
storage itself, and there are already CSI adapters that work with ZFS and MooseFS.
The setup you describe is very cool on paper/in theory, but overly complicated (at least it is to me), as troubleshooting any issues from that setup might become a nightmare in the long run. I am a really big fan of simple, but feature-complete systems, that's why I chose to develop under FreeBSD in a first place.
Here are my thoughts on how to improve and simplify your setup, but still keep the core concept - K8s with hyper-converged and highly available storage:
All in all, I recommend keeping things simple - such setups are easier to maintain in the long run and easier to troubleshoot in case you are having issues.
Hello,
What do you think about possibility of integrating your
bhyve
orchestrator into XygmaNAS to produce something similar toSmartOS
immutable OS if using only LiveCD version ofXygmaNAS
?Then we could create a VM with Talos Linux on several
XygmaNAS
nodes, install Cozystack into this K8s cluster and get a HA hyperconverged cloud-native HA storage managed by Linstor PiraeusK8s
operator.Actually
Cozystack
already includes many cloud-native components likePiraeus
operator and easily installed into a readyTalos
K8s
cluster bootstraping of which could be automated by some tools likeAnsible
.If having a VM guest with two block devices: 1) Give first block device to Talos for its local storage for K8s 2) Give second block device directly to
DRBD
managed byPiraeus
inCozystack
WITHOUT creatingLVM
orZFS
pool forLinstor
.This VM is going to be run in a
bhyve
onFreeBSD
XygmaNAS
, which will pass through twoZFS
zvols into VM as block devices to be used forTalos
(first) and forDRBD
(second). I just need to enable HA storage using 2-3 nodes with a such VM on each of them. It would be a so called hyperconvereged cloud-native storage on the same hosts (but in VM guests) whereXygmaNAS
itself runs.