search

seaweedfs / seaweedfs-csi-driver

SeaweedFS CSI Driver https://github.com/seaweedfs/seaweedfs
Apache License 2.0
210 stars 50 forks source link

Behaviour of seaweedfs-csi-driver with 2 clusters #124

Closed rohitchormale closed 1 year ago

rohitchormale commented 1 year ago

I have a kubernetes setup where 2 seaweedfs clusters are installed in same namespace. Seaweedfs-csi-driver is pointed to only one cluster but I am noticing sometimes system input/output errors. This never happend with single cluster. So my question is - will seaweedfs-cs-driver work properly in this setup ? Is there any possible conflict I should check ?

chrislusf commented 1 year ago

no clue. need to reproduce and debug.

rohitchormale commented 1 year ago

@chrislusf thanks for the reply. I will try to get you steps soon. More details on setup -

Basically I have a bare git repos on seaweedfs and I am mouting seaweedfs on multiple pods via csi driver. In cluster, there are 3 filers but csi driver is pointed to only one filer.

On few pods, few files from git objects are missing. These are very small files. If I run git fsck, I can see unable to mmap ./objects/00/blah-blah-blah no such file or directory. object currupt or missing.

I am just wondering why those are missing only from few pods, not from all. And is there a way to sync them again ?
Restarting pods solves my problem but that option is not alway available.

rohitchormale commented 1 year ago

It turns out having 2 clusters has nothing to do with our issue. It looks like issue was happening because of slow initial syncing. After inserting some delay in out operations, it got resolved.