search

splunk / splunk-operator

Splunk Operator for Kubernetes
Other
209 stars 115 forks source link

Unusable filesystem (GlusterFS) #95

Closed jmaas closed 4 years ago

jmaas commented 4 years ago

Hi, i'm trying to get a distributed splunk running on k8s with glusterfs storage. I see successful provisioning of PV and PVC. The Indexer pods however are not spinning up, the error i see on that end is: "unusable filesystem" 

STDERR:

homePath='/opt/splunk/var/lib/splunk/audit/db' of index=_audit on unusable filesystem.
Validating databases (splunkd validatedb) failed with code '1'.  If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue

Is the pod expecting a filesystem or raw block device? Or are there any other issues that can cause this?

Thanks!

mikedickey commented 4 years ago

This is not related to containers or k8s, but rather is a general Splunk Enterprise limitation. Please see https://docs.splunk.com/Documentation/Splunk/8.0.4/Troubleshooting/FSLockingIssues

jmaas commented 4 years ago

Could be the case, but how can I address this?

The way I see it, i'm using splunk-operator, docker-splunk and splunk enterprise and it just doesn't seem to work. All I need to know, i guess, is whether raw block devices for persistent storage are supported (i'm using gluster volumes)?

mikedickey commented 4 years ago

Splunk Enterprise does not support raw block devices. It requires a locking filesystem such as ext4 or xfs. GlusterFS is not supported. splunk-operator and docker-splunk have the same filesystem requirements as Splunk Enterprise.

jmaas commented 4 years ago

A more appropriate answer would have been: No the splunk docker image, ansible playbooks and whats not in there do not support detecting a raw block device and doesn't create a filesystem for you.

Effectively not supporting dynamic provisioning of block devices. Would you guys be willing to add such a feature?

mikedickey commented 4 years ago

I admit that I hadn't thought of that, and agree it's an interesting idea. Right now we're pretty focused on minimal requirements for a supportable GA release, but I could certainly see us considering this down the road. Please feel free to open another issue to track that (or even PR if you'd like to take a stab), although I suspect it would probably have to be implemented inside of either the docker-splunk container image, or the splunk-ansible playbooks that it uses.