search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
14.86k stars 1.95k forks source link

CSI hostpath plugin not working #18678

Closed scrhicks closed 1 year ago

scrhicks commented 1 year ago

Nomad version

Nomad v1.6.2 BuildDate 2023-09-13T16:47:25Z Revision 73e372ad94033db2ceaf53468b270a31544c23fd

Operating system and Environment details

MacOs m1 nomad cluster is running inside multipass instance on linux 22.04

Issue

I'm doing everything what is here: https://github.com/hashicorp/nomad/tree/main/demo/csi/hostpath but instead of having volumes mounted to the container i have an error.

§Setup Failure: failed to setup alloc: pre-run hook "csi_hook" failed: mounting volumes: volume "861b8a01-642c-11ee-851e-0242ac110002" could not be found: rpc error: code = NotFound desc = volume id 861b8a01-642c-11ee-851e-0242ac110002 does not exist in the volumes list§

I also tried to update to the latest version but it did't help.

Reproduction steps

https://github.com/hashicorp/nomad/tree/main/demo/csi/hostpath

Expected Result

Volumes are mounted

Actual Result

Error occur

Job file (if appropriate)

DevOps

Nomad Server logs (if appropriate)

Nomad Client logs (if appropriate)

FourLeafTec commented 1 year ago

Same problem with me.

And occasionally it becomes OK

jrasell commented 1 year ago

Hi @scrhicks and @FourLeafTec; unfortunately due to limitations with the CSI specification and hostpath plugin, the demo will only work correctly where you have a single client running. On clusters with multiple clients, you will see the random behaviour you have experienced. This is because the hostpath plugin can only create volumes that exist on a single client. This implementation is not covered by the CSI specification and the hostpath plugin has no way to tell Nomad that information.

If you're looking to test CSI without complex external dependencies, I would suggest checking out the democratic-csi implementation. I have opened a linked PR to clarify the demo readme content to point out the limitation.