search

knative / serving

Kubernetes-based, scale-to-zero, request-driven compute
https://knative.dev/docs/serving/
Apache License 2.0
5.57k stars 1.16k forks source link

Support volume type "hostPath" for volume mounting #15546

Open vthurimella opened 1 month ago

vthurimella commented 1 month ago

In what area(s)?

/area API

Describe the feature

Adding support for the "hostPath" volume type in Knative Serving would allow users to mount directories from the host node's filesystem into their Knative service containers. This feature would enable access to local storage on the node, facilitating use cases such as accessing node-specific data or utilizing local caches, while maintaining the serverless benefits of Knative.

skonto commented 1 month ago

Hi @vthurimella see comment here https://github.com/knative/serving/issues/13690#issuecomment-1444055308. Could you elaborate on this one? Is it an AI use case?

vthurimella commented 1 month ago

My use case involves scientific workflows with AI components. We use Knative as a dynamic resource allocator across a shared cluster. Our Knative tasks generate files that must be written to the specific node where execution occurs. We have a robust system for file tracking and inter-node file transfer. Without hostPath support, we cannot efficiently manage data locality, which is crucial for our workflow performance (right now we have the task call a file service on the execution node that has a hostPath mounted). This feature would significantly help me integrate Knative into my application.

skonto commented 1 month ago

@dprotaso gentle ping, wdyth?

vthurimella commented 1 month ago

Thank you for considering this feature request! I understand that adding hostPath support might not be feasible for the main project due to various constraints or design considerations.

If it's not possible to integrate this into Knative directly, I would appreciate any guidance on how I could modify the code myself to add the hostPath functionality for my specific use case and deploy it in my own application. This way, I could get the functionality I need for my installation without affecting the broader project.

Any information on which parts of the codebase I should focus on, or any architectural considerations I should keep in mind, would be extremely helpful. This would allow me to tailor the solution to my applications needs while maintaining the core benefits of Knative in my setup.

rhuss commented 1 month ago

I think It makes sense to allow hostPath volumes, similar to support persistent volumes. This can be hidden behind a feature flag.

@vthurimella looks for kubernetes.podspec-persistent-volume-claim in the source, starting from pkg/apis/config/features.go and follow its usage trail. You should be able to implement hostPath in a similar way.

@vthurimella, just out of curiosity, what is the reason for choosing Knative as resource allocator ? Which feature of Knative serving is the most important to you ?

skonto commented 1 month ago

@vthurimella You can have hostPath access via a pvc however there are limitations see discussion here, this has been asked multiple times before. @dprotaso what is your take on this one? Should we stay opinionated here or add it as a feature as discussed before?

dprotaso commented 1 month ago

I think it should be fine to add behind a feature flag.

skonto commented 4 weeks ago

/assign @skonto

skonto commented 4 weeks ago

@amarflybot hi, are you interested on working on https://github.com/knative/serving/pull/13693 again? If so I can assign it to you and re-open it.

amarflybot commented 4 weeks ago

Yes, I an work on this.

skonto commented 4 weeks ago

This is no longer required as we can achieve hostpath mount via pvc.

True but others have reported certain limitations on this, such as being restricted to a specific node.

skonto commented 4 weeks ago

/assign @amarflybot

skonto commented 4 weeks ago

@amarflybot a few comments about the PR you had: a) schema changes need to be behind a flag, make sure you add that in the schemapatch-config. b) hostpath needs certain validation as we want to catch errors early before we deploy real resources eg. path cannot be empty. K8s has this kind of validation here. c) we need more tests under k8s validation

vthurimella commented 3 weeks ago

@rhuss

@vthurimella looks for kubernetes.podspec-persistent-volume-claim in the source, starting from pkg/apis/config/features.go and follow its usage trail. You should be able to implement hostPath in a similar way.

Thank you for the tips!

@vthurimella, just out of curiosity, what is the reason for choosing Knative as resource allocator ? Which feature of Knative serving is the most important to you ?

The most important feature for my use case is autoscaling. The wide range of tunable parameters provides the flexibility I need to achieve good performance for my application.

@skonto @dprotaso Thank you two again for taking the time to considering this feature request!

skonto commented 3 weeks ago

@amarflybot hi, if you need any pointers here is what I had. Please ping me to review.

skonto commented 2 weeks ago

@amarflybot hi, gentle ping.

amarflybot commented 4 days ago

Give me sometime, was out for vacation. Will solve this asap.

amarflybot commented 4 days ago

@amarflybot hi, if you need any pointers here is what I had. Please ping me to review.

https://github.com/knative/serving/commit/56ef531961a44f79566f1ac1485e8fe6d5f242f0

These are some changes in validation I could find I had done before, if this helps.

skonto commented 1 day ago

Hi @amarflybot that branch is a bit old. Pls take a look of my branch or update yours.