Integrate Dataset Lifecycle Framework to support S3+NFS Data Sources

[YiannisGkoufas]

Last year or so we have been working on DLF ( https://github.com/IBM/dataset-lifecycle-framework ) which introduces a new CRD Dataset. This works as a pointer to an existing remote S3/NFS Datasource and we also generate a PVC for it which users can mount directly to their pods and access the bucket/nfs data via a mount-point. Would love to explore if it would be useful to integrate it with kui or develop a plugin for it.

[starpit]

Howdy @YiannisGkoufas, thanks for the interesting idea, we really appreciate it.

We recently added a fairly simple/primitive VFS layer to Kui. Would this be a place that you could target for enhancement? With Kui today, one may do the following, assuming Kui is able to identify your S3 credentials:

ls /s3/aws/myBucket/*.txt
ls /s3/minio/myOtherBucket
ls /s3/ibm/myThirdBucket
mkdir /s3/ibm/myThirdBucket
cp /s3/aws/myBucket/*.txt /s3/ibm/myThirdBucket

etc. Kui tries to intuit the access credentials for these providers... Is this something you could make more real?

[starpit]

Here is the VFS interface: https://github.com/IBM/kui/blob/master/plugins/plugin-bash-like/fs/src/vfs/index.ts#L42

and here is the s3 impl: https://github.com/IBM/kui/blob/master/plugins/plugin-s3/src/vfs/index.ts#L38

[YiannisGkoufas]

Hi @starpit ! Just to add a bit more clarification about what our project does. We offer a mountpoint for the bucket but only inside pods running on Kubernetes, while the plugin-s3 it's about giving the same functionality to the KUI user/developer. Would be interesting to explore how the user can interface with s3 buckets that are defined in Kubernetes as Datasets. One idea that comes to mind is the same way that the user can get information about the pod when invoking kubectl get pods and clicking on the pod, the same way they could get data-related information (size,list of files, preview of contents) when invoking kubectl get datasets and clicking on the S3/NFS data sources

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/kui/issues/6374#issuecomment-846557422): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.