iterative / dvc

🦉 ML Experiments and Data Management with Git
https://dvc.org
Apache License 2.0
13.62k stars 1.18k forks source link

microsoft sharepoint support for dvc remotes #2432

Open alokprasad opened 5 years ago

alokprasad commented 5 years ago

It would be nice if the DVC also supports uploading of the model to sharepoint servers.

shcheklein commented 5 years ago

@alokprasad hi! could you elaborate a little bit and describe your use case in more details please?

alokprasad commented 5 years ago

@shcheklein like the S3, Local , SSH support for Model and Data storage , Some organization also uses Sharepoint for Data storage Some links i found https://www.dmcinfo.com/latest-thinking/blog/id/9388/how-to-upload-a-file-to-sharepoint-on-premises-using-python

shcheklein commented 5 years ago

@alokprasad makes sense, thank you. It looks like it's more or less regular HTTP post/get with NTML auth. I wonder if we should first implement POST for regular HTTP(s) remote we have.

efiop commented 4 years ago

We are currently working on http suppport in https://github.com/iterative/dvc/issues/3247 . Closing this ticket as it seems there is not enough interest from users. If anyone else is interested in this, please leave a comment and we'll reconsider support for this specific remote. Thanks! :slightly_smiling_face:

abravalheri commented 3 years ago

Hello, thank you very much for DVC 📦 I was searching for how to use DVC with SharePoint/OneDrive and stumbled upon this issue. I wonder if there is any documentation on how to use the HTTP support in DVC to achieve that...

scottcode commented 3 years ago

I am interested in using DVC with SharePoint as well. I'm not opinionated about the kind of integration. If an HTTP method with the right kind of auth can work, that would suit my needs.

scottcode commented 3 years ago

I've found these resources about how to perform operations with SharePoint via HTTP:

brodsa commented 3 years ago

Hi, I would also be interested in this feature, using dvc and share point.

efiop commented 3 years ago

Thanks for the interest, guys! Reopening.

Related https://github.com/intake/filesystem_spec/issues/280

austinyhc commented 3 years ago

I am also interested in this feature. 😁

skshetry commented 3 years ago

Shameless plug: I have been working on a WebDAV library (https://github.com/skshetry/webdav4) based on fsspec that hopefully will be used in DVC.

I'd be interested in supporting Sharepoint Webdav after the basic features are implemented. I'd be happy to take contributions too (PTAL at how it's done in rclone: https://github.com/rclone/rclone/blob/cb30a8c80e339ce011bbdacd4c459b90cf5eb2a4/backend/webdav/odrvcookie/fetch.go). Though I have to tell you, I haven't used Sharepoint, so it might take some time and it still needs to go a long way to be included in DVC.

khursani8 commented 3 years ago

Hi, I'm also interested for this feature

marigianna commented 3 years ago

Hello, I'm also interested in this feature. I see there has been a while since the ticket opened. Is there any progress?

skshetry commented 3 years ago

@marigianna, we are moving our remotes to the fsspec.

If you want to have support for the sharepoint through HTTP, you'll need to implement a fsspec-compliant filesystem for it. And, we'll provide a plugin mechanism to integrate (soon!). For sharepoint via Webdav, see here https://github.com/iterative/dvc/issues/2432#issuecomment-823846974.

rmoreas commented 3 years ago

also interested for this feature!

louiscpalma commented 3 years ago

Not explicitly SharePoint, but also supposing OneDrive would be amazing.

sulibo commented 3 years ago

Hi, firstly, thank you very much for the DVC package. I am also interested in using DVC with sharepoint/OneDrive.

mvpalheta commented 2 years ago

Hi, I am also interested in this feature.

papsebestyen commented 2 years ago

I'm also interested in the integration of OneDrive/SharePoint

dberenbaum commented 2 years ago

As mentioned above, this depends on https://github.com/fsspec/filesystem_spec/issues/280. If you are interested in this feature in DVC, please upvote that fsspec issue or look into contributing to it. Thanks for your interest and support!

karelin commented 2 years ago

Thank you for DVC and its functioning on different platform. I'm also interested in DVC on OneDrive/SharePoint

reinososamanta commented 2 years ago

Hi, I'm also interested in this feature! Thanks a lot! :)

narbhar commented 2 years ago

My company uses Onedrive/sharepoint extensively. I'd like to see support for DVC integration with it.

edwardwbarber commented 2 years ago

We are also interested in this. Related, being able to use dvc import-url for SharePoint assets would be hugely helpful for putting a pin in "living" documents (kept on SharePoint) that drive model updates/requirements. I believe that use has the same auth/access issue as a remote hosted in SharePoint. Leaving a comment in fsspec/filesystem_spec/issues/280 as well...

osbm commented 2 years ago

Yes, my university uses microsoft sharepoint/onedrive. It would be awesome for DVC to support this platform.

FarisHijazi commented 1 year ago

also interested in this feature

mbojan commented 1 year ago

I'd be interested too.

aggo commented 1 year ago

Also interested in this :)

gurpreet-ai commented 1 year ago

Hi, I'm also interested in this feature! Thanks a lot! :)

bijonguha commented 1 year ago

Hello, This feature would be greatly valued !

Hish15 commented 1 year ago

To weight in : This would be great :)

chrislemke commented 1 year ago

It would be handy for us too. Thanks!

Jarartur commented 1 year ago

SharePoint is exceptionally useful when it comes to managing access in my work.

danielnortef commented 1 year ago

I would also be interested in this feature. Thanks

hectormz commented 1 year ago

I would also be interested in this ability

mbojan commented 1 year ago

For those determined to use Sharepoint as a remote for DVC I came up with the following temporary solution on Linux.

Note that:

  1. in fact you should be able to use any rclone-supported service as DVC remote.
  2. I use it for a rather small-scale project so perhaps not feasible for larger applications
  3. It does not seem to be incredibly fast

Looking forward to the native support of SharePoint in DVC!

Jarartur commented 1 year ago

To iterate on @mbojan answer: on Windows you can sync Sharepoint sites with OneDrive app for business. So, it works similarly - you sync the folder using OneDrive app and choose in dvc settings a local remote.

Also you don't need to download all the files from the sharepoint site as the file explorer keeps just the references to the files and downloads them automatically when you open them - if you use OneDrive you already know how it works.

It's not ideal as this way can cause some merging conflicts if you work collaboratively (which is the point of dvc). It won't overwrite somebody's changes or merge them - it will create a copy of the files with your machine's name appended which means you will need to resolve those conflicts manually unfortunately.

jemissik commented 1 year ago

My university uses Sharepoint/onedrive, it would be great if this were supported!

Sergio-Quijano-Stratesys commented 3 months ago

This feature would be really nice