search

jupyterhub / hubshare

A directory sharing service for JupyterHub
BSD 3-Clause "New" or "Revised" License
57 stars 21 forks source link

Mostly final spec. #2

Closed ellisonbg closed 7 years ago

yuvipanda commented 7 years ago

Have we also looked at using a pre-existing protocol, such as Web Distributed Authoring and Versioning (WebDAV)? It seems like a decent fit for what we want to do (am studying it more to see if it really is), and is an IETF standard with multiple implementations already present.

yuvipanda commented 7 years ago

Update: Nevermind the WebDAV comment. There doesn't seem to be too much activity around it rn, unfortunately. Probably should still look at it once before we finalize this though :)

yuvipanda commented 7 years ago

I thake that back - I found out that git (and github) use webdav for pushing changes over HTTP(S), which gives me a bit more confidence in them :D

yuvipanda commented 7 years ago

I'm going to read the WebDAV RFCs later in the evening on the train and provide more info here.

yuvipanda commented 7 years ago

https://github.com/owncloud/core/issues/12504 talks about OwnCloud and their decision to use WebDAV or implement their own protocol (and tradeoffs around that)

yuvipanda commented 7 years ago

What is the scope of this API? Is this for a default implementation of sharing that can be easily replaced, or is this the API that all sharing implementations must conform to?

yuvipanda commented 7 years ago

I just read through http://www.ics.uci.edu/~ejw/papers/dav-ecscw.pdf (rather than the RfCs yet) and depending on what exactly this service is trying to do, it looks like we might be re-inventing webdav?

parente commented 7 years ago

Is there a use case or two for this approach to sharing documented somewhere? It's hard to comment on the features of the spec without knowing more about the problem(s) it's trying to solve. For instance, I think the specified sharing API will work well for professors distributing assignments to students, and students sharing their completed assignments back. On the other hand, I'm less certain of how it will work for teams that want to contribute to a shared, evolving project, for example.

Is the first example in scope? The second? Both?

yuvipanda commented 7 years ago

+1 to Peter. Let's ignore most of my WebDAV stuff for now :)

On Mon, Nov 14, 2016 at 5:46 PM, Peter Parente notifications@github.com wrote:

Is there a use case or two for this approach to sharing documented somewhere? It's hard to comment on the features of the spec without knowing more about the problem(s) it's trying to solve. For instance, I think the specified sharing API will work well for professors distributing assignments to students, and students sharing their completed assignments back. On the other hand, I'm less certain of how it will work for teams that want to contribute to a shared, evolving project, for example.

Is the first example in scope? The second? Both?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jupyterhub/hubshare/pull/2#issuecomment-260522487, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB23h-QXwwF7nI7PPmp6wDYVnbVVCfCks5q-Q8MgaJpZM4Kwkik .

Yuvi Panda T http://yuvi.in/blog

willingc commented 7 years ago

@yuvipanda @parente The sharing API came out of discussion on Saturday afternoon with @ellisonbg and @minrk on sharing assignments as Peter mentions.

yuvipanda commented 7 years ago

Ah, I see. So is this purely for sharing assignments? From the name I assumed this was something much more generic.

On Mon, Nov 14, 2016 at 6:39 PM, Carol Willing notifications@github.com wrote:

@yuvipanda https://github.com/yuvipanda @parente https://github.com/parente The sharing API came out of discussion on Saturday afternoon with @ellisonbg https://github.com/ellisonbg and @minrk https://github.com/minrk on sharing assignments as Peter mentions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyterhub/hubshare/pull/2#issuecomment-260530861, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB23jHwaEEyRPMOinkS2DnwaFxhwK1yks5q-RtmgaJpZM4Kwkik .

Yuvi Panda T http://yuvi.in/blog

minrk commented 7 years ago

The motivation is largely for sharing assignments for things like nbgrader, but the target is fairly generic mostly one way sharing. The gist is coarse, simple push/pull of directories. It's sketched out on the roadmap, and largely discussed at the last team meeting.

The primary use cases are:

  1. instructors managing assignments with nbgrader (or similar)
  2. users publishing work for other users to see

Building a long-term team collaboration is not the target, and better handled through existing tools like GitHub, GitLab, etc. Specifically, conflict resolution will always be out of scope here. If there's something to build for that, it's VCS integration for JupyterLab.

Going forward, this will likely serve as the storage/sharing mechanism for deploying shared servers with real-time collaboration (at which point you are no longer pushing/pulling, but running in-place).

@yuvipanda I'm AOK with WebDAV if it seems to be a better fit. I'll do a bit of reading, too. The main thing we would need to add to a generic server is managing access via the Hub. It's possible that what we end up with is WebDAV for file transfer and a small REST API for managing access.

yuvipanda commented 7 years ago

There's also https://remotestorage.io/ which is a simpler and much more modern protocol. WebDAV has some trappings because of the fact it was done in an earlier era.

On Tue, Nov 15, 2016 at 9:40 AM, Min RK notifications@github.com wrote:

The motivation is largely for sharing assignments for things like nbgrader, but the target is fairly generic mostly one way sharing. The gist is coarse, simple push/pull of directories. It's sketched out on the roadmap https://github.com/jupyter/roadmap/blob/master/jupyterhub.md, and largely discussed at the last team meeting https://jupyter.hackpad.com/Spring-2016-Dev-Meeting-h0y1TIAWxz1#:h=JupyterHub .

The primary use cases are:

  1. instructors managing assignments with nbgrader (or similar)
  2. users publishing work for other users to see

Building a long-term team collaboration is not the target, and better handled through existing tools like GitHub, GitLab, etc. Specifically, conflict resolution will always be out of scope here. If there's something to build for that, it's VCS integration for JupyterLab.

Going forward, this will likely serve as the storage/sharing mechanism for deploying shared servers with real-time collaboration (at which point you are no longer pushing/pulling, but running in-place).

@yuvipanda https://github.com/yuvipanda I'm AOK with WebDAV if it seems to be a better fit. I'll do a bit of reading, too. The main thing we would need to add to a generic server is managing access via the Hub. It's possible that what we end up with is WebDAV for file transfer and a small REST API for managing access.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyterhub/hubshare/pull/2#issuecomment-260711792, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB23ixuHXtz7gMI0i0thJhGtx3fB0bNks5q-e6ngaJpZM4Kwkik .

Yuvi Panda T http://yuvi.in/blog

minrk commented 7 years ago

Awesome, thanks @yuvipanda!

minrk commented 7 years ago

Merging this as a complete spec of what we want to do. We can still explore WebDAV, remotestorage for implementations of actual file storage.

yuvipanda commented 7 years ago

:+1: as long as the JupyterHub side of this is pluggable (as in, other implementations can thrive without having to recreate the exact same API!) I'm happy :)