search

jupyterhub / hubshare

A directory sharing service for JupyterHub
BSD 3-Clause "New" or "Revised" License
57 stars 21 forks source link

[WIP] Initial implementation of REST API #17

Closed Zsailer closed 3 months ago

Zsailer commented 5 years ago

I began coding up the REST API described in specification.md.

This is definitely a work in progress!

In this PR, I'm exploring the idea of extending jupyter_server's ContentsManager to handle storage.

Highlights:

Things to do: So many things! Once things are further along, I'll create a checklist here.

ellisonbg commented 5 years ago

Nice!

On Mon, Dec 17, 2018 at 3:07 PM Zachary Sailer notifications@github.com wrote:

I began coding up the REST API described in specification.md https://github.com/jupyterhub/hubshare/blob/master/specification.md.

This is definitely a work in progress!

In this PR, I'm exploring the idea of extending jupyter_server's ContentsManager to handle storage.

Highlights:

  • Hubshare is launched as a jupyterhub-managed service.
  • Hubshare is HubAuthenticated, so user authentication is handled by JupyterHub
  • Posting/fetching content is handled by the ContentsManager
  • Permissions are handled by a new PermissionsManager and stored in a (configurable) database.

Things to do: So many things! Once things are further along, I'll create a checklist here.

You can view, comment on, or merge this pull request online at:

https://github.com/jupyterhub/hubshare/pull/17 Commit Summary

  • wip
  • sketch out handlers
  • wip, building orm api for dirs
  • wip, initial permissions manager

File Changes

Patch Links:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jupyterhub/hubshare/pull/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0JLtZFGNLSxcaLbz66gDMtlD5dTbks5u6CPKgaJpZM4ZXS4f .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

yuvipanda commented 5 years ago

Thanks for getting started on this, @Zsailer!

Have you looked through the various options in #14? I'd love for us to base this on an existing tool than build our own from scratch. It's possible that no existing tools match what we want, in which case building our own is reasonable.

I know @betatim has worked on implementing something similar in https://exploratory.openhumans.org/, which I find pretty cool. Source for that is available in https://github.com/OpenHumans/jupyter-gallery.

ellisonbg commented 5 years ago

Zach and I talked more about this yesterday. The key point is that hubshare has a different set of requirements than other document sharing systems I am aware of. The key requirements are treating directories as immutable, and being able to track through a directory's lineage, through a linked list of shas (both from the versioning and sharing perspective). This is needed to eventually replace the part of nbgrader than distributes and collects homework. In that context users are making copies of assignments and then submitting them back to the instructor. I think that papermill also has similar requirements (I know it treats documents as immutable). We spent a significant amount of time doing the design of the REST API to capture the different usage cases. I am completely fine with research on the implementation side of things, I just want to make sure we are not throwing out the requirements that drive the implementation.

From my perspective, this should be handled like we handle the other services in the Jupyter server. We have a REST protocol, and then an extensible "manager" API that can be implemented by different backends. I think it is important to preserve the ability for different groups to extend hubshare with different backend storage approaches (s3, static files, SQL, etc.).

On Tue, Dec 18, 2018 at 9:49 AM Yuvi Panda notifications@github.com wrote:

Thanks for getting started on this, @Zsailer https://github.com/Zsailer!

Have you looked through the various options in #14 https://github.com/jupyterhub/hubshare/issues/14? I'd love for us to base this on an existing tool than build our own from scratch. It's possible that no existing tools match what we want, in which case building our own is reasonable.

I know @betatim https://github.com/betatim has worked on implementing something similar in https://exploratory.openhumans.org/, which I find pretty cool. Source for that is available in https://github.com/OpenHumans/jupyter-gallery.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyterhub/hubshare/pull/17#issuecomment-448309068, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0MYYNhHbdJM-kiYb_10qTlcjJiGKks5u6SqTgaJpZM4ZXS4f .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

choldgraf commented 5 years ago

Just a quick thought - it sounds like you guys have done a lot of thinking on an API and approach for hubshare. It sounds like others have made efforts in this direction as well (e.g. https://github.com/jupyterhub/hubshare/issues/14). Given that the spec defined in https://github.com/jupyterhub/hubshare/blob/master/specification.md is nearly two years old, perhaps it is worth having the community revisit this and come to an agreement on the high-level approach to implementation to make sure that we've got buy-in from the community, before diving into details on a PR...

ellisonbg commented 5 years ago

I think that is a good idea and captures an aspect of what I am trying to communicate - let's focus on the design requirements and how those are encoded in a protocol. What is the JupyterHub teams preference on how to move forward with that broader discussion?

I am fine with @Zsailer playing with different implementations in the meantime - he has been trying a ton of different things in a very exploratory way that spans the different ideas expressed here. That is useful as a test of different ideas.

betatim commented 5 years ago

(Nitpick: I didn't write any code for the OpenHumans explroatory that is all @gedankenstuecke's work)

Zsailer commented 5 years ago

@yuvipanda and @choldgraf, thank you for your feedback! 😃 I did read through #14 and (as @ellisonbg said) have been exploring many implementations currently in the wild. I agree, we don't want to reinvent the wheel...

Perhaps it would be helpful if I opened a discussion on discourse?

Before I do, I'll try to summarize the previous discussion from #14 and describe my thoughts hubshare's purpose/scope?

choldgraf commented 5 years ago

Before I do, I'll try to summarize the previous discussion from #14 and describe my thoughts hubshare's purpose/scope?

I think this is a good idea :-) something tangible to iterate on and discuss would be helpful!

yuvipanda commented 3 months ago

It makes me sad, but I think we should close this PR and archive the repo. Thoughts?

yuvipanda commented 3 months ago

I'm actually going to be slightly bolder, and close this PR. I opened https://github.com/jupyterhub/team-compass/issues/725 to discuss archiving the repo.

Please re-open @Zsailer if you think the close is inappropriate!