search

2i2c-org / infrastructure

Infrastructure for configuring and deploying our community JupyterHubs.
https://infrastructure.2i2c.org
BSD 3-Clause "New" or "Revised" License
103 stars 62 forks source link

New Hub: Geospatial workshop in Ghana #473

Closed choldgraf closed 3 years ago

choldgraf commented 3 years ago

Background

@paigem works with @rabernat, and is helping to lead/organize a workshop around geospatial analytics (e.g., the "Pangeo stack") in Ghana. In previous years, they have asked attendees to install things on their local machines, but she would love to have access to cloud infrastructure via 2i2c that supports this workshop.

The team behind this workshop currently does not have funding for infrastructure/services, so this would be a pro-bono case. In my opinion, it is well worth the time investment because it is a great cause, and a way to see how our infrastructure could serve those in non-North America/Europe countries.

@paigem could you help us answer some of the questions in the section below?

Setup Information

Important Information

Deploy To Do

sgibson91 commented 3 years ago
  • It appears that I cannot access datasets stored on Pangeo Cloud. For instance, I tried to load the ECCO dataset as I do in Pangeo Cloud:
import intake
cat = intake.open_catalog("https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/ocean.yaml")
ecco_monthly_ds = cat.ECCOv4r3.to_dask()

But I get the following error: OSError: Forbidden: https://storage.googleapis.com/download/storage/v1/b/pangeo-ecco-eccov4r3/o/eccov4r3%2F.zmetadata?alt=media Caller does not have serviceusage.services.use access to the Google Cloud project. Is there any way we can have access to the Pangeo Cloud datasets found here?

Omg, I think I have solved this too! At least now when I run that snippet in a notebook, I don't get any error. @paigem can you confirm?

Thank you @rabernat and @TomAugspurger for your helpful input! 🙏🏻

sgibson91 commented 3 years ago
  • Am I understanding the documentation correctly that, if I want content from a public GitHub repo to populate in every user's Hub, then I should distribute an nbgitpuller link for the Hub, instead of the base Hub link: https://coessing.pangeo.2i2c.cloud?

Yes, this is correct

paigem commented 3 years ago

Omg, I think I have solved this too! At least now when I run that snippet in a notebook, I don't get any error. @paigem can you confirm?

It works!! 😄 Amazing - thank you @sgibson91!

rabernat commented 3 years ago
  • then I should distribute an nbgitpuller link for the Hub

In case you don't know about it, you can use this great website to generate an nbgitpuller link. I generated this one for example

https://coessing.pangeo.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fpangeo-gallery%2Fphysical-oceanography&urlpath=lab%2Ftree%2Fphysical-oceanography%2F&branch=master

And can even put your link behind a fancy looking badge

Open with Jupyter

[![Open with Jupyter](https://img.shields.io/badge/Open%20with-Jupyter-orange?style=for-the-badge&logo=Jupyter)](https://coessing.pangeo.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fpangeo-gallery%2Fphysical-oceanography&urlpath=lab%2Ftree%2Fphysical-oceanography%2F&branch=master)
paigem commented 3 years ago

Thanks @rabernat! Yes, I have already made a link using nbgitpuller (thanks to 2i2c docs!) to sync files from a GitHub repo, but I like the fancy badge! I assume this badge is something you include in your GitHub repo?

rabernat commented 3 years ago

You can certainly put the badge in a repo README. The version I shared was a markdown version, so it works well in github. But you could put such badge on any website anywhere, such as the workshop website. The html version would look like

<a href="https://coessing.pangeo.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fpangeo-gallery%2Fphysical-oceanography&urlpath=lab%2Ftree%2Fphysical-oceanography%2F&branch=master"><img alt="Open with Jupyter" src="https://img.shields.io/badge/Open%20with-Jupyter-orange?style=for-the-badge&logo=Jupyter" /></a>
paigem commented 3 years ago

Thanks @rabernat - the html version will be very helpful for the website!

damianavila commented 3 years ago

Omg, I think I have solved this too! At least now when I run that snippet in a notebook, I don't get any error. @paigem can you confirm?

@sgibson91 just to confirm the fix was running this snippet or something else?

gcloud projects add-iam-policy-binding pangeo-181919 \
  --member serviceAccount:pangeo@pangeo-181919.iam.gserviceaccount.com \
  --role roles/serviceusage.serviceUsageConsumer
sgibson91 commented 3 years ago

@damianavila I think the code in this comment did it. The key was the k8s annotation so it knows to use it.

(For ref because it took me some time to figure this out: the part of the gcloud command inside [] is [k8s_namespace/helm_namespace])

damianavila commented 3 years ago

Thanks for the info, @sgibson91. It would be nice to consolidate this info in our docs somehow/somewhere.

sgibson91 commented 3 years ago

It would be nice to consolidate this info in our docs somehow/somewhere.

Yeah, I'd also like a review of it to make sure we understand what's going on and that we're not unnecessarily granting elevated privileges

sgibson91 commented 3 years ago

@paigem if you're happy with the state the hub is in now, I'm going to close this ticket.

Should I continue asking questions specific to my Hub in this thread, or should I start a new issue in 2i2c/pilot as mentioned in the documentation?

We are actually trialling a new support framework using FreshDesk and tickets can be submitted by emailing support@2i2c.org. Are you happy to be a guinea pig and send any issues through this system?

paigem commented 3 years ago

Yes, the Hub is working great! Thank you!! And I'm happy to trial the FreshDesk support framework! :)