Hello again, Astronomy

[mrocklin]

@dsludwig has set up a separate deployment for pangeo-astro folks

I've given @SimonKrughoff @Cadair and @DanRyanIrish admin rights to this repository so that they can:

• Control the software environment by modifying deployments/astro.pangeo.io/image/binder/environment.yml, like in https://github.com/pangeo-data/astro.pangeo.io-deploy/pull/2 (one of you should merge that by the way)
• Add example notebooks or scripts to deployments/astro.pangeo.io/image/
• Give other collaborators rights to the repository, like in https://github.com/pangeo-data/astro.pangeo.io-deploy/pull/3

The result is available at https://astro.pangeo.io

(also cc @rabernat @jhamman)

[jhamman]

@mjuric and @connolly - you may want to take a look at this issue and/or say hello here.

[mjuric]

Hey @jhamman et al. (& hi @SimonKrughoff !!). Would there be a way to get access to https://astro.pangeo.io, to try it out?

Looking forward to checking out Pangeo, and see how we could integrate it with our Spark work.

CC: @ctslater, @zecevicp

[timothydmorton]

@mrocklin @martindurant nice to meet you both in person today; is this the issue you were referring to to get admin rights? Thanks!

[mrocklin]

Hi @timothydmorton , you now have full admin rights over this repository. Welcome!

You may want to look at the PRs currently open. Feel free to merge them (or make newer better versions). Please go wild.

[SimonKrughoff]

@timothydmorton I second this. I haven't merged simply because I've had other commitments.

[SimonKrughoff]

@mjuric is there an org that you'd like me to add to give you access?

[connolly]

Thanks - dirac-institute

[timothydmorton]

Thanks @mrocklin; are either you or @martindurant available sometime today for me to chat to understand some more about how this works? Perhaps during the after-lunch tutorial slot (before the dask one)?

[timothydmorton]

e.g., I am a part of one of the organizations that have been white-listed, but I still get a 403 when trying to connect. How does the redeployment work? Shouldn't it redeploy after the auth settings are changed?

[mrocklin]

You need to make sure that your membership in one of the white-listed organizations is public. Github doesn't make this information public by default. You can change it by going to the people page of that org.

Thanks @mrocklin https://github.com/mrocklin; are either you or @martindurant https://github.com/martindurant available sometime today for me to chat to understand some more about how this works? Perhaps during the after-lunch tutorial slot (before the dask one)?

I'm still finishing up materials for this tutorial :) I volunteer Martin :)

Happy to help out afterwards though.

On Fri, Oct 19, 2018 at 9:21 AM Timothy Morton notifications@github.com wrote:

e.g., I am a part of one of the organizations that have been white-listed, but I still get a 403 when trying to connect. How does the redeployment work? Shouldn't it redeploy after the auth settings are changed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/astro.pangeo.io-deploy/issues/4#issuecomment-431360813, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszDZh0j9lymcoLiLJ1uTLCuB1NuT6ks5umdG9gaJpZM4XaCEe .

[martindurant]

I am around now (front of the venue near the elevators), happy to help however I can.

[timothydmorton]

Thanks guys-- I'm still waiting on a train so will probably be in just before lunch. Will look for you, Martin.

[timothydmorton]

At what point does it check the membership for authentication? Does it build a list of individuals when it deploys or does it check whether I belong to one of the orgs when I try to log in? My membership in lsst-sqre is now public, but I still get a 403 when trying to log in.

[timothydmorton]

@dsludwig can you help me understand how to redeploy this service? Do someone need to do something manually? #3 and #5 added/changed the org whitelist, but I still can't log in, so I'm suspecting things haven't rebooted since those changes.

[SimonKrughoff]

Is there a way to check the white list from within a JupyterLab pod? If so, I can have a look.

[jhamman]

The idea is that just committing to this repo should trigger a redeploy. That said, I'm not convinced circle-CI is running on this repo at the moment. @dsludwig - can you point us to the build following @mrocklin's last commit?

[dsludwig]

The latest deploy is here: https://circleci.com/gh/pangeo-data/astro.pangeo.io-deploy/6

[SimonKrughoff]

That includes my commit to add the org @timothydmorton is in.

And I've confirmed he is a member and is public.

[timothydmorton]

And I confirm that I still get a 403 when trying to access astro.pangeo.io and authenticating with github.

[timothydmorton]

But at that time my membership in the org wasn't public (now it is). So if the deployment itself builds a whitelist of GH users by reading orgs at deployment time (rather than checking at login time whether the user belongs to any of the orgs), that would explain this (and be somewhat inconvenient).

[dsludwig]

It's definitely checking at login time. I think I know what's going on, I'll make a change right now.

[timothydmorton]

Confirming that #7 fixed my login issue. I can access now, thanks!

[mrocklin]

I'm glad to hear it. Thanks for sticking with things.

On Fri, Oct 19, 2018 at 3:23 PM Timothy Morton notifications@github.com wrote:

Confirming that #7 https://github.com/pangeo-data/astro.pangeo.io-deploy/pull/7 fixed my login issue. I can access now, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/astro.pangeo.io-deploy/issues/4#issuecomment-431471893, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszOaOkiFtwsdqlhcQaNw_UWBelWUZks5umia5gaJpZM4XaCEe .

[martindurant]

The Gaia DR2 data which I converted to parquet can now be found at 'gcs://pangeo-data/gaia_dr2.parquet/'

As an intake entry, it would look something like

sources:
  gaia2:
    args:
      urlpath: gcs://pangeo-data/gaia_dr2.parquet/
      storage_options: {'token': 'anon'}
    description: |
      Gaia DR2 source catalog, see https://www.cosmos.esa.int/web/gaia/dr2
      Converted from CSV to parquet, original online object catalog:
      http://cdn.gea.esac.esa.int/Gaia/gdr2/gaia_source/csv/
    driver: parquet

[Cadair]

Thanks @mrocklin and everyone, this looks really awesome! I am hoping in the next few months there will be a static archive of the AIA dataset hosted by the virtual solar observatory, which I will be really interested in playing with on this deployment.

also pinging @wtbarnes on this as he might have the time to write an example using the dataset we created at scipy.

[jhamman]

Hi all (and @pangeo-data/pangeo-astro) - I wanted to make sure you all saw the info about the upcoming Pangeo Community Meeting: https://github.com/pangeo-data/pangeo/issues/604. I think it would be wonderful if we had some astro folks attend.

[martindurant]

This deployment seems to have gone quiet... I'm not sure if people tried it and found it lacking (installs, versions, functionality...), whether it was too hard to integrate with existing astro tools or if simply not many came to try it.

[SimonKrughoff]

I have been meaning to get back to it for a while. My main issue was that I couldn't get one of my dask intensive notebooks running, but didn't have time to debug. I'd be happy to rubber duck it with someone else for an hour some time to see if I can make progress.

In the mean time, what do I need to do get things running again. It seems the certificate has expired.

I would be very interested in attending the meeting in August if I can get away during that week.