examples directory is not downloading correctly

[rabernat]

In #44, @martindurant implemented a new system for downloading example notebooks, using the following code:

https://github.com/pangeo-data/helm-chart/blob/7f2f89e5834376499cf028c4c5f707741e6292a6/docker-images/notebook/prepare.sh#L8-L21

It does not appear to be working. Here is the kubernetes log of a new user logging in to pangeo.pydata.org for the first time

$ kubectl --namespace=pangeo logs jupyter-buaamuxu
+ echo 'Copy files from pre-load directory into home'
Copy files from pre-load directory into home
+ cp --update -r -v /pre-home/. /home/jovyan
'/pre-home/./examples' -> '/home/jovyan/./examples'
'/pre-home/./config.yaml' -> '/home/jovyan/./config.yaml'
'/pre-home/./worker-template.yaml' -> '/home/jovyan/./worker-template.yaml'
+ '[' -z '' ']'
+ export EXAMPLES_GIT_URL=https://github.com/pangeo-data/pangeo-example-notebooks
+ EXAMPLES_GIT_URL=https://github.com/pangeo-data/pangeo-example-notebooks
+ '[' '!' -d examples ']'
+ cd examples
+ git remote set-url origin https://github.com/pangeo-data/pangeo-example-notebooks
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git fetch origin
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git reset --hard origin/master
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git merge --strategy-option=theirs origin/master
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ '[' '!' -f DONT_SAVE_ANYTHING_HERE.md ']'
+ echo 'Files in this directory should be treated as read-only'

As a result, no no users are getting any example notebooks loaded.

This is a pretty serious problem, and I'm surprised it hasn't ben caught until now.

I think the fix might be to set EXAMPLES_GIT_URL to https://github.com/pangeo-data/pangeo-example-notebooks.git

[rabernat]

Or maybe we are missing a git init statement?

[jacobtomlinson]

Leaving open until tested

[rabernat]

I just re-deployed the cluster with the latest helm chart, and I am still getting the same fatal: not a git repository messages in my logs.

$ kubectl --namespace=pangeo logs jupyter-rabernat
+ echo 'Copy files from pre-load directory into home'
+ cp --update -r -v /pre-home/. /home/jovyan
Copy files from pre-load directory into home
+ '[' -z '' ']'
+ export EXAMPLES_GIT_URL=https://github.com/pangeo-data/pangeo-example-notebooks
+ EXAMPLES_GIT_URL=https://github.com/pangeo-data/pangeo-example-notebooks
+ rmdir examples
'/pre-home/./config.yaml' -> '/home/jovyan/./config.yaml'
'/pre-home/./worker-template.yaml' -> '/home/jovyan/./worker-template.yaml'
+ '[' '!' -d examples ']'
+ cd examples
+ git remote set-url origin https://github.com/pangeo-data/pangeo-example-notebooks
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git fetch origin
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git reset --hard origin/master
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ git merge --strategy-option=theirs origin/master
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
+ '[' '!' -f DONT_SAVE_ANYTHING_HERE.md ']'

[rabernat]

The live version is 0.1.1-8c92298

[martindurant]

Is your examples directory empty?

[rabernat]

No, it is full of old stuff.

On Tue, Aug 21, 2018 at 2:30 PM Martin Durant notifications@github.com wrote:

Is your examples directory empty?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/helm-chart/issues/54#issuecomment-414776460, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJFJt-L3crp19OPf7bZjfX0h2VSahruks5uTFHdgaJpZM4V4rop .

[martindurant]

The script is not supposed to overwrite a directory with old stuff in it. It will remove, replace and populate an empty directory, or create and populate a non-existent directory. Not squashing someone's work was one of the considerations when making this, unless I misunderstood.

[martindurant]

I advise you to handle gitting up your directory, since there are sure to be weird merge conflicts when you first fetch anyway.

[mrocklin]

I'm going to again suggest that we just nuke the user's examples directory every time. I think that this will be simpler.

On Tue, Aug 21, 2018 at 4:11 PM, Martin Durant notifications@github.com wrote:

I advise you to handle gitting up your directory, since there are sure to be weird merge conflicts when you first fetch anyway.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/helm-chart/issues/54#issuecomment-414805470, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszLWIB6MRZUyiD9ArOykxqg2LevPZks5uTGlYgaJpZM4V4rop .

[rabernat]

Ok I think I understand the situation.

This approach to examples may go away soon as we evolve our approach to these clusters (and incorporate more binder). So I don't think any action is required right now.

[guillaumeeb]

I'm reviving some old issue here, but I'd like to revive this repo. So sorry for the noise in advance.

I think this one can be closed as it seems to work perfectly on newly depoyed cluster.