Deploy script fails

[rabernat]

I know I am making a mess here. But I can't understand how it is possible that the original deployment script ever actually worked.

With my new modifications (#12), the deploy now fails.

This is the error it gives:

Deploying application
Cloning into 'pangeo-e853181'...
Warning: Permanently added the RSA host key for IP address '192.30.253.113' to the list of known hosts.
Switched to a new branch 'gh-pages'
Branch 'gh-pages' set up to track remote branch 'gh-pages' from 'origin'.
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.3/bin/chartpress", line 11, in <module>
    load_entry_point('chartpress==0.2.0.dev0', 'console_scripts', 'chartpress')()
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/chartpress.py", line 292, in main
    extra_message=args.extra_message,
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/chartpress.py", line 223, in publish_pages
    '--destination', td + '/',
  File "/opt/python/3.6.3/lib/python3.6/subprocess.py", line 286, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/opt/python/3.6.3/lib/python3.6/subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/opt/python/3.6.3/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/opt/python/3.6.3/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'helm': 'helm'
Script failed with status 1

[rabernat]

To summarize how we got here.

• I updated the docker image tag in #9.
• I realized that the chart was not getting deployed and read #2.
• I scanned the travis build history and could not find a single build that actually deployed! They all say Skipping a deployment with the script provider because this is not a tagged commit.
• I concluded that, because of this, we never actually reached the true error in the deploy script, which was the fact that it contained multiple lines (see travis-ci/travis-ci#7641). https://github.com/pangeo-data/helm-chart/blob/0cf0d00eb6d24d4f9da144d136be8436248cc66b/.travis.yml#L13-L16
• I replaced this multi-line deploy script with an external script in #12.
• Now I am stuck on this error.

Help needed! cc @jacobtomlinson, @yuvipanda

[tjcrone]

On a separate but related note, it is still not entirely clear to me what a separate helm chart in another repo gets us. To make pangeo work we still need to include many configuration changes in a config.yaml (and a secrets.yaml), and so a separate helm chart just splits some of the required configurations into another repo, making changes to pangeo more complicated. I'm probably just not understanding it. What am I missing?

[mrocklin]

That helm chart is also used in other deployments, particularly the UK Met's deployment. The stuff in this repository is specific to the deployment in pangeo.pydata.org

That being said, there is clearly more sorting out that should be done to make things work cleanly for everyone.

On Sun, Apr 29, 2018 at 11:04 AM, Tim Crone notifications@github.com wrote:

On a separate but related note, it is still not entirely clear to me what a separate helm chart in another repo gets us. To make pangeo work we still need to include many configuration changes in a config.yaml (and a secrets.yaml), and so a separate helm chart just splits some of the required configurations into another repo, making changes to pangeo more complicated. I'm probably just not understanding it. What am I missing?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/helm-chart/issues/13#issuecomment-385257765, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszIknEBp5ZY8VCQaFkXFZnqqwcjOyks5ttdaTgaJpZM4Tq_Da .

[tjcrone]

There are definitely many ways of doing things! I am currently experimenting with stripping almost everything out of the notebook image and using pip --user installs on the first user login (through minor mods to prepare.sh). I am also working on giving the worker pods the same home directory as the user. This is allowing me to massively reduce the size of the images which has greatly reduced the time required to launch workers and notebook server pods, especially when deploying to a new node. I have increased the singleuser storage capacity slightly to deal with this. Just another way of doing things. I agree that a discussion of the overall structure of Pangeo might be a good thing to have at some point in the future.

[mrocklin]

Sharing the home directory does sound very nice. I think that the UK met deployment does this to good effect. CC @jacobtomlinson

It would also be nice to give up controlling the environment entirely to the user

On Sun, Apr 29, 2018 at 11:31 AM, Tim Crone notifications@github.com wrote:

There are definitely many ways of doing things! I am currently experimenting with stripping almost everything out of the notebook image and using pip --user installs on the first user login (through minor mods to prepare.sh). I am also working on giving the worker pods the same home directory as the user. This is allowing me to massively reduce the size of the images which has greatly reduced the time required to launch workers and notebook server pods, especially when deploying to a new node. I have increased the singleuser storage capacity slightly to deal with this. Just another way of doing things. I agree that a discussion of the overall structure of Pangeo might be a good thing to have at some point in the future.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pangeo-data/helm-chart/issues/13#issuecomment-385259569, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszIHUe1uB4LEEdEdVW8Zkf0UZeWfWks5ttdy-gaJpZM4Tq_Da .

[tjcrone]

I agree this would be great, and I am looking into various options in Azure, which does have options for ReadWriteMany PersistantVolume access modes. It's not clear how easy this would be for GCE, although there was some discussion a while back in pangeo-data/pangeo#88. I feel like I have pulled this thread off topic, but maybe it would be worth picking it back up over there, or even starting a new issue focused on linking worker/notebook pods more generally or giving more control to users.

[jacobtomlinson]

I think the issue is to do with it being run from a PR. Only a branch build can decode the encrypted info for security reasons. Also it looks like there is a bug with checking our the right refs.

To answer @tjcrone the idea behind a separate repo is to allow people (like me) to use it as a basis for my own customized pangeo stack. helm has the concept of dependencies and I wanted to create a custom chart which depends on pangeo. As @mrocklin says the pangeo-data repo is for https://pangeo.pydata.org deployment specific stuff, this repo is for generic stuff.

We are still looking into shared home directories. Our main block is using ReadWriteOnce volumes backed by EBS. I'm planning on switching home directories to ReadWriteMany volumes backed by NFS, which means they can be mounted by the workers too.

I totally agree that having the environment installed in the user home would save a lot of headaches, however moving away from conda is a big step and will be a problem for some users as not everything is available from pip.

[jacobtomlinson]

Also just to add the deploy script has never worked. The version available now was deployed locally.

I'm just working on it now to get it deploying automatically.

[jacobtomlinson]

I have updated the deployment script to successfully deploy and also added an index page based on the jupyterhub one to make it easier to find the latest chart.

https://pangeo-data.github.io/helm-chart/

[rabernat]

Thank you so much for sorting this out! Sorry for making a mess of the repo. On the upside, now I understand better how chartpress works.