Cloud Foundry deployment - kernel shutting down

[crawles]

I have been trying to deploy a Jupyter notebook and am experiencing difficulty with the kernel shutting down. I am able to get the cf scotch example working in my environment.

I am specifying 2gb of both memory and disk in the manifest. Here it is:

---
applications:
- name: sentiment-compute
  memory: 2GB
  disk_quota: 2GB
  instances: 1
  buildpack: https://github.com/cloudfoundry/python-buildpack.git
  command: sh start_service.sh

# start_service.sh
jupyter-kernelgateway --KernelGatewayApp.port=8080\
                      --KernelGatewayApp.ip=0.0.0.0\
                      --KernelGatewayApp.api=notebook-http\
                      --KernelGatewayApp.seed_uri='https://raw.githubusercontent.com/crawles/text-analytics-service-example/master/Untitled.ipynb'

A "hello world" notebook works successfully. E.g., https://gist.github.com/crawles/8a53396aff6bd14d7709540244025a65

However, if I try another example where I read a pickle object (6 mb) - similar to the example - I run into trouble. https://gist.github.com/crawles/f191f90f188ccaa21728767c364e40b3 The kernel immediately shuts down and the POST is unaccessible.

2016-08-23T16:23:58.96-0400 [CELL/0]     OUT Creating container
2016-08-23T16:23:59.31-0400 [CELL/0]     OUT Successfully created container
2016-08-23T16:24:15.90-0400 [CELL/0]     OUT Starting health monitoring of container
2016-08-23T16:24:17.68-0400 [APP/0]      ERR [KernelGatewayApp] Kernel started: cb8cbcc5-e2ab-4013-bba3-cbe7c532cce9
2016-08-23T16:24:23.20-0400 [APP/0]      ERR [KernelGatewayApp] Kernel shutdown: cb8cbcc5-e2ab-4013-bba3-cbe7c532cce9
2016-08-23T16:24:23.20-0400 [APP/0]      ERR [KernelGatewayApp] Registering endpoint_path: /, methods: (['POST'])
2016-08-23T16:24:23.20-0400 [APP/0]      ERR [KernelGatewayApp] The Jupyter Kernel Gateway is running at: http://0.0.0.0:8080
2016-08-23T16:24:23.40-0400 [CELL/0]     OUT Container became healthy

[parente]

Have you setup an environment.yml or requirements.txt with all the necessary dependencies of the notebook? It might be crashing on startup if the imports fail.

[crawles]

Yes, I left that file out, but I have been able to successfully import the dependencies. I tried commenting everything except the dependencies during testing

[parente]

I just noticed the startup shell script you're using. What happens when you inline the commands right in the manifest so that there's no shell parent process? Kernels have been known to crash immediately in docker containers that are not configured with a proper init process. I wonder if something similar is happening here.

[crawles]

Removing shell script didn't fix it. Figured it out. It's related to SSL certificates and opening a file from a url. The problem occurs using urllib.requests in CF instead of the requests library. Using urllib, there is an error: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed. Didn't appear in error log originally, but it popped up this time. Also implemented this code using flask and saw same error message.

1) This example using urllib.requests in Python 3 does not work - see commented code for error message. https://gist.github.com/crawles/dbc1c655b0ded929c181a80879cbcc78

2) This example using requests in Python 3 works. https://gist.github.com/crawles/d6dae819d371a1db0d798e7f7a3ed1f0

Thanks for your help @parente