Cells not runing, changes are not saved in datalab notebook

[Izel]

I can't save, run, create a folder or interact with Google Cloud Datalab When I create a folder I get the error: "Creating Folder Failed: An error occurred while creating a new folder", saving doesn't work and cells don't run (even for the example books).

The components were updated and the datalab vm instance was created ad below

gcloud components update
datalab create my-book

I'm facing exactly the same situation described here This issue was solved in the past just rolling back the last update, but it was 2 years ago.

[Izel]

Similar issue #2168 but it does not mention the problem with the cells. Keep it open while the issue owner confirms he has problems running the cells as well.

[rctay]

Issue 1: API requests

I ran into this as well, my setup is to run datalab from Cloud Shell. I was puzzled as to why some network requests to datalab worked, but some did not (like POST?), so you could view a notebook, but not create a new one, for example. I tried so many things, from ssh keys, to checking for out of storage, ....what was strange was the network call would fail, but if I hit the failing URL without any headers, it worked! I plonked the network request as a curl command and deleted the headers half-by-half (bisection). As the saying goes, "Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth." The bad header turned out to be ... drumroll... the Origin header.

It turns out because Cloud Shell preview is no longer dot-devshell.appspot.com but is now on cloudshell.dev, Jupyter (via Tornado) is not accepting the requests, which you can see in your datalab container at /datalab/config/settings.json, which comes from https://github.com/googledatalab/datalab/blob/master/sources/web/datalab/config/settings.json#L22

{
  ...
  "jupyterArgs": [
    ...
    "--NotebookApp.allow_origin_pat=\".*-dot-devshell.appspot.com$\"",
    ...
  ]
  ...
}

To fix this, I changed the pattern to .*[.].cloudshell[.]dev$, like this:

{
  ...
  "jupyterArgs": [
    ...
    "--NotebookApp.allow_origin_pat=\".*[.].cloudshell[.]dev$\"",
    ...
  ]
  ...
}

Issue 2: Websocket connections

The second issue is websocket connections - they were getting dropped. What this means is cell results won't get through and will get stuck at 'Running...' forever. Thankfully we get a hint from the log message that in GCP Logging:

"Rejected websocket request with headers: {
  host: 'localhost:8080',
  'user-agent': 'xxx',
  'accept-encoding': 'gzip, deflate, br',
  'accept-language': 'en-US,en;q=0.9',
  'cache-control': 'no-cache',
  connection: 'Upgrade',
  cookie: 'xxx',
  origin: 'https://8080-cs-999-default.asia-southeast1.cloudshell.dev',
  pragma: 'no-cache',
  'sec-websocket-key': 'xxx,
  'sec-websocket-version': '13',
  upgrade: 'websocket'
}"

The offending code is as below. This one is kind of surprising, datalab (or most processes in containers) listen on localhost, and shouldn't have been affected by whether datalab preview starts on cloudshell.dev or a different domain. Perhaps: 1) Chrome began sending the Origin header in 83 ? The fix should be better, like, say, comparing with the host in a X-Forwarded header, but at this point I am out of time, so I added this to /datalab/web/server.js (which is luckily unminified from the TypeScript https://github.com/googledatalab/datalab/blob/master/sources/web/datalab/server.ts#L315).

 function requestIsCrossOrigin(request: http.ServerRequest) {
   if (request.headers.origin) {
     if (url.parse(request.headers.origin).host.endsWith('.cloudshell.dev')) return false;   // <-- added
     return url.parse(request.headers.origin).host !== request.headers.host;
   }
   ...
 }

Summary

You can make the above changes and get it picked up by killing the Jupyter process and its parent node app.js. But this is not permanent and will get lost the next time datalab starts, either the upstream datalab image needs to get fixed or you can push your own image.

[epellin]

I have the same problem as @Izel . I've tried with different browsers, computers and OS just in case, but still not running any notebook. :weary: Time zone: europe-west1-b

[LeonardoMendoncaInfoglobo]

I'm in the same saga. I tried several things and I still haven't succeeded.

[JulienVincendeau]

I've got the same problem as @dragonsan17 #2168 . I tried to create a new datalab on different zone : datalab create --project xxx --zone us-central1-a datalab-xxx

wihtout backups : datalab create --zone europe-west1-b --no-backups datalab-xxx

nothing better

[GabiAndradeD]

I am also getting the same issue

[dushmanthajoba]

I am facing the same issue. Tried many options, but still no good.

1. Can not create folder or notebook

2. Run cells doesn't show anything

[JulienVincendeau]

I bypassed the problem by changing product: I'm using notebooks on GCP the product seems more up to date

[rctay]

This issue can be closed, I was able to do the things that previously reported broken here, like creating/duplicating a notebook, evaluating cells, etc., in the latest datalab image and after the latest datalab commits which fixed the 2 issues I mentioned in my comment https://github.com/googledatalab/datalab/issues/2169#issuecomment-663944128 arising from the domain change for Google Cloud Shell to cloudshell.dev.