Open Izel opened 4 years ago
Similar issue #2168 but it does not mention the problem with the cells. Keep it open while the issue owner confirms he has problems running the cells as well.
I ran into this as well, my setup is to run datalab from Cloud Shell. I was puzzled as to why some network requests to datalab worked, but some did not (like POST?), so you could view a notebook, but not create a new one, for example. I tried so many things, from ssh keys, to checking for out of storage, ....what was strange was the network call would fail, but if I hit the failing URL without any headers, it worked! I plonked the network request as a curl command and deleted the headers half-by-half (bisection). As the saying goes, "Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth." The bad header turned out to be ... drumroll... the Origin header.
It turns out because Cloud Shell preview is no longer dot-devshell.appspot.com
but is now on cloudshell.dev
, Jupyter (via Tornado) is not accepting the requests, which you can see in your datalab container at /datalab/config/settings.json
, which comes from https://github.com/googledatalab/datalab/blob/master/sources/web/datalab/config/settings.json#L22
{
...
"jupyterArgs": [
...
"--NotebookApp.allow_origin_pat=\".*-dot-devshell.appspot.com$\"",
...
]
...
}
To fix this, I changed the pattern to .*[.].cloudshell[.]dev$
, like this:
{
...
"jupyterArgs": [
...
"--NotebookApp.allow_origin_pat=\".*[.].cloudshell[.]dev$\"",
...
]
...
}
The second issue is websocket connections - they were getting dropped. What this means is cell results won't get through and will get stuck at 'Running...' forever. Thankfully we get a hint from the log message that in GCP Logging:
"Rejected websocket request with headers: {
host: 'localhost:8080',
'user-agent': 'xxx',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9',
'cache-control': 'no-cache',
connection: 'Upgrade',
cookie: 'xxx',
origin: 'https://8080-cs-999-default.asia-southeast1.cloudshell.dev',
pragma: 'no-cache',
'sec-websocket-key': 'xxx,
'sec-websocket-version': '13',
upgrade: 'websocket'
}"
The offending code is as below. This one is kind of surprising, datalab (or most processes in containers) listen on localhost, and shouldn't have been affected by whether datalab preview starts on cloudshell.dev
or a different domain. Perhaps: 1) Chrome began sending the Origin header in 83 ? The fix should be better, like, say, comparing with the host in a X-Forwarded header, but at this point I am out of time, so I added this to /datalab/web/server.js (which is luckily unminified from the TypeScript https://github.com/googledatalab/datalab/blob/master/sources/web/datalab/server.ts#L315).
function requestIsCrossOrigin(request: http.ServerRequest) {
if (request.headers.origin) {
if (url.parse(request.headers.origin).host.endsWith('.cloudshell.dev')) return false; // <-- added
return url.parse(request.headers.origin).host !== request.headers.host;
}
...
}
You can make the above changes and get it picked up by killing the Jupyter process and its parent node app.js
. But this is not permanent and will get lost the next time datalab starts, either the upstream datalab image needs to get fixed or you can push your own image.
I have the same problem as @Izel . I've tried with different browsers, computers and OS just in case, but still not running any notebook. :weary: Time zone: europe-west1-b
I'm in the same saga. I tried several things and I still haven't succeeded.
I've got the same problem as @dragonsan17 #2168 .
I tried to create a new datalab on different zone : datalab create --project xxx --zone us-central1-a datalab-xxx
wihtout backups : datalab create --zone europe-west1-b --no-backups datalab-xxx
nothing better
I am also getting the same issue
I am facing the same issue. Tried many options, but still no good.
Can not create folder or notebook
Run cells doesn't show anything
I bypassed the problem by changing product: I'm using notebooks on GCP the product seems more up to date
This issue can be closed, I was able to do the things that previously reported broken here, like creating/duplicating a notebook, evaluating cells, etc., in the latest datalab image and after the latest datalab commits which fixed the 2 issues I mentioned in my comment https://github.com/googledatalab/datalab/issues/2169#issuecomment-663944128 arising from the domain change for Google Cloud Shell to cloudshell.dev
.
I can't save, run, create a folder or interact with Google Cloud Datalab When I create a folder I get the error: "Creating Folder Failed: An error occurred while creating a new folder", saving doesn't work and cells don't run (even for the example books).
The components were updated and the datalab vm instance was created ad below
I'm facing exactly the same situation described here This issue was solved in the past just rolling back the last update, but it was 2 years ago.