GUI does not work with some proxy servers

HelenClifton commented 6 years ago

Expected behavior

@forman, @VPriemer, @papesci Expect GUI to work

Actual behavior

See blank blue screen

blue_gui

Steps to reproduce the problem

Install cate-2.0.0-dev.9
Open GUI

Specifications

openSUSE 42.3

forman commented 6 years ago

@HelenClifton please select Main Menu / View / DevTools Window and make a screenshot / text copy of errors

VPriemer commented 6 years ago

gui_error

Here is the error from my GUI. I'm using openSUSE 42.3 as well. This is written in the console:

./cate-desktop-2.0.0-dev.9-x86_64.AppImage installed: X-AppImage-BuildId=69c35e50-4248-11a8-10f4-a7bee6d9a898 image: X-AppImage-BuildId=69c35e50-4248-11a8-10f4-a7bee6d9a898 [13:01:26.565] [info] process.versions = { http_parser: '2.7.0', node: '8.2.1', v8: '5.9.211.38', uv: '1.13.1', zlib: '1.2.11', ares: '1.10.1-DEV', modules: '57', openssl: '1.0.2l', electron: '1.8.4', chrome: '59.0.3071.115', 'atom-shell': '1.8.4' } [13:01:26.572] [info] User preferences successfully loaded from "/home/vpriemer/.cate/preferences.json" [13:01:26.572] [warn] debugWorldView = false [13:01:26.572] [info] process.env.NODE_ENV = production [13:01:26.573] [info] appPath: /tmp/.mount_cate-d9jge2x/app/resources/app.asar [13:01:26.573] [info] appConfig: { webAPIConfig: { servicePort: 9090, serviceAddress: '', serviceFile: '/home/vpriemer/.cate/webapi-info.json', processOptions: {} } } [13:01:26.755] [info] Ready. [13:01:27.599] [info] Splash says: Waiting for Cate service... [13:01:27.599] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:27.618] [info] No response from Cate service after 0.02 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:27.618] [info] Splash says: Waiting for Cate service (0.02s) [13:01:27.622] [info] setupInfo: { oldCateDir: '/cmsaf/cmsaf-ops8/vpriemer/cate-1.0.0.dev3', newCateDir: '/home/vpriemer/cate-2.0.0.dev9', oldCateVersion: '2.0.0.dev9', newCateVersion: '2.0.0.dev9', setupReason: null } [13:01:27.623] [info] Splash says: Searching unused port... [13:01:27.627] [info] Starting Cate service: source "/cmsaf/cmsaf-ops8/vpriemer/cate-1.0.0.dev3/bin/activate" "/cmsaf/cmsaf-ops8/vpriemer/cate-1.0.0.dev3"; exec cate-webapi-start --caller cate-desktop --port 9090 --file "/home/vpriemer/.cate/webapi-info.json" --verbose [13:01:27.631] [info] Cate service started (pid=2370). [13:01:28.133] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:28.135] [info] No response from Cate service after 0.54 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:28.135] [info] Splash says: Waiting for Cate service (0.54s) [13:01:28.637] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:28.638] [info] No response from Cate service after 1.04 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:28.638] [info] Splash says: Waiting for Cate service (1.04s) [13:01:29.140] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:29.141] [info] No response from Cate service after 1.54 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:29.141] [info] Splash says: Waiting for Cate service (1.54s) [13:01:29.643] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:29.644] [info] No response from Cate service after 2.05 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:29.644] [info] Splash says: Waiting for Cate service (2.05s) [13:01:30.146] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:30.147] [info] No response from Cate service after 2.55 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:30.147] [info] Splash says: Waiting for Cate service (2.55s) [13:01:30.390] [error] cate-webapi: WARNING:root:invalid proxy URL "None"

[13:01:30.648] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:30.649] [info] No response from Cate service after 3.05 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:30.649] [info] Splash says: Waiting for Cate service (3.05s) [13:01:31.151] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:31.155] [info] No response from Cate service after 3.56 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:31.155] [info] Splash says: Waiting for Cate service (3.56s) [13:01:31.658] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:31.659] [info] No response from Cate service after 4.06 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:31.659] [info] Splash says: Waiting for Cate service (4.06s) [13:01:32.160] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:32.162] [info] No response from Cate service after 4.56 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:32.162] [info] Splash says: Waiting for Cate service (4.56s) [13:01:32.663] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:32.665] [info] No response from Cate service after 5.07 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:32.665] [info] Splash says: Waiting for Cate service (5.07s) [13:01:33.166] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:33.167] [info] No response from Cate service after 5.57 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:33.167] [info] Splash says: Waiting for Cate service (5.57s) [13:01:33.668] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:33.670] [info] No response from Cate service after 6.07 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:33.670] [info] Splash says: Waiting for Cate service (6.07s) [13:01:34.171] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:34.173] [info] No response from Cate service after 6.57 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:34.173] [info] Splash says: Waiting for Cate service (6.57s) [13:01:34.674] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:34.675] [info] No response from Cate service after 7.08 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:34.675] [info] Splash says: Waiting for Cate service (7.08s) [13:01:35.176] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:35.178] [info] No response from Cate service after 7.58 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:35.179] [info] Splash says: Waiting for Cate service (7.58s) [13:01:35.679] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:35.680] [info] No response from Cate service after 8.08 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:35.680] [info] Splash says: Waiting for Cate service (8.08s) [13:01:36.181] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:36.183] [info] No response from Cate service after 8.58 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:36.183] [info] Splash says: Waiting for Cate service (8.58s) [13:01:36.684] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:36.686] [info] No response from Cate service after 9.09 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:36.686] [info] Splash says: Waiting for Cate service (9.09s) [13:01:37.187] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:37.189] [info] No response from Cate service after 9.59 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:37.190] [info] Splash says: Waiting for Cate service (9.59s) [13:01:37.690] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:37.691] [info] No response from Cate service after 10.09 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:37.691] [info] Splash says: Waiting for Cate service (10.09s) [13:01:38.192] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:38.194] [info] No response from Cate service after 10.60 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:38.194] [info] Splash says: Waiting for Cate service (10.60s) [13:01:38.695] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:38.697] [info] No response from Cate service after 11.10 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:38.697] [info] Splash says: Waiting for Cate service (11.10s) [13:01:39.199] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:39.200] [info] No response from Cate service after 11.60 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:39.200] [info] Splash says: Waiting for Cate service (11.60s) [13:01:39.702] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:39.703] [info] No response from Cate service after 12.10 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:39.703] [info] Splash says: Waiting for Cate service (12.10s) [13:01:40.205] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:40.206] [info] No response from Cate service after 12.61 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:40.206] [info] Splash says: Waiting for Cate service (12.61s) [13:01:40.707] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:40.708] [info] No response from Cate service after 13.11 seconds: Error: connect ECONNREFUSED 127.0.0.1:9090 [13:01:40.709] [info] Splash says: Waiting for Cate service (13.11s) [13:01:41.210] [info] Waiting for response from Cate service http://127.0.0.1:9090/ [13:01:41.215] [info] Connected to Cate service. Response: {"status": "ok", "content": {"name": "cate-webapi", "version": "2.0.0.dev9", "timestamp": "2018-04-23"}} [13:01:41.218] [info] Splash says: Loading user interface... [13:01:41.230] [error] cate-webapi: INFO:tornado.access:200 GET / (127.0.0.1) 0.79ms [I 180423 13:01:41 web:2106] 200 GET / (127.0.0.1) 0.79ms

[13:01:43.225] [info] Splash says: Done. [13:01:43.268] [info] Installing update-check... [13:01:43.364] [info] Main window ready to show. [13:01:43.384] [info] Checking for update [13:01:44.896] [info] Update for version 2.0.0-dev.9 is not available (latest version: 2.0.0-dev.9, downgrade is disallowed. [13:01:44.897] [info] Update-check result: { versionInfo: { version: '2.0.0-dev.9', files: [ [Object] ], path: 'cate-desktop-2.0.0-dev.9-x86_64.AppImage', sha512: 'dTIezZK2lZfFBveKEm02AlS/lE93nEexruPNqdSi48SvfRg6w5GDxkAuEuh4uslJ1PHzfOFOHWxsDGHdFqhhow==', releaseDate: '2018-04-17T14:06:08.934Z', releaseName: '2.0.0-dev.9', releaseNotes: '

New Features and UX Improvements

\n

Representative default variables #590
Drag and drop files on Cate\'s main window #597
Some (text) areas in the UI should be user-selectable #608
Allow proxy configuration in Cate Desktop #544
Improve error messages and handling #393
Provided help text with links for the Python code editor opened from the compute operation dialog.

\n

Fixes

\n

Cate Desktop reuses service started by another user #577
User Informed of "Raised Exception"s Rather Than Gracefully Informing User #593
Error - Removal of Resource Causes Failure Instead of Gracefully Informing User #589
Re-starting Cate Hangs #582
Closing Cate on Mac #551

' }, updateInfo: { version: '2.0.0-dev.9', files: [ [Object] ], path: 'cate-desktop-2.0.0-dev.9-x86_64.AppImage', sha512: 'dTIezZK2lZfFBveKEm02AlS/lE93nEexruPNqdSi48SvfRg6w5GDxkAuEuh4uslJ1PHzfOFOHWxsDGHdFqhhow==', releaseDate: '2018-04-17T14:06:08.934Z', releaseName: '2.0.0-dev.9', releaseNotes: '

New Features and UX Improvements

\n

Representative default variables #590
Drag and drop files on Cate\'s main window #597
Some (text) areas in the UI should be user-selectable #608
Allow proxy configuration in Cate Desktop #544
Improve error messages and handling #393
Provided help text with links for the Python code editor opened from the compute operation dialog.

\n

Fixes

\n

Cate Desktop reuses service started by another user #577
User Informed of "Raised Exception"s Rather Than Gracefully Informing User #593
Error - Removal of Resource Causes Failure Instead of Gracefully Informing User #589
Re-starting Cate Hangs #582
Closing Cate on Mac #551

' } }

HelenClifton commented 6 years ago

@forman, @papesci : I'm not using Linux myself, but please see @VPriemer comment above

forman commented 6 years ago

@VPriemer it seems you have a problem with your network proxy. Did you configure a proxy in $HOME/.cate/conf.py?

papesci commented 6 years ago

@forman @VPriemer i can open the GUI setting the proxy but i still have to start manually the webservicewith the command pyton cate/webapi/start.py --port 9090 --caller cate-desktop

forman commented 6 years ago

@VPriemer is it a native Linux installation or a VM, e.g. on VirtualBox?

VPriemer commented 6 years ago

@forman it is a native one

forman commented 6 years ago

@VPriemer please send me your $HOME/.cate/conf.py.

VPriemer commented 6 years ago

@forman I've sent it via email.

forman commented 6 years ago

@VPriemer and @papesci

Ok, it seems the reason for this is your organisation's proxy server. According to this article a proxy server may decide to close WebSockets connections, which are used by Cate for the inter-process communication of the Cate WebAPI service (Python) and the GUI (JavaScript/Electron):

HTTP proxy servers — which were originally designed for document transfer — may choose to close streaming or idle WebSocket connections, because they appear to be trying to connect with an unresponsive HTTP server.

Here is described a similar issue in another project deployed on CloudFoundry.

I don't really know what to do about this in our case, especially because we are connecting to localhost which should no involve a proxy server at all. I'll try using a different port range. We currently start with port 9090, if it is in use we try the next port.

forman commented 6 years ago

@VPriemer and @papesci

Can you confirm that the same version of Cate runs fine in your intranet on Windows 10 or Mac?

VPriemer commented 6 years ago

@forman Is the same port range used for the GUI and the CLI? ...because my CLI works well.

forman commented 6 years ago

@VPriemer and @papesci

Could you please ask your network admin whether there is any (new) proxy server / firewall configuration in your intranet regarding use of WebSockets with localhost (127.0.0.1).

VPriemer commented 6 years ago

@forman I have forwarded it. Unfortunately, the main admin is not available until Friday.

VPriemer commented 6 years ago

@forman A colleague has found out that my computer is trying to reach the local address 127.0.0.1 via the DWD proxy server. He has shown me the following workaround: I should start the GUI without proxy in a new shell.

export http_proxy= export https_proxy= export no_proxy=

Then I start the GUI and write in file -> preferences -> proxy configuration: 'http://ofsquid.dwd.de:8080'

The GUI looks a bit strange and in the console you can see a 'CeasiumGlobe' error and that the connection timed out.
gui

I can load local data sources but I cannot load data sources from the ODP. Following error mesage: dataload_error

kbernat commented 6 years ago

@forman is it GUI related issue or Core?

If it's Core issue - as @VPriemer mentioned, on linux there is 'no_proxy' environment variable which allows to exclude praticular hosts from using proxy

https://curl.haxx.se/docs/manpage.html#NOPROXY

no_proxy="127.0.0.1, localhost"

For GUI, Electron also allows custom proxy configuration from application level

https://github.com/electron/electron/blob/master/docs/api/chrome-command-line-switches.md#--proxy-bypass-listhosts

forman commented 6 years ago

@kbernat Thanks, sounds promising!

@VPriemer Could you please try setting an environment variable in a shell:

 $ export no_proxy="127.0.0.1, localhost"

Then start Cate Desktop from the same shell.

VPriemer commented 6 years ago

@forman The originally error ("Establishing a tunnel via proxy server failed" (see my comment from 27th April) occurs again.

forman commented 6 years ago

@VPriemer that should be ok, as there was no update since you've observed the problem.

VPriemer commented 6 years ago

@forman ok. So I've to wait for the next update? When is it planned?

forman commented 6 years ago

Hopefully today. Note that the background map you see is Cate's static world image. It will be shown when Cate cannot reach http://dev.virtualearth.net which provides Cate's default Bing Maps imagery. This may be another problem caused by your proxy server.

forman commented 6 years ago

I just released 2.0.0.dev11

papesci commented 6 years ago

Hi, i've got the same exception of @VPriemer originally solved starting the web service manually python cate/webapi/start.py --port 9090 --caller cate-desktop before to run the GUI. The problem is not present any more in the new release dev11. I think the problem i have found is different from the one reported by @VPriemer maybe the network configuration is playing a role in this case. We could use this link web socket test to check if is possible to get through a proxy opening a socket. The test try to connect to an echo server.

forman commented 6 years ago

@papesci Vivien's browser passed the web socket test. When she runs the cate-webapi-start -p 9090 first, Cate Desktop works fine, which is strange enough.

forman commented 6 years ago

Issue could be partly solved by letting Cate Desktop recognizing environment variables HTTP_PROXY, HTTPS_PROXY, SOCKS_PROXY, NO_PROXY, and their lower-case counterparts to

configure Electron's proxy command-line switches
configure Electron's BrowserWindow.webContents.session proxy settings

This solves the primary problem namely allowing making WebSockets connections to localhost, but does not yet allow fetching Bing map tiles from dev.virtualearth.net.

CCI-Tools / cate