search

openvstorage / framework

The Framework is a set of components and tools which brings the user an interface (GUI / API) to setup, extend and manage an Open vStorage platform.
Other
27 stars 23 forks source link

web-ui doesn't show vdisks when a node goes down #2142

Open yongshengma opened 6 years ago

yongshengma commented 6 years ago

I have 3-node cluster. When a node goes down, i can still access web-ui from other nodes. However, it's not easy for the UI to show vdisks and the content or list is often empty. GET operation doesn't return anything and it looks hanging up.

JeffreyDevloo commented 6 years ago

Hi Could you give us an overview of the packages that were installed (dpkg -l | grep openstorage).

Best regards

yongshengma commented 6 years ago 
dpkg -l | grep openvstorage

ii  blktap-openvstorage-utils            2.0.90-2ubuntu5                       amd64        utilities to work with VHD disk images files
ii  libblktapctl0-openvstorage           2.0.90-2ubuntu5                       amd64        Xen API blktapctl shared library (shared library)
ii  libvhd0-openvstorage                 2.0.90-2ubuntu5                       amd64        VHD file format access library
ii  libvhdio-2.0.90-openvstorage         2.0.90-2ubuntu5                       amd64        Xen API blktap shared library (shared library)
ii  openvstorage                         2.9.5-1                               amd64        openvStorage
ii  openvstorage-backend                 1.9.1-1                               amd64        openvStorage Backend plugin
ii  openvstorage-backend-core            1.9.1-1                               amd64        openvStorage Backend plugin core
ii  openvstorage-backend-webapps         1.9.1-1                               amd64        openvStorage Backend plugin Web Applications
ii  openvstorage-core                    2.9.5-1                               amd64        openvStorage core
ii  openvstorage-hc                      1.9.1-1                               amd64        openvStorage Backend plugin HyperConverged
ii  openvstorage-sdm                     1.9.0-1                               amd64        Open vStorage Backend ASD Manager
ii  openvstorage-webapps                 2.9.5-1                               amd64        openvStorage Web Applications
ii  tgt-openvstorage                     99:1.0.63-0ovs1.4-1ubuntu1.1          amd64        Linux SCSI target user-space daemon and tools - OpenvStorage support
JeffreyDevloo commented 6 years ago

Hi Do you see any Arakoon related errors in the webapi logging? journalctl -u ovs-webapp-api.service It could be that the Arakoon cluster is temporarily without a master and he might be electing a new one.

Best regards

yongshengma commented 6 years ago

I have to wait some seconds, maybe 30s, but the vdisk list finally comes out. If I go to other pages and come back, I have to wait some time again before it shows up.

` Aug 14 12:20:06 test-2 gunicorn[2509]: 2018-08-14 06:20:06 16900 +0200 - test-2 - 3108/140622612104688 - hybrids/storagedriver.py - _edge_clients - 285 - ERROR - Error loading edge clients from pool-001PXP0lu82BYCjnaQ1 Aug 14 12:20:06 test-2 gunicorn[2509]: Traceback (most recent call last): Aug 14 12:20:06 test-2 gunicorn[2509]: File "/opt/OpenvStorage/ovs/dal/hybrids/storagedriver.py", line 81, in _edge_clients Aug 14 12:20:06 test-2 gunicorn[2509]: for item in self.vpool.storagedriver_client.list_client_connections(str(self.storagedriver_id), req_timeout_secs=2): Aug 14 12:20:06 test-2 gunicorn[2509]: RuntimeError: failed to send XMLRPC request ListClientConnections Aug 14 12:20:06 test-2 gunicorn[2509]: 2018-08-14 06:20:06 34300 +0200 - test-2 - 3121/140622609734544 - hybrids/storagedriver.py - _edge_clients - 958 - ERROR - Error loading edge clients from pool-001PXP0lu82BYCjnaQ1 Aug 14 12:20:06 test-2 gunicorn[2509]: Traceback (most recent call last): Aug 14 12:20:06 test-2 gunicorn[2509]: File "/opt/OpenvStorage/ovs/dal/hybrids/storagedriver.py", line 81, in _edge_clients Aug 14 12:20:06 test-2 gunicorn[2509]: for item in self.vpool.storagedriver_client.list_client_connections(str(self.storagedriver_id), req_timeout_secs=2): Aug 14 12:20:06 test-2 gunicorn[2509]: RuntimeError: failed to send XMLRPC request ListClientConnections Aug 14 12:20:09 test-2 gunicorn[2509]: 2018-08-14 06:20:09 34900 +0200 - test-2 - 3108/140622612104688 - hybrids/vdisk.py - fetch_statistics - 286 - ERROR - Error loading statistics_volume from 86fd1161-0a7d-4049-85bf-5e9d81f0a8b3: failed to send XMLRPC request volumePerformanceCountersV3 Aug 14 12:20:09 test-2 gunicorn[2509]: 2018-08-14 06:20:09 34900 +0200 - test-2 - 3121/140622609734544 - hybrids/vdisk.py - fetch_statistics - 959 - ERROR - Error loading statistics_volume from 86fd1161-0a7d-4049-85bf-5e9d81f0a8b3: failed to send XMLRPC request volumePerformanceCountersV3 Aug 14 12:20:09 test-2 gunicorn[2509]: 2018-08-14 06:20:09 43100 +0200 - test-2 - 3108/140622612104688 - api/decorators.py - new_function - 287 - WARNING - API call vdisks.list took 5.32s

`

JeffreyDevloo commented 6 years ago

Hi yongshengma

Thank you for the feedback. The GUI wants to display information that it needs from the volumedriver. It looks like certain management requests are taking too long to resolve because that volumedriver is down. We will look into optimizing the GUI so the essential data can be retrieved first.

Best regards

wimpers commented 6 years ago

@JeffreyDevloo didn't https://github.com/openvstorage/framework/issues/1850 fix this ?

JeffreyDevloo commented 6 years ago

No, it does not. The GUI should go back to lazy loading certain fields to display the essential data faster.