Screen Stream Stops Sending Data on Inactive Browser Tabs After 12-15 Minutes

[merih-sakarya]

Description When a browser tab containing the DeviceFarmer application is inactive for about 12-15 minutes, the WebSocket connection responsible for streaming screen data stops sending binary image data, even though the connection remains open and ping requests continue.

Steps to Reproduce

1. Open the DeviceFarmer application in a browser tab.
2. Establish a WebSocket connection that streams screen data from a device.
3. Make the tab inactive (e.g., switch to a different tab) and wait for approximately 12-15 minutes.
4. Return to the DeviceFarmer tab.

Expected Behavior Upon returning to the DeviceFarmer tab, the WebSocket connection should resume sending binary image data, as the connection is still open and actively sending ping requests. The screen stream should continue without any manual intervention.

Actual Behavior After returning to the DeviceFarmer tab, the WebSocket connection remains open and ping requests continue, but binary image data (screen stream) is no longer being received. This suggests a potential issue with how the onScreenInterestLost method handles inactive browser tabs and stops the data stream.

Additional Information

• Component Affected:
  • Screen WebSocket connection in screen-directive.js.
• Behavior Observed:
  • The WebSocket stops sending image data after about 12-15 minutes of tab inactivity.
  • Upon returning to the tab, the socket connection is open, and ping requests continue, but no image data is received.
• Logs and Observations:
  • Extra logging has been added in screen-directive.js, specifically in the checkEnabled, onScreenInterestGained, and onScreenInterestLost functions.
  • The issue appears to be related to the onScreenInterestLost method. It sends an 'off' message through the WebSocket when the screen is disabled due to the tab going inactive, which might instruct the server to stop sending binary data.

Relevant code snippets:

ws.onopen = function openListener() {
  console.log('DEBUG ::: SCREEN ::: ws : Connection opened successfully.');
  checkEnabled();

  // Send a ping every 30 seconds
  pingInterval = setInterval(function() {
    console.log('DEBUG ::: SCREEN ::: ws : state:', ws.readyState, 'at:', new Date().toUTCString());
    if (ws.readyState === WebSocket.OPEN) {
      console.log('DEBUG ::: SCREEN ::: ws : Sending ping at:', new Date().toUTCString());
      // Ping the server to keep the connection alive
      ws.send('ping');
    }
  }, 30000)
}

function checkEnabled() {
  var newEnabled = shouldUpdateScreen()

  if (newEnabled === cachedEnabled) {
    console.log('DEBUG ::: SCREEN ::: checkEnabled: No change in enabled state, updating bounds')
    updateBounds()
  }
  else if (newEnabled) {
    console.log('DEBUG ::: SCREEN ::: checkEnabled: Screen enabled, updating bounds and gaining interest')
    updateBounds()
    onScreenInterestGained()
  }
  else {
    console.log('DEBUG ::: SCREEN ::: checkEnabled: Screen disabled, clearing canvas and losing interest')
    g.clearRect(0, 0, canvas.width, canvas.height)
    onScreenInterestLost()
  }

  cachedEnabled = newEnabled
}

// ... rest of your existing code

function onScreenInterestLost() {
  if (ws.readyState === WebSocket.OPEN) {
    console.log('DEBUG ::: SCREEN ::: onScreenInterestLost : WebSocket is open, sending "off"');
    ws.send('off')
  } else {
    console.log('DEBUG ::: SCREEN ::: onScreenInterestLost : WebSocket is not open, state:', ws.readyState);
  }
}

// ... rest of your existing code

scope.$on('visibilitychange', function() {
  console.log('DEBUG ::: SCREEN ::: Visibility change detected');
  checkEnabled();
});

scope.$watch('$parent.showScreen', function(newVal, oldVal) {
  console.log('DEBUG ::: SCREEN ::: showScreen value changed from', oldVal, 'to', newVal);
  checkEnabled();
});

Relevant log snippets:

// ....
// Approximately 12 minutes ping log
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:15:36 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:15:36 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:16:06 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:16:06 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:16:36 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:16:36 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:17:06 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:17:06 GMT
DEBUG ::: SCREEN ::: Visibility change detected
DEBUG ::: SCREEN ::: checkEnabled: Screen enabled, updating bounds and gaining interest
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:17:36 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:17:36 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:18:06 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:18:06 GMT
DEBUG ::: SCREEN ::: Visibility change detected
DEBUG ::: SCREEN ::: checkEnabled: Screen disabled, clearing canvas and losing interest
DEBUG ::: SCREEN ::: onScreenInterestLost : WebSocket is open, sending "off"
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:18:36 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:18:36 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:19:06 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:19:06 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:19:36 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:19:36 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:20:06 GMT
DEBUG ::: SCREEN ::: ws : Sending ping at: Sun, 04 Feb 2024 23:20:06 GMT
DEBUG ::: SCREEN ::: ws : state: 1 at: Sun, 04 Feb 2024 23:20:36 GMT

[denis99999]

@merih-sakarya , did you try to change the value of the --group-timeout option (15 mn by default) at provider side ?

[merih-sakarya]

Thank you for your response, @denis99999. I tried changing the value of the --group-timeout option on the provider side from the default 15 minutes, and I can confirm that it resolved the issue. Thanks again for your assistance!