search

openhab / openhab-webui

Web UIs of openHAB
Eclipse Public License 2.0
224 stars 242 forks source link

MJPEG and Basic UI Video cuases network failure #1693

Open woolf100 opened 1 year ago

woolf100 commented 1 year ago

Which UI are you reporting an issue for?

The problem

Since I upgraded To OH 3.4.1 (coming from 2.5.12) Streaming MJPEG data causes an interesting Problem. Inserting even multiple Videos in the sitemap using code like this was never a problem with OH 2.5 Video label="vidstream" icon="video" url="XXX" encoding="mjpeg" visibility=[Object==ON] they would dynamically appear and disappear based on the item state of "Object" With OH3.4.1 this still works in general but after a while will crash my network stack on all installations I tested. Using "tcptrack -i wlan0" shows that more and more connections are opened using always new ports with only one of them causing network traffic. However the old connections seem to remain open and occupy ports. After a while it appears as if all ports available are occupied and wlan0 becomes unreachable. ## Your suggestion

Let the browser stream the video and don't route it through openhab, or alternatively ensure that unused ports are closed or maybe cleaned up afer a while.

Your environment

Reproducibility and Enironments tested: Operating Systems: Debian Buster and Bullseye

Devices The Issue can be reproduced on the following Systems: Raspberry 4 8GB Openhabian Image Raspberry 4 8GB Manual installation Raspberry Pi3B+ Openhabian Image Raspberry Pi3B+ Manual installation Rock PI 4 2gb manual OH installation (not tried anything else)

The client info below is misleading as it was copied from a PC environment and does not represent the always on display strategy using an I-Pad with Safari to display the basicUI sitemap

one of the setups:

runtimeInfo:
  version: 3.4.1
  buildString: Release Build
locale: de-DE
systemInfo:
  configFolder: /etc/openhab
  userdataFolder: /var/lib/openhab
  logFolder: /var/log/openhab
  javaVersion: 11.0.16
  javaVendor: Debian
  osName: Linux
  osVersion: 5.15.84-v8+
  osArchitecture: aarch64
  availableProcessors: 4
  freeMemory: 542971912
  totalMemory: 1677721600
  startLevel: 100
bindings:
  - astro
  - dwdpollenflug
  - dwdunwetter
  - ekey
  - exec
  - fineoffsetweatherstation
  - hpprinter
  - http
  - hue
  - ipcamera
  - keba
  - knx
  - miio
  - mikrotik
  - mqtt
  - network
  - openweathermap
  - rfxcom
  - shelly
  - sonos
  - systeminfo
  - tcpudp
  - telegram
  - tr064
  - zwave
clientInfo:
  device:
    ios: false
    android: false
    androidChrome: false
    desktop: true
    iphone: false
    ipod: false
    ipad: false
    edge: false
    ie: false
    firefox: true
    macos: false
    windows: true
    cordova: false
    phonegap: false
    electron: false
    nwjs: false
    webView: false
    webview: false
    standalone: false
    os: windows
    pixelRatio: 1.25
    prefersColorScheme: light
  isSecureContext: false
  locationbarVisible: true
  menubarVisible: true
  navigator:
    cookieEnabled: true
    deviceMemory: N/A
    hardwareConcurrency: 16
    language: de
    languages:
      - de
      - en-US
      - en
    onLine: true
    platform: Win32 (Misleading the alwasy on is an IPAD with Safari)
  screen:
    width: XX
    height: XX
    colorDepth: XX
  support:
    touch: false
    pointerEvents: true
    observer: true
    passiveListener: true
    gestures: false
    intersectionObserver: true
  themeOptions:
    dark: dark
    filled: true
    pageTransitionAnimation: default
    bars: light
    homeNavbar: default
    homeBackground: default
    expandableCardAnimation: default
  userAgent: 
timestamp: 2023-02-08T20:40:12.744Z

Additional information

lolodomo commented 1 year ago

more and more connections are opened using always new ports with only one of them causing network traffic.

What connections are you talking about?

he always on display strategy using an I-Pad with Safari to display the basicUI sitemap

Is it something specific to Safari? Do you show the same issue with Chrome or Firefox on your PC for example?

woolf100 commented 1 year ago

What connections are you talking about?:

Client Server
Openhabserver:PortA Camera:PortX Openhabserver:PortB Camera:PortX Openhabserver:PortC Camera:PortX .... Ipad:PortR Openhabserver:8080 Ipad:PortV Openhabserver:8080 Ipad:PortW Openhabserver:8080 ....

the raspberry sends about 90Gigabytes a day with a single 640*480 mjpeg stream camera this way, I took a closer look and found multiple parallel streams of about the same datarate to be active today so they seem to be not all unused but some even transmit in parallel.

I have not tried using only a PC as a client till now

lolodomo commented 1 year ago

Please show the code of your sitemap page containing all your video elements. Maybe something due to the usage of visibility attribute?

lolodomo commented 1 year ago

Could you check with only one video element without visibility attribute?

I would not be surprised that a video stream is started even for your hidden widgets.

How many video elements do you have on your page?

lolodomo commented 1 year ago

If you properly set the encoding attribute, normally the visibility is correctly considered when you open the page. https://github.com/openhab/openhab-webui/blob/main/bundles/org.openhab.ui.basic/src/main/java/org/openhab/ui/basic/internal/render/VideoRenderer.java#L76

And if the visibility changes while the page is already opened, this should be property handled in JavaScript code too. But maybe this code is not working in Safari?

So several questions for you:

  1. Is it a bug only present with your Safari browser?
  2. Is it a bug only when using visibility attribute?
  3. Is your sitemap designed in MainUI?
woolf100 commented 1 year ago

I try to answer all the above questions in one post:

I am not an expert but If I read the code you linked correctly each time the visibility is set to "On" the url is pulled and a new connection to render the video is started. If visibility switches to "off" the url is removed but I do not see that the video connection which is currently playing is closed? Am I missing something?

The issue happens with one, two three and four streams so for analysis assuming a single camera with the code in the first post is sufficient

Streams were tested from different sources tested so far:

Sitemap is as it was desigend in notepad++ sometime starting in openhab 2.4.x

I will test using Safari with only one stream not hidden overnigth and come back with the result, depending on the outcome next step would then be firefox with the same stream but hiding and unhiding or is there anything else that would be of more help to find the issue?

Looking at the code why is the MJPEG stream proxied but the HLS is not? is there a possibility to override the MJPEG proxying to test if that solves the issue?

woolf100 commented 1 year ago

Update:

Runing without visibility works somehow OK using Safari

For testing purposes I installed The Openhab app for IOs and had it run there with three streams tagged by a visibility flag activated and deactivated fom time to time. After 57 hours I ended up having about 20 parallel streams into the Openhab system and outwards to theTablet having sent ~700Gigabytes and received a total of ~700Gigabyte within 57 hours. I cannot prove hoewver which rendering engin is used by the Openhab App. Would it be of interest to also test this with firefox or any other distinct browser?

lolodomo commented 1 year ago

Runing without visibility works somehow OK using Safari

Ok so it is a problem with hidden widget streaming the video.

For testing purposes I installed The Openhab app for IOs and had it run there with three streams tagged by a visibility flag activated and deactivated fom time to time. After 57 hours I ended up having about 20 parallel streams into the Openhab system

Would mean that video streams are not stopped when hidden.

Would it be of interest to also test this with firefox or any other distinct browser?

Yes please, if you can. I would be a little surprised that noone discovered that if it affects all WEB browsers. In the meanwhile, I will check in the (JavaScript) code if I see something strange.

woolf100 commented 1 year ago

Firefox on an IOS Tablet results in the same issue as Safari and the Openhab app. I will try to setup a test on an android tablet in the next days.

lolodomo commented 1 year ago

Can you please propose an example of sitemap that shows the problem and that I can reuse for my own tests? How do you monitor the number of opened streams?

woolf100 commented 1 year ago

To start I would suggest something like this:

sitemap test label="test" { Frame label="A" { Switch item=WasserzirkulationOsramOnOff_Toggle label="Strom Türklingel" icon="wallswitch" visibility=[WasserzirkulationOsramOnOff_Toggle==OFF] valuecolor=[OFF="red"] Video url="http://zoneminderIP/zm/cgi-bin/nph-zms?mode=jpeg&monitor=2&scale=100" encoding="mjpeg" visibility=[Kamera==ON] Video url="http://zoneminderIP/zm/cgi-bin/nph-zms?mode=jpeg&monitor=1&scale=100" encoding="mjpeg" visibility=[Kamera==ON] Text item=DoorStatus valuecolor=[OPEN="red"] Switch item=DoorbellMute valuecolor=[ON="red"] visibility=[DoorbellMute==ON] Text item=WindowStatus valuecolor=[OPEN="red", OPEN_DG="red", OPEN_EG="red", OPEN_EG_DG="red", OPEN_DG_UG="red", OPEN_EG_UG="red", OPEN_EG_DG_UG="red"] {Group item=gWindows} Text item=GarageDoor valuecolor=[OPEN="red"] visibility=[GarageDoor==OPEN] Switch item=Garage_control label="Position [%.0f %%]" icon="garagedoor" valuecolor=[0="red"] Switch item=WaterAlert visibility=[WaterAlert==ON] Switch item=FireAlert visibility=[FireAlert==ON] mappings=[OFF="OK", ON="FIRE"] } }

Open streams are monitored with calling sudo tcptrack -i wlan0 and simply counting streams with data flow from and to the IPaddesses

additionally I use teh data from the network binding as I can typically see a dramatic increas in download/upload total amount once the issue has started. As I wrote above worst case were about 700 Gigabytes in under 60 hours.

lolodomo commented 1 year ago

This may be a browser bug. https://bugs.chromium.org/p/chromium/issues/detail?id=73395 When the widget is hidden, we replace the img src by the source of the "none" icon. I was expecting that the previous source to be stopped and destroyed. Apparently for Chrome, setting the source to "" would stop the video. But the message is old.

woolf100 commented 1 year ago

interesting, I currently ran a separate test streaming directly to a second window not through openhab proxy four streams in parallel without hiding to see if that would also cause strange behaviour which it did not for almost a week now. I found something interesting in my network config which migth have made it worse. If dhcp is set in /etc/network then apparently this overrides dhcpd settings not only for the element it is set but for all network cards mentioned below its first call at least at some timepoint throughout startup. This led to one interface having two IP adresses one showing up using "ip route" and the other showing up in ifconfig. Openhab somehow knew both of them in its management gui. I could imagine that the faulty hiding and unhiding described in your post might have caused the system to call new streams through the same physical network card but using different IPs leading to the network stalls observed, but this is just speculation for now. I will run a test the next days to check if at least the total blocking vanishes with only one IP per network card.

lolodomo commented 1 year ago

What is doing the JavaScript code is just to update the src attribute of the HTML img tag, switching between your video URL and a static none icon. The video URL is in fact a proxied URL delivered by the OH server.

woolf100 commented 1 year ago

I did some more tests, The issue can be forced by opening multiple streams (I use 5 for testing now 640*360 pixels mjpeg) put them in one subelement e.g. Text item=Kamera label="Kameras" icon="camera"{ }

and when you enter this it works fine, however when I use the backbutton something goes wrong and it takes long to leave this page or in some cases I cannot leave at all. after doing thate few times I have numeros streams still existing and streaming even when back on the first basicui screen where no such elements are located at all. The streams vanish if I close the browser.