local_llm: no way to change or disable ICE server for webrtc

[TadayukiOkada]

I'd like to run live llava completely locally on Jetson including a web browser. However, if I turn off wifi before starting live llava, the video won't play on the browser. If I turn off wifi after starting it, the video continues to play. I'm guessing this is because the default ICE server is set to google's STUN server. I tried to find a way to change or disable it but it seems there's no way to change it using option flags right now. I'm willing to create a PR for this, but not sure what's the cleanest way to set it.

[dusty-nv]

@TadayukiOkada it looks like these can be disabled like this: https://stackoverflow.com/questions/30742431/webrtc-on-isolated-lan-without-ice-stun-turn-server

That is on the javascript/client side. However on the server side, I'm not sure if gstreamer webrtcbin will work without STUN server. Here are the references to it in jetson_utils: https://github.com/search?q=repo%3Adusty-nv%2Fjetson-utils%20stun&type=code

I would get jetson_utils working first without STUN, and then the javascript in NanoLLM can be updated to reflect it. It may be tricky however to support both ways.

[manhntm3]

Hi @dusty-nv,

I've tried to disable STUN server on both javascript and gstreamer by commenting out the STUN part and it works when the server(gstreamer) side is not connected to Internet.

However, it still fails to connect when viewed from a PC (same LAN) that doesn't connect to the Internet.

[TadayukiOkada]

@manhntm3 have you tried setting up a local TURN/STUN server such as coturn? It shouldn't be necessary but it might be helpful to move forward.

[POLROTY]

@manhntm3 I am having the same problem, I can't view the video from an other device on the lan if not connected to internet. My jetson also crashes because the queue frames fills up all the RAM. Have you found a fix?

[Rz-Rz]

@manhntm3 I am also having the same problem. This is very problematic.

[dusty-nv]

Can you try this attempted fix from jetson-utils? https://github.com/dusty-nv/jetson-utils/commit/72c4e095f0575091992a5803d79af08e138e3369

video-viewer /dev/video0 webrtc://@:8554/output --stun-server disabled

Then navigate your browser to http://JETSON_IP:8554

When it is set to disabled, I set the ice-servers in the built-in HTML/Javascript to empty array as suggested here:

https://groups.google.com/g/discuss-webrtc/c/s7WsDPjHIj4/m/v8a6jPWtF8QJ

It worked for me both with it disabled and the default server (stun.l.google.com:19302)

The web-side changes are only for the jetson-utils built-in debug viewer on port 8554 so far.

[manhntm3]

For me, it only works when the computer that views Jetson's webpage is connected to the Internet. If we disable the Internet from the laptop that runs the browser and see Jetson's webpage through port 8554, it still doesn't work.

[dusty-nv]

Can you confirm in the clientside HTML/Javascript that the ICE stun servers are in fact being set to empty?

---

From: Manh Nguyen @.> Sent: Monday, June 24, 2024 2:18:21 AM To: dusty-nv/jetson-containers @.> Cc: Dustin Franklin @.>; Mention @.> Subject: Re: [dusty-nv/jetson-containers] local_llm: no way to change or disable ICE server for webrtc (Issue #535)

For me, it only works when the computer that views Jetson's webpage is connected to the Internet. If we disable the Internet from the laptop that runs the browser and see Jetson's webpage through port 8554, it still doesn't work. Screenshot.2024-06-24.at.13.15.52.png (view on web)https://github.com/dusty-nv/jetson-containers/assets/53960609/a21f44f0-ea48-4201-9883-3037224456f7

— Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-containers/issues/535#issuecomment-2185698667, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK5HQVZV2W56CYZQUNLZI6223AVCNFSM6AAAAABIEEFNGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGY4TQNRWG4. You are receiving this because you were mentioned.Message ID: @.***>

[POLROTY]

It seems to be working for me, I made a new connection mechanism client side directly with a websocket and with a local adapter-latest.js, as I think this import doesn't work without access to the internet : line 44 of WebrtcServer.cpp

[manhntm3]

Can you confirm in the clientside HTML/Javascript that the ICE stun servers are in fact being set to empty? … ____ From: Manh Nguyen @.> Sent: Monday, June 24, 2024 2:18:21 AM To: dusty-nv/jetson-containers @.> Cc: Dustin Franklin @.>; Mention @.> Subject: Re: [dusty-nv/jetson-containers] local_llm: no way to change or disable ICE server for webrtc (Issue #535) For me, it only works when the computer that views Jetson's webpage is connected to the Internet. If we disable the Internet from the laptop that runs the browser and see Jetson's webpage through port 8554, it still doesn't work. Screenshot.2024-06-24.at.13.15.52.png (view on web)https://github.com/dusty-nv/jetson-containers/assets/53960609/a21f44f0-ea48-4201-9883-3037224456f7 — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK5HQVZV2W56CYZQUNLZI6223AVCNFSM6AAAAABIEEFNGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGY4TQNRWG4. You are receiving this because you were mentioned.Message ID: @.***>

Yes, I can confirm the clientside HTML/Javascript is being set to empty.

What I did to stop Internet from my laptop and keep the LAN connection is change to static IP and empty the router box. It might affect only on my side. On your side, does the WebRTC video work when you disable Internet on the client machine or both Jetson and client?