Zombie TCP server after closing Kit app(s)

[mati-nvidia]

It seems like the TCP server within the extension is not being shutdown. This happens to me every time I use the extension. It doesn't seem to be dependent on whether I've crashed or anything like that. This prevents me from starting a new TCP server when I start/restart another Omniverse app.

I've been working around this by finding the process that's still listening on the port and killing the process. It's odd because I do see that you are closing the socket on shutdown in the code. Have you had this happen?

[Toni-SM]

Hi @mati-nvidia

Hmmm, does this happen to you on Linux or Windows?

[mati-nvidia]

I'm Windows. Is the server shutting down correctly on Linux for you? I think I read Linux allows for multiple servers on one port, but Windows doesn't.

[Toni-SM]

Hi,

Yes, I can reproduce the issue in Windows. I created a ticket for this issue (Internal ticket number TICKET-FOR-ME: VS Code extension port in use)

I always wanted to say that 🤣🤣🤣🤣

---

Do you also have the semu.misc.jupyter_notebook extension enabled? If this is the case, I think the problems come from this extension. It seems to be Jupyter notebook/lab is not closing successfully... a problem with the subprocess exit in Windows...

I need to press Ctrl + C in a terminal to make sure the Jupyter interrupted notification appears after closing the Omniverse application. But it is possible because I open the application from a terminal... I can not make sure this extension is closed by launching the Omniverse app from the launcher

Also, in Windows Powershell, I can identify the processes. The first is for kit (Jupyter parent process) and the second for Python (VS Code extension process)

Get-Process -Id (Get-NetTCPConnection -LocalPort 8225).OwningProcess
Get-Process -Id (Get-NetTCPConnection -LocalPort 8226).OwningProcess

[mati-nvidia]

Lol. You got me with the internal ticket.

I'm not using the jupyter plugin. Here's the error message from the console in case it helps: 2022-09-03 15:34:52 [Error] [semu.misc.vscode.scripts.extension] [Errno 10048] error while attempting to bind on address ('0.0.0.0', 8226): only one usage of each socket address (protocol/network address/port) is normally permitted Here's port check after open/closing Code with the extension set to autoload: I think this was after multiple launches of Code due to the multiple UDP connections.

[mati-nvidia]

That "OwningProcess" query was helpful. I found the zombie Kit/Code process in the task manager that I couldn't spot before.

Two more bits of information:

1. If I disable this extension from the Extension Manager, the TCP server closes correctly as soon as I disable and I get a clean app shutdown.
2. Running Code without the this extension enabled seems to give me a clean app shutdown.

I'm guessing this extension is keep Code/Kit alive. Just a shot in the dark, but you could try running your server shutdown code when the app shutdown event is fired as an extra try to shutdown the connection if the extension shutdown isn't doing it:

import carb.events
import omni.kit.app

# Stream where app sends shutdown events
shutdown_stream = omni.kit.app.get_app().get_shutdown_event_stream()

def on_event(e: carb.events.IEvent):
    if e.type == omni.kit.app.POST_QUIT_EVENT_TYPE:
        print("We are about to shutdown")

sub = shutdown_stream.create_subscription_to_pop(on_event, name="name of the subscriber for debugging", order=0)

[Toni-SM]

Yes, I found it also strange because, as you mention, if the extension is disabled from ExtensionManager, the app closes...

Ohh, I am going to try your snippet after dinner...

Meanwhile, I wrote and I currently using the next code to ensure the port is really closed (by killing the processes with the port opened) before creating the sockets. It is a very hard solution but its works... and because the extension closes the port if deactivated from Extension Manager, there is no problem with killing the current application process when the extension is enabled again...

(semu.misc.vscode\semu\misc\vscode\scripts\extension.py line 119), not updated to repository yet

        # ensure port is free
        if sys.platform == "win32":
            process_pid = []
            cmd = ["netstat", "-ano"] #, "|", "findstr", ":8226"]
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
            for line in p.stdout:
                if str(self._socket_port).encode() in line:
                    process_pid.append(line.strip().split(b" ")[-1].decode())
            p.wait()
            for pid in process_pid:
                carb.log_warn(f"Forced process shutdown with PID {pid}")
                cmd = ["taskkill", "/PID", pid, "/F"]
                subprocess.Popen(cmd).wait()

[Toni-SM]

Hi @mati-nvidia

Subscribing to the shutdown stream solved the problem. I also keep the implementation of killing the processes with the port in use during startup, configurable from extension.toml file

Many thanks for the feedback and support... I have drawn a new release: v0.0.3-beta