Kytos running in foreground takes forever to stop

[italovalcy]

Hi,

While running newest version of Kytos (from amlight/kytos:latest docker container), with tmux, Kytos takes forever to stop (in fact, I waited a long time and then just gave up and issue a CTRL+C):

kytos $> (CTRL+D)
Stopping Kytos daemon... Bye, see you!
2024-08-05 13:26:39,305 - INFO [kytos.core.controller] (MainThread) Stopping Kytos controller...
2024-08-05 13:26:39,305 - INFO [kytos.core.controller] (MainThread) Stopping Kytos
...
2024-08-05 13:26:39,335 - INFO [kytos.core.api_server] (MainThread) The Rest endpoints from /api/kytos/of_core/ were disabled.
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(msg_in, min_hits=5, min_size=512, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(msg_out, min_hits=5, min_size=512, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(raw, min_hits=5, min_size=512, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(app, min_hits=5, min_size=1024, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(threadpool_sb, min_hits=5, min_size=256, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(threadpool_app, min_hits=5, min_size=512, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping QueueMonitor(threadpool_db, min_hits=5, min_size=256, delta_secs=5)...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping API Server...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopped API Server
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopping TCP Server...
2024-08-05 13:26:39,335 - INFO [kytos.core.controller] (MainThread) Stopped TCP Server

[italovalcy]

Running without tmux results in the same behavior:

# kytosd -f --database MongoDB
Web update - Web UI was not updated
2024-08-05 13:30:42,805 - INFO [kytos.core.db] (MainThread) Starting DB connection
2024-08-05 13:30:42,806 - INFO [kytos.core.db] (MainThread) Trying to run 'hello' command on MongoDB...
2024-08-05 13:30:42,832 - INFO [kytos.core.db] (MainThread) Ran 'hello' command on MongoDB successfully. It's ready!
...
kytos $> (you type CTRL+D)
Stopping Kytos daemon... Bye, see you!
2024-08-05 13:31:00,578 - INFO [kytos.core.controller] (MainThread) Stopping Kytos controller...
2024-08-05 13:31:00,578 - INFO [kytos.core.controller] (MainThread) Stopping Kytos
...
2024-08-05 13:31:00,612 - INFO [kytos.core.controller] (MainThread) Stopping API Server...
2024-08-05 13:31:00,612 - INFO [kytos.core.controller] (MainThread) Stopped API Server
2024-08-05 13:31:00,612 - INFO [kytos.core.controller] (MainThread) Stopping TCP Server...
2024-08-05 13:31:00,612 - INFO [kytos.core.controller] (MainThread) Stopped TCP Server
^CException ignored in: <module 'threading' from '/usr/lib/python3.11/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1583, in _shutdown
    lock.acquire()
KeyboardInterrupt:
# date
Mon Aug  5 13:36:05 UTC 2024

Waited for 5min without success.

[Alopalao]

Found 1 cause of a possible hanging from coloring. But it is not the case reported above.

[Alopalao]

Reasons for a long shutdown:

• A @retry was triggered just before a shutdown
• time.sleep() was not finished. We could be using self.__event.wait() from the Event in base instead so we do not wait for slept threads to wake up.

These two do not hang, but with a large topology the shutdown could take up to a minute.

[viniarck]

It's believed that with https://github.com/kytos-ng/kytos/pull/501 this issue has been indirectly fixed too. Since it hasn't been exactly reproduced we'll leave this issue here open for some more time and if it doesn't manifest or get reproduced it'll get closed. I've removed the 2024.1 label due to this considered indirectly fixed for the moment. Thanks, guys