Tcl event loop callback woes

[d02097bd-6b6a-401d-832d-5fbf093931f9]

BPO	452973
Nosy	@gvanrossum, @loewis, @warsaw, @tiran

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['expert-tkinter'] title = 'Tcl event loop callback woes' updated_at = user = 'https://bugs.python.org/egnor' ``` bugs.python.org fields: ```python activity = actor = 'christian.heimes' assignee = 'none' closed = True closed_date = closer = 'christian.heimes' components = ['Tkinter'] creation = creator = 'egnor' dependencies = [] files = [] hgrepos = [] issue_num = 452973 keywords = [] message_count = 6.0 messages = ['6059', '6060', '6061', '6062', '6063', '59290'] nosy_count = 5.0 nosy_names = ['gvanrossum', 'loewis', 'barry', 'egnor', 'christian.heimes'] pr_nums = [] priority = 'normal' resolution = 'out of date' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue452973' versions = [] ```

[d02097bd-6b6a-401d-832d-5fbf093931f9]

I have C code which handles I/O and needs an event dispatcher. It is running in the context of a Python GUI application, and uses the Tcl event loop, using Tcl_CreateFileHandler and friends. The C code gets callbacks from the Tcl event loop fine, but when it attempts to call into the Python application from one of these callbacks, things go wrong.

• Tkinter has released the GIL and hidden the current thread state. I can work around this by re-acquiring the GIL from the callback and creating a new thread state.

• When the callback is invoked, Tkinter's tcl_lock is held. If the Python code invoked from the callback ultimately calls some other Tkinter function, the tcl_lock is still held, and deadlock results. The only way to work around this is to use a single-threaded Python build.

• If the Python code returns an error, there's no way to stop the event loop to report the error up. Tkinter's error-reporting mechanisms are inaccessible.

In general, Tkinter has a lot of infrastructure for managing callbacks from the Tcl event loop. If a third party C library wants to use the same event loop, that infrastructure is unavailable, and it is very difficult to work with Python.

Unfortunately, short of using threads (which have their own problems), there's no other alternative for an external C library to do I/O without blocking the GUI. I've seen several problem reports from people trying to do exactly this, though they almost never figure out all of what's going on, and nobody else ever has any good advice to offer.

[warsaw]

Logged In: YES user_id=12800

For lack of a volunteer or better victim... er, assigning to /F

[gvanrossum]

Logged In: YES user_id=6380

Unassigning -- /F is a black hole. :-(

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

It seems that all you need to have is access to the Tcl lock. Would that solve your problem?

As for reporting the error up: This is certainly possible. Just implement a _report_exception method on your widget; define it as

    def _report_exception(self):
        raise

[d02097bd-6b6a-401d-832d-5fbf093931f9]

Logged In: YES user_id=128950

I *think* external access to the tcl_lock would do it (it's been a while). There are a bunch of helper functions/macros inside the Tkinter code for handling this situation; exposing and documenting those would be ideal.

As far as error reporting, I don't think that suffices. (Again, it's been a while.) The problem is that the exception is returned to the C side, which must figure out what to do with it. Specifically, the Tcl event loop should be stopped and the exception reported to whoever invoked it... but there's no direct way to stop the Tcl event loop. (Fuzzy memories here.)

[tiran]

The bug report is several years old and probably outdated. Please create a new issue if you are still interested in the problem.