python / cpython

The Python programming language
https://www.python.org
Other
62.16k stars 29.88k forks source link

tkinter window fails to appear approx 10% of the time on MacOS 12.3+ #95769

Open PySimpleGUI opened 2 years ago

PySimpleGUI commented 2 years ago

Bug report

1 in 10 or 15 windows fails to appear when running tkinter on MacOS version 12.3+ and tkinter 8.6.12 using this simplified program:

import tkinter as tk

hmr = tk.Tk()
hmr.attributes('-alpha', 0)
hmr.withdraw()

root = tk.Toplevel(class_='toplevel')
root.attributes('-alpha', 0)

root.title('My Test Title')

frame = tk.Frame(root)
frame.pack(side=tk.TOP, anchor='nw', padx=0, pady=0, expand=False, fill=tk.NONE)
a = tk.Label(frame, text ="Hello World")
a.pack()

root.update_idletasks()
root.geometry("+600+600")
root.update_idletasks()
root.attributes('-alpha', 1)

root.mainloop()

Your environment

MacOS 12.3.1 Python 3.10.4 tkinter 8.6.12

Background

This problem was first reported to the PySimpleGUI project in mid-May. We've been working to find work-arounds and to narrow down the problem as much as possible. Because tkinter 8.6.11 isn't available pre-built on Python.org for 64 bit Macs, it's not been possible to test to see if 8.6.12 is the origin of the problem. While it is an intermittent problem, it's thankfully frequent enough to make duplication possible by running the program several times manually.

I assume it's an 8.6.12 problem as no one has reported the problem on 8.6.11 on any platform. Numerous MacOS users have been running into the problem.

The example code supplied is the most basic, shortest number of steps we've been able to create that perform the same operations that PySimpleGUI uses to build and move the window.

PySimpleGUI's default location for user windows is the center of the screen. So that users do not see the window "jump" from the starting location, the alpha channel is set to zero when the window is created and remains 0 while it's built and moved. Once moved into the final position, the Alpha channel is set to 1 and thus the window appears in the correct location without a visible jump.

PySimpleGUI commented 2 years ago

A recent PySimpleGUI release, 4.60.3, added a workaround that has been performing well for several weeks.

MacOS users running 12.3+ have the option of telling PySimpleGUI to set the final alpha channel to be 0.99 instead of 1.0.

image

It's a hack that works but I can't think of a logical reason as to why... at least the window does appear.

[EDIT - ooops on the "close issue".... clicked close instead of comment]

PySimpleGUI commented 2 years ago

I've not logged a bug here for the tkinter team. Can someone give me some idea of what to expect in terms of the process, timeline etc? Can someone get back with me when it's reproduced? It's been seen by numerous users and happens frequently using the code provided.

It's a serious problem that impacts all releases of PySimpleGUI on newer Macs that don't have the new emergency dot release of PySimpleGUI with the workaround described above.

ronaldoussoren commented 2 years ago

I've tweaked the code a little to make it easer to stop the script:

import tkinter as tk

hmr = tk.Tk()
hmr.attributes('-alpha', 0)
hmr.withdraw()

root = tk.Toplevel(class_='toplevel')
root.attributes('-alpha', 0)

root.title('My Test Title')

frame = tk.Frame(root)
frame.pack(side=tk.TOP, anchor='nw', padx=0, pady=0, expand=False, fill=tk.NONE)
a = tk.Label(frame, text ="Hello World")

a.pack()

b = tk.Button(frame, text="QUIT", command=frame.quit)
b.pack()

root.update_idletasks()
root.geometry("+600+600")
root.update_idletasks()
root.attributes('-alpha', 1)

root.mainloop()

Then ran the script in a loop, and I seem to reliably get a GUI shown (I ran the script about 40 times)

This is with Python 3.10.6 (installer on Python.org) on macOS 12.5.1 on an M1 system.

W.r.t. process: there's not a lot of core developers that are experts on Tkinter, and most Mac specific Tkinter issues end up being a bug in Tk itself. This really needs a reliable reproducer before we can do anything, and even then it might take time when this is a Tk bug because we'll only end up with a fix then when the fix is in their repo and there is a new release.

In general moving backwards with Tk releases is not worthwhile, new releases contain bug fixes for other bugs in the macOS platform support.

PySimpleGUI commented 2 years ago

Thank you for the update and the information about process.

There's another 8.6.12 problem that was logged in the older bug tracking system that's impacting PySimpleGUI, but my account isn't listed because of the migration. I'll be sure and reply to that one to at least get subscribed.

I will let you know how the testing goes.

PySimpleGUI commented 2 years ago

The problem remained when we tried it on 3.10.6 as suggested.

We even upgraded to 12.5.1 to be sure we had the same environment.

It still failed within about 20 tries. It varies due to it being intermittent. I have no clue what the cause if for the intermittent nature, so I can't predict with any certainty many runs it takes.

We always run a fresh run every time. There is no loop done in the Python file. If a loop is added inside the file, then we're not simulating the real-world condition this is happening under. It doesn't take very many tries to launch it by hand a few times before it happens.

PySimpleGUI commented 2 years ago

In general moving backwards with Tk releases is not worthwhile, new releases contain bug fixes for other bugs in the macOS platform support.

I don't agree with this viewpoint. I run tkinter from 8.6.2 to 8.6.12 and haven't had trouble to the point I am compelled to upgrade or else I won't be able to complete an operation or the system is unstable.

Rarely are tkinter problems encountered, especially on Windows and Linux. The Mac has been problematic for quite some time, but it's hard to pinpoint exactly why that was early on. Once users started to install from Python.org, then the Mac issue lessened. 8.6.12 has had more problems than any other release I've run.

ronaldoussoren commented 2 years ago

There have been a lot of bug fixes in macOS support for Tk in the last couple releases, thanks to hard work by a couple of Cocoa Tk maintainers. There may be some regressions as well, but in general new versions are better than old ones.

IIRC the upgrade from 8.6.11 to 8.6.12 was necessary to fix easily reproduced issues with Tkinter, possibly in IDLE.

As I mentioned in my previous message I haven't been able to reproduce this issue on my machine. Because of this I cannot debug the issue, let alone provide a fix. A reproducer is also necessary to be able to forward this to the Tk maintainers (preferably in Tcl, but so far they've been fine with Tkinter reproducers as well).

PySimpleGUI commented 2 years ago

I'll run the code I gave you past the many users with this issue and have them try it specifically. They've been writing their own code as they have frantically trying to figure out how to get around the problem.

You were able to initially get it to happen? Did you run it manually, not in a loop? One run at a time? That's the test. Maybe that's why you're not able to reproduce it. How many machines did you try it on?

Until you know the root cause, I don't think you can point to a version of Python and say, that fixed it, when multiple other people are hitting the same problem with the same environment you are.

Most of my users don't user IDLE....I would say less than 1%, if that. So those bug fixes simply aren't relavent. The other bug we filed last year that got migrated over to github is another one that started in 8.6.12 and happens on all operating systems.

You've got the experience to understand intermittent / timing problems as well as something magically going away doesn't mean it's fixed. I hope this issue can get attention that will go at the root cause.

One strange thing that I touched on earlier is that the fix is to set the alpha to 0.99 instead of 1.0. Setting to 1.0 fails. Setting to 0.99 always works. To me, that's a strong hint of a root cause.

Would it be helpful for the end users on the Mac that are having the issue participate in our discussion so you can log information? I can go grab a handful and get them sending you info.

If you can tell me a bit more about what you need from me, I would be happy to supply it. I would like to see another machine perhaps tried. We bought an M1 just to work on this issue.

resnbl commented 2 years ago

Out of 30 runs, mine failed on the 9th attempt and succeeded every other time. I ran the script from a Terminal window ("Command prompt" in Windows-speak) as opposed to from an editor/IDE. Python 3.10.5 (main, Jul 30 2022, 20:40:22) [Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin installed via pyenv (i.e.: built from source) Tkinter 8.6.12 iMac 12.5.1 (Intel)

Note: AFAIK, the program is still running (responds to Cmd-Q), just not displaying the window. Can someone modify the test script so that when the problem occurs, we can hit some keystroke that will cause a "dump" of the window's state (or whatever info would be useful)?

EdwardChamberlain commented 1 year ago

I am able to recreate this issue using the posted script above. Python 3.10.4 tinter 8.6 MacBook Pro (14-inch, 2021) M1 Pro

This issue is present in a number of programs I have written and makes TK impossible to use on Mac.

EdwardChamberlain commented 1 year ago

@ronaldoussoren I have also included here a video of me reproducing the issue. In the video you see me run the script you posted above successfully, close the script and then run it again. On the second run you can see no window is created. I find that this happens around 1 in 5 times that the script is run.

After the window fails to appear I use the Mac "Mission control" to organise open windows and a space is left for an invisible window alongside VSCode. I then close the program using ctrl+C and then re-enter mission control and you will see that the space left for the invisible window is now gone.

https://user-images.githubusercontent.com/7659338/194956191-c1acc88e-0286-4988-b6a7-3fe629963d21.mov

I hope that can help progress this issue somewhat.

Bobronium commented 1 year ago

DISCLAIMER: I haven't read through all comments and I'm not using PySimpleGUI either. My comment is possibly unrelated or is a dup.

I just want to share my experience since it may mean that the problem is not with the PySimpleGUI/tkinter/CPython, but rather with macOS, and tkinter just provides a way to reproduce that OS bug.

I stumbled upon https://github.com/PySimpleGUI/PySimpleGUI/issues/5471 while searching for the fix for invisible windows throughout the macOS. It happened with every opened window once. Now my Google Chrome windows disappeared.

I'm pretty confident that I've encountered it on earlier versions of macOS, and was able to reproduce this effect with any window using expose, but I've forgotten the way to do it.

Other examples of similar behaviour

System info

PySimpleGUI commented 1 year ago

rather with macOS, and tkinter just provides a way to reproduce that OS bug.

You could say that any application that calls tkinter that calls the OS provides a way to reproduce the bug. Top to bottom, it doesn't matter how the bug is demonstrated, the bug still exists.

The question is, where does it get fixed? Does it not get fixed? Should users just live with 1 in 20 windows not appearing?

There have been dozens of problems on the Mac that have been addressed by tkinter and PySimpleGUI. It's part of supplying a library for users to create applications. For years tkinter didn't support colors other than white for all TK.Buttons, on the Mac. It eventually got fixed but was a difficult problem until then. TTK.Buttons were the only workaround I could find.

IMO, if there's a bug "below" your code, there's an obligation to "fix" the problem if you're supplying a library that users expect to function. The application developer doesn't care where the problem is, they just want their window to appear.

terryjreedy commented 1 year ago

Does IDLE have the same problem of disappearing windows?

Some variations in the test program to try. Given the .99 fix, try 3. first.

  1. Delete hmr = tk.Tk() and the next two lines.
  2. Add hmr as a positional argument to the Toplevel call.
  3. Replace the 'alpha' manipulations with root.withdraw() and root.deiconify().
  4. Don't move the window (or was this tested already?).
PySimpleGUI commented 1 year ago

I've got workarounds running. The suggestions are more about PySimpleGUI, not about the underlying problem that I'm asking for assistance looking at. The idea, call me crazy, is for the existing PySimpleGUI installs from the past several years to not have to change and instead when Apple and tkinter update, they do so in a way that doesn't impact all those users.

We started working in May on the troubleshooting, workarounds, etc, so that the user base can continue running. That investment was made and got the results I was after in terms of finding the best solution possible without loss of functionality


EDIT:

I think perhaps there may be some confusion about the code provided. It's provided so that the problem can be easily replicated. It's not the code that's being used by anyone. It's a test case. It's a way for me to give the tktiner team something other than "it doesn't work" as a bug submission that doesn't make the replication task easy will rarely get worked on.

ronaldoussoren commented 1 year ago

@culler, is this something you've seen before with Cocoa Tk?

I've been unable to reproduce this issue on my laptop (M1, macOS 12.6, Python 3.11 or 3.10.8 both installed using the python.org installer, both link to Tk 8.6.12). That makes it close to impossible for me to even determine if this is a problem in Tkinter or in Tk itself.

Btw. @PySimpleGUI: I much appreciate that you wrote a small reproducer for the issue, even if it refuses to fail for me personally.

PySimpleGUI commented 1 year ago

That's so strange that we've been able to reproduce it quite easily, and rebuilt tkinter from source as well to do additional tests.

You're welcome for the code! Of course I'm going to supply you with code that makes it as easy as possible to reproduce. It's the "help you help me" rule of software issue submissions. If you have to work hard just to start to debug, then odds are the problem won't get any attention.

The sample code was designed so that you can paste the command, press return, see if it failed, exit the window, paste again, etc. It's very manual on purpose because it shouldn't take very many tries and we're trying to make it as natural of a use case as possible.

I doubt if the Python version matters, but maybe try matching the version numbers exactly with those that other users reported above?