Improve Python type-stubs

lojack5 commented 7 months ago

The wxPython type-stubs are nice in that they expose what classes, methods, etc are available, but there is no type information in them (there's even a comment in makePyArgsString referencing implementing this). This is my 90% of the way there implementation of exposing this type information to the type-stubs.

I also didn't find any open issues regarding type-stubs, so I haven't linked an issue.

For the most part, all of the type-information was already there in the doxygen files, it just had to be pulled into the args strings. There are a few exceptions, mostly revolving around types that are C++ typedefs (ex: WindowID and Coord).

By using from __future__ import annotations, all of the type-hints are stringized, so even in the cases where the name referenced is undefined (ex: richtext's TextAttrDimensionFlags), the type-stub files are still valid. The hints in those cases will just provide no information.

I do have a few discussion points to probably resolve before this is truly ready though:

From what I can tell by the buildbot output, we build as low as Python 3.6. So I'm assuming any new code doesn't need to be compatible with earlier versions, is this a good assumption?
How to handle typing.ParamSpec: related to the above, ParamSpec was added in Python 3.10. Currently it's only used to type-hint CallAfter and CallLater, how would you like this import handled:
- If you're OK with a new dependency typing-extensions could be added for Python versions < 3.10, then import from there.
- I can skip out on typing CallAfter and CallLater, although if other methods come up in the future that would benifit from this sort of typing, we run into the issue again.
Code style - I tried to keep with the same code style in surrouding areas but I probably named a few variables differently at some point. I also used f-strings since those are in Python 3.6+. I can go back and use Python 2 style formatting strings if that's what's wanted however.
There are still some typedef types that aren't covered. I could dig through and find every instance in the PYI files where a type is referenced that's undefined, but that may end up with a lot of hard-coded conversions (usually to int) in the type-name fixup code. Let me know of any changes there I need to make.
I have no knowledge of PI files (WingIDE files according to the header), do these changes break those?
My choice of how to handle C++ enums. I went the route of using the stdlib's enum.IntEnum and enum.IntFlag here. My reasoning is - they're still typed as ints this way (so compatible), and I define the enums with a prefixed _ so the type-stubs don't expose them as some sort of real type that is actually accessible in actual wxPython code. I then make a TypeAlias for the name that includes the enum values or an int, so type-checkers won't complain to users that pass raw ints into methods typed with these enums. The enum then basically serves the purpose of grouping various values together and signaling the the callers which of the named values are expected at the call sites. I can go back to having enums just being typed as plain old int, but I feel we lose the information of "what enum value or flags should you be using here".

I also just realized while writing this that subscripting list and unions with | weren't added until Python 3.9 and 3.10, respectively, so some for me:

[x] Replace list[...] with List[...] and import List from typing.
[x] Replace X | Y with Union[X, Y] and import Union from typing.
[x] Replace tuple[...] with Tuple[...] and import Tuple from typing.
[x] Replace from collections.abc import Callable with from typing import Callable - subscripting was introduced in Python 3.10

swt2c commented 7 months ago

On October 18, 2023 6:10:28 PM EDT, Lojack @.***> wrote:

@lojack5 commented on this pull request.

@@ -3,3 +3,4 @@ numpy < 1.17 ; python_version <= '2.7' numpy ; python_version >= '3.0' and python_version < '3.12'

pillow < 3.0

six +typing-extensions; python_version < '3.10'

Syntactically not needed, but I'm more than happy to change it to match the style used in the rest of the file.

Oh..I assumed that's why the CI was unhappy with it.

Currently working though issued shown to me through the buildbot, so I'll get to it after I solve all of those issues that come up.

BTW, I was looking for a CONTRIBUTING.md or a style guide somewhere but didn't find anything, am I missing it?

There isn't one AFAIK.

lojack5 commented 7 months ago

Ok, I think this is ready for review and whatever fixes you want from me. The CI is failing on all Windows builds - appears to be a MSVC version mismatch?

error STL1001: Unexpected compiler version, expected MSVC 19.36 or newer.

For reference, on my machine it builds fine for Python 3.12, Win 11 with the latest MSVC build tools installed.

Buildbot output (for Python 3.11):

Will build using: "C:\hostedtoolcache\windows\Python\3.11.6\x64\python.exe"
3.11.6 (tags/v3.11.6:8b6ee5b, Oct  2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]
Python's architecture is 64bit
cfg.VERSION: 4.2.2a1

Running command: build_py
Checking for D:\a\1\s\bin\waf-2.0.24...
Not found.  Attempting to download...
Connection successful...
Data downloaded...
Checking for D:\a\1\s\bin\waf-2.0.24...
CL.exe: C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\cl.exe
"C:\hostedtoolcache\windows\Python\3.11.6\x64\python.exe" D:\a\1\s\bin\waf-2.0.24 --msvc_arch=x64 --jobs=4 --python="C:\hostedtoolcache\windows\Python\3.11.6\x64\python.exe" --out=build/waf/3.11/x64/release configure build 
Setting top to                           : D:\a\1\s 
Setting out to                           : D:\a\1\s\build\waf\3.11\x64\release 
Checking for program 'CL'                : C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.35.32215\bin\HostX64\x64\CL.exe 
Checking for program 'CL'                : C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.35.32215\bin\HostX64\x64\CL.exe 
Checking for program 'LINK'              : C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.35.32215\bin\HostX64\x64\LINK.exe 
Checking for program 'LIB'               : C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.35.32215\bin\HostX64\x64\LIB.exe 
Checking for program 'MT'                : C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\\x64\MT.exe 
Checking for program 'RC'                : C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\\x64\RC.exe 
Checking for program 'python'            : C:\hostedtoolcache\windows\Python\3.11.6\x64\python.exe 
Checking for python version >= 3.7.0     : 3.11.6 
'configure' finished successfully (3.838s)

vs what I see on my machine:

Will build using: "C:\Users\Lojack\Desktop\code\wxPython\.venv\Scripts\python.exe"
3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)]
Python's architecture is 64bit
cfg.VERSION: 4.2.2a1

Running command: build
Running command: build_wx
CL.exe: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\cl.exe
wxWidgets build options: ['--wxpython', '--unicode']
Updating wx/msw/setup.h
setting build options...
nmake.exe -f makefile.vc UNICODE=1 OFFICIAL_BUILD=1 COMPILER_VERSION=140 SHARED=1 MONOLITHIC=0 USE_OPENGL=1 USE_GDIPLUS=1 BUILD=release

Microsoft (R) Program Maintenance Utility Version 14.37.32825.0
Copyright (C) Microsoft Corporation.  All rights reserved.

WARNING: msgfmt and/or make commands not found, message catalogs not 
         rebuilt. Please install gettext and associated tools.
Finished command: build_wx (0m1.6s)
Running command: build_py
Checking for C:\Users\Lojack\Desktop\code\wxPython\bin\waf-2.0.24...
CL.exe: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\cl.exe
"C:\Users\Lojack\Desktop\code\wxPython\.venv\Scripts\python.exe" C:\Users\Lojack\Desktop\code\wxPython\bin\waf-2.0.24 --msvc_arch=x64 --python="C:\Users\Lojack\Desktop\code\wxPython\.venv\Scripts\python.exe" --out=build/waf/3.12/x64/release configure build
Setting top to                           : C:\Users\Lojack\Desktop\code\wxPython 
Setting out to                           : C:\Users\Lojack\Desktop\code\wxPython\build\waf\3.12\x64\release
Checking for program 'CL'                : C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\CL.exe 
Checking for program 'CL'                : C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\CL.exe 
Checking for program 'LINK'              : C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\LINK.exe 
Checking for program 'LIB'               : C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX64\x64\LIB.exe
Checking for program 'MT'                : C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\\x64\MT.exe
Checking for program 'RC'                : C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\\x64\RC.exe
Checking for program 'python'            : C:\Users\Lojack\Desktop\code\wxPython\.venv\Scripts\python.exe
Checking for python version >= 3.7.0     : 3.12.0 
'configure' finished successfully (2.133s)
Waf: Entering directory `C:\Users\Lojack\Desktop\code\wxPython\build\waf\3.12\x64\release'
Waf: Leaving directory `C:\Users\Lojack\Desktop\code\wxPython\build\waf\3.12\x64\release'
'build' finished successfully (0.518s)

The only thing that sticks out to me is the buildbot is calling into executables found in 14.35.32215, whereas on my machine its 14.37.32822.

lojack5 commented 7 months ago

@swt2c sorry to bug. Are you able to review this (or help track down the windows compile error)? Or maybe know who to ping for it?

swt2c commented 7 months ago

@swt2c sorry to bug. Are you able to review this (or help track down the windows compile error)? Or maybe know who to ping for it?

Yes, I will try to track down the windows compiler error eventually. Too much to do and too little time.. :(

Metallicow commented 7 months ago

@lojack5
... ummm I'm not sure how to explain my personal wx wrapper. It is similar to wrye bashes original 1file... it would be explained something like this ...

def VARIABLE(VARIABLEARG, VARIABLEKEYWORDARG, *VARIABLE, VARIABLE):=

I know you don't want to eat shit and dream about tommorrow... but ... this is a vary real situation that could occur since the wx license states you can do anything you want to when you compile it, and such things as micropython have occured... ... so as one unmonkey to another monkey ... Guido might say "This could be dangerous" Take all that into consideration... and follow the "Monkey godness of order: PAID MEGA MODDING", which may not have much order now... and try your best to make it all work out.

Good luck in what you are doing.

lojack5 commented 7 months ago

Not sure what you did on the backend, but now everything except Python 3.7 are succeeding, nice! I looked into that, there wasn't even a build run for Python 3.7, so maybe a mismatch between what the GitHub check wants and what Azure was commanded to build?

Metallicow commented 7 months ago

Did scot Talbert just tell me he was gonna bash at my skull in or should I look for another candidate...? What the fuck is wrong with you....

On Tue, Oct 31, 2023, 9:21 PM Lojack @.***> wrote:

Not sure what you did on the backend, but now everything except Python 3.7 are succeeding, nice! I looked into that, there wasn't even a build run for Python 3.7, so maybe a mismatch between what the GitHub check wants and what Azure was commanded to build?

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1788297124, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRHBGGY2345WPQQXMMLYCGW3BAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBYGI4TOMJSGQ . You are receiving this because you commented.Message ID: @.***>

Metallicow commented 7 months ago

I believe he said I will be the first to attack him...

On Wed, Nov 1, 2023, 9:28 PM Metalio Bovinus @.***> wrote:

Did scot Talbert just tell me he was gonna bash at my skull in or should I look for another candidate...? What the fuck is wrong with you....

On Tue, Oct 31, 2023, 9:21 PM Lojack @.***> wrote:

Not sure what you did on the backend, but now everything except Python 3.7 are succeeding, nice! I looked into that, there wasn't even a build run for Python 3.7, so maybe a mismatch between what the GitHub check wants and what Azure was commanded to build?

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1788297124, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRHBGGY2345WPQQXMMLYCGW3BAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBYGI4TOMJSGQ . You are receiving this because you commented.Message ID: @.***>

lojack5 commented 7 months ago

???

Metallicow commented 7 months ago

I'd like to know if you have been getting spam/scam calls? This has gotten bad.

On Wed, Nov 1, 2023, 10:17 PM Lojack @.***> wrote:

???

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1789999438, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRDIOSACR7VIEGWGOSDYCMGDFAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZHE4TSNBTHA . You are receiving this because you commented.Message ID: @.***>

Metallicow commented 7 months ago

Scott, what do you have to say? Are you spamming people or it is just a "bot"?

On Tue, Oct 31, 2023, 3:30 PM Scott Talbert @.***> wrote:

Reopened #2468 https://github.com/wxWidgets/Phoenix/pull/2468.

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#event-10825717361, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRGK3ACKVKRF6E2K4NLYCFNXNAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJQHAZDKNZRG4ZTMMI . You are receiving this because you commented.Message ID: @.***>

Metallicow commented 7 months ago

@Scott Talbert, you seem to be a nice guy and are not on my bad list... Only scammers

Let me relay a little bit of history about these types of people... Once upon a time, I was globally assigned a quack. I politely showed up and entered his office. After a heated discussion I proceeded the ever-loving act of kicking the dog shit out of him, and then left shutting the door ever so gently. I was then diagnosed with COVID-19. The politicians were a 51/50, half was radical and the other half couldn't stop laughing. I told them like a past lawyer I had, that he was fired. Then for the next 2 years they rounded all these types of people up and vaccinated them like Nazi Jews. Let's not let another pandemic infect the airwaves again please. Thank you.

On Thu, Nov 2, 2023, 12:51 AM Metalio Bovinus @.***> wrote:

Scott, what do you have to say? Are you spamming people or it is just a "bot"?

On Tue, Oct 31, 2023, 3:30 PM Scott Talbert @.***> wrote:

Reopened #2468 https://github.com/wxWidgets/Phoenix/pull/2468.

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#event-10825717361, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRGK3ACKVKRF6E2K4NLYCFNXNAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJQHAZDKNZRG4ZTMMI . You are receiving this because you commented.Message ID: @.***>

swt2c commented 7 months ago

Not sure what you did on the backend, but now everything except Python 3.7 are succeeding, nice! I looked into that, there wasn't even a build run for Python 3.7, so maybe a mismatch between what the GitHub check wants and what Azure was commanded to build?

Yeah, I fixed the CI builds and also removed Python 3.7 as it is EOL. Your old builds with Python 3.7 before that removal are still there. So at least your changes all build. Unfortunately, I think these changes probably need to be reviewed by @RobinD42 before merging.

lojack5 commented 7 months ago

Awesome :). Thanks for the work getting that cleared up (and the thread cleanup). Hopefully Robin has some time in the future to look at this

lojack5 commented 7 months ago

Just checking in to see if any dev's have time to review this.

Metallicow commented 6 months ago

Thank you LoJack and especially thank Scott for cleanup. I do not see anything abnormal in the stubs. Tomorrow or the next day I might... Lol.

On Sun, Nov 12, 2023, 11:08 PM Lojack @.***> wrote:

Just checking in to see if any dev's have time to review this.

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1807486102, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRBLJYWE7YY7PC55MSLYEGTODAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBXGQ4DMMJQGI . You are receiving this because you commented.Message ID: @.***>

Metallicow commented 6 months ago

... About the type stuff... This doesn't have anything to do with most of that tho the "random" underscores might need to be watched. Generators are known to jump out of documentation strings, and into code, which at that point it becomes destructive.

On Tue, Dec 5, 2023, 7:48 AM Metalio Bovinus @.***> wrote:

Thank you LoJack and especially thank Scott for cleanup. I do not see anything abnormal in the stubs. Tomorrow or the next day I might... Lol.

On Sun, Nov 12, 2023, 11:08 PM Lojack @.***> wrote:

Just checking in to see if any dev's have time to review this.

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1807486102, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRBLJYWE7YY7PC55MSLYEGTODAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBXGQ4DMMJQGI . You are receiving this because you commented.Message ID: @.***>

lojack5 commented 3 months ago

Checking back in on this, any chance for a dev to take a look at this?

Metallicow commented 3 months ago

What do you want to test? I'm not sure. Python editors or dynamic python editors? You can use the interactive interpreter or Sourcecoder. Otherwise, you could use PyCharm, or any other editor NOT written in python. Yes I know it sucks choosing an editor and not all of them are open source, so what you want to do is sometimes colleged behind a idea that has been there for years, but has not yet surfaced, well... publicly. I'm just asking for your specs on how to do your "idea".

On Sat, Feb 24, 2024, 11:14 PM Lojack @.***> wrote:

Checking back in on this, any chance for a dev to take a look at this?

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/pull/2468#issuecomment-1962815676, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRDJ6FDEKJQC624TT3TYVLCCPAVCNFSM6AAAAAA6F72FX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSHAYTKNRXGY . You are receiving this because you commented.Message ID: @.***>

wxWidgets / Phoenix

Improve Python type-stubs #2468

pillow < 3.0