wxPython and Kicad packages request for comment

[stahta01]

Kicad depends on wxPython and wxWidgets3.1 packages.

And, wxPython depends on wxWidgets3.1 package.

I was thinking that removing the depends on the wxWidgets3.1 package would remove the wxPython as being a block when wxWidgets3.1 package version is updated.

It looks like it could be a lot of work on both wxPython and Kicad packages; but, I do not wish to start effort till someone says the changes are likely to be accepted.

Tim S.

[jeremyd2019]

wxPython depends on wxWidgets, it's a wrapper of it. Are you talking about building wxPython's vendored wxWidgets copy and packaging that into wxPython? That would remove the block, but doesn't seem like a very good idea to have two copies of wxWidgets3.1 running around to worry about maintaining/patching.

[stahta01]

Yes, I am talking about using the wxWidgets version that wxPython uses in the "ext" folder of the tar file. So, there will be two version of wxWidgets the one inside wxPython and the one in the wxWidgets package(s).

So, you raised the first serious negative to the suggestion I posted.

A positive there would likely be a slight decrease in run-time problems chance of happening because of the miss match between wxPython internal wxWidgets and MSys2 system wxWidgets source code being a tiny bit difference.

Tim S.

[lazka]

Would that even work, having two different wxwidget builds in the same process?

What's the underlying problem here that you are trying to solve?

[stahta01]

Would that even work, having two different wxwidget builds in the same process?

What's the underlying problem here that you are trying to solve?

We currently have two different wxWidgets in the wxPython build progress; my change will be to have only one wxWidgets version in the wxPython package build. But, if I succeed we will have two different sets wxWidgets binaries libraries instead of one set. The wxPython set of binaries will have to have something different about them to prevent conflicts with the ones from the wxWidgets package.

Edit: Main problem prevent wxPython being a blocker from wxWidgets version upgrade. Very minor Secondary problem; prevent version miss match of wxPython wxWidgets and system wxWidgets causing low risk run-time problems. Great version miss match seems to cause compile time problems; but, it is possible, I have no idea how likely, that minor changes can cause run-time issues. Since, I do not use KiCAD or wxPython I would not likely see them. I hope to start using KiCAD in the near future; but, since KiCAD is reducing MSys2 build support maybe I will not be using the KiCAD msys2 package.

Tim S.

[jeremyd2019]

Would that even work, having two different wxwidget builds in the same process?

I am not familiar with kicad, but assuming it's a C++ wxWidgets program that exposes wxWidgets objects to python extensions via wxPython, I expect having wxPython using a different version of wxWidgets to break pretty badly.

If it managed to keep the extensions out of process, and serialized OS-level window handles (HWND on Windows) to the extension process, it might just work.

[jeremyd2019]

What's the underlying problem here that you are trying to solve?

I believe the underlying problem that we can't update our wxWidgets3.1 package until we get a version of wxPython that supports that version of wxWidgets3.1

[lazka]

Would packaging/upgrading to a development version of wxpython help?

[jeremyd2019]

No. They haven't added support for 3.1.6 yet upstream, even in git. (upstream development has slowed the last couple of years, to the point that the next release appears to be code-named "Rumors of my death are only slightly exaggerated")

[stahta01]

wxPython is developed using a version of wxWidgets that is frozen in the wxPython repo and updated rarely. If the msys2 system wxWidgets package is a large amount difference from the wxPython frozen wxWidgets version you get build errors. If it is close enough you do not get build errors; if it builds okay you can get run-time issues but, I have no idea if this really is likely to happen; but, the wxPython people warn of it.

Tim S.

[jeremyd2019]

They may warn of it, but I think every Linux distro that packages wxPython de-vendors it anyway :wink:

[stahta01]

They may warn of it, but I think every Linux distro that packages wxPython de-vendors it anyway 😉

Which Linux distro do you know of that uses wxWidgets 3.1 base build of wxPython?

One or two is enough, I am trying find package/build directions to help understand the wxPython build options.

I have decided to work on building wxPython using an Msys2 command line without using an MSys2 package because that looks like easier to find directions.

Looks like the feedback is currently undecided with a slight lean to no to my idea.

Tim S.

[lazka]

It looks like Kicad could maybe be built without Python support, so if this all holds back wxwidgets we could think about dropping wxpython.

[jeremyd2019]

• https://github.com/archlinux/svntogit-community/blob/d540aa532235cb81637ca8c7e4b0b46d1c0d43ea/trunk/PKGBUILD#L30
• https://salsa.debian.org/freewx-team/wxpython4.0/-/blob/master/debian/rules#L8
• https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/wxpython/wxpython-4.0.7-r1.ebuild#n79

Oh, I guess they are all using wxWidgets 3.0 still (since I think 3.1 is considered a development release?)

[stahta01]

Last time I suggested that the answer was no to dropping the wxPython support; and, since KiCAD is reducing support of MSys2 build in version 6.0.x I think it would not be worth doing at this time.

Tim S.

[stahta01]

• https://github.com/archlinux/svntogit-community/blob/d540aa532235cb81637ca8c7e4b0b46d1c0d43ea/trunk/PKGBUILD#L30
• https://salsa.debian.org/freewx-team/wxpython4.0/-/blob/master/debian/rules#L8
• https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/wxpython/wxpython-4.0.7-r1.ebuild#n79

Oh, I guess they are all using wxWidgets 3.0 still (since I think 3.1 is considered a development release?)

Yep, all the ones I have found have used wxWidgets 3.0, too.

Tim S.

[stahta01]

We could also see if building the wxPython package against wxWidgets 3.0 is a good way to get rid of the wxWidgets 3.1 package version [upgrade blocking isssue], good or bad idea?

Tim S.

[jeremyd2019]

You would probably have to downgrade to wxPython 4.0.7

[lazka]

fine with me, that would bring us more in line with Linux distros.

[stahta01]

You would probably have to downgrade to wxPython 4.0.7

I figured that might be needed and is a negative; likely also have to downgrade KiCAD to wxWidgets 3.0.5.

Tim S.

[lazka]

Last time I suggested that the answer was no to dropping the wxPython support; and, since KiCAD is reducing support of MSys2 build in version 6.0.x I think it would not be worth doing at this time.

I don't remember what I (??) answered, but I remember wanting to keep providing them with a dev env.. but as you said they have moved to MSVC now and dropped mingw support, so this is less important now.

I figured that might be needed and is a negative; likely also have to downgrade KiCAD to wxWidgets 3.0.5.

fine with me

[jeremyd2019]

fine with me too, with the caveat that it may require care to find what patches need backporting from 4.1.x.

[stahta01]

I will work on building KiCAD using wxPython built using wxWidgets 3.0 version at first I will try building without MSys2 packages because the wxPython directions is best for that. Then, I will work on an wxPython 4.0 MSys2 package using wxWidgets 3.0 package. Likely based on the ArchLinux and the MSys2 current packages. And, after I get KiCAD 5.1.12 to build I will submit that msys2 package; it will likely be a while my internet connection is slow and I have a paying gig right now that will take up time.

I see either using the folder "mingw-w64-wxPython" or using a new folder "mingw-w64-wxPython4.0" as a path for my wxPython change to wxWidgets 3.0 based wxPython.

Any preference?

Tim S.

[jeremyd2019]

fine with me too, with the caveat that it may require care to find what patches need backporting from 4.1.x.

Here are the patches I use on wxPython 4.0.7 (which I contributed and were accepted into wxPython 4.1). The second one is only really relevant when building for python 2.7 (so might reasonably be omitted from a package here).

0001-pywxrc-fix-embedded-binary-resources-for-python3.patch

From b78720badf38829ea9b39339dd7c0bc93852c4ef Mon Sep 17 00:00:00 2001
From: jeremyd2019
Date: Wed, 10 Jun 2020 11:33:55 -0700
Subject: [PATCH 1/2] pywxrc: fix embedded binary resources for python3

Previously, I was getting errors 'No handler found for image type' and 'Unknown image data format' after updating from wxPython 'classic'.
---
 wx/tools/pywxrc.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/wx/tools/pywxrc.py b/wx/tools/pywxrc.py
index da47c9a0..40d963aa 100644
--- a/wx/tools/pywxrc.py
+++ b/wx/tools/pywxrc.py
@@ -216,7 +216,7 @@ def __init_resources():
     __res.Load('%(resourceFilename)s')"""

     FILE_AS_STRING = """\
-    %(filename)s = '''\\
+    %(filename)s = b'''\\
 %(fileData)s'''

 """
@@ -226,7 +226,7 @@ def __init_resources():
 """

     ADD_FILE_TO_MEMFS = """\
-    wx.MemoryFSHandler.AddFile('XRC/%(memoryPath)s/%(filename)s', %(filename)s)
+    wx.MemoryFSHandler.AddFile('XRC/%(memoryPath)s/%(filename)s', memoryview(%(filename)s))
 """

     LOAD_RES_MEMFS = """\
-- 
2.27.0.windows.1

0002-Try-both-old-and-new-buffer-protocols-on-python2.7.patch

From a10e1c0cc9f1bb8504749024b6ab799a03e24d2c Mon Sep 17 00:00:00 2001
From: jeremyd2019
Date: Thu, 2 Jul 2020 15:09:54 -0700
Subject: [PATCH 2/2] Try both old and new buffer protocols on python2.7

While some objects on 2.7 might only implement the old buffer protocol,
some (notably memoryview) only implement the new buffer protocol, so
both need to be tried.
---
 src/wxpybuffer.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/wxpybuffer.h b/src/wxpybuffer.h
index 221a7633..ad464e0f 100644
--- a/src/wxpybuffer.h
+++ b/src/wxpybuffer.h
@@ -31,8 +31,9 @@ public:
         #if PY_MAJOR_VERSION < 3
             // Old buffer protocol
             int rv = PyObject_AsReadBuffer(obj, (const void**)&m_ptr, &m_len);
-            return rv != -1;
-        #else
+            if (rv != -1)
+                return true;
+        #endif
             // New buffer protocol
             Py_buffer view;
             if (PyObject_GetBuffer(obj, &view, PyBUF_SIMPLE) != 0)
@@ -41,7 +42,6 @@ public:
             m_len = view.len;
             PyBuffer_Release(&view);
             return true;
-        #endif
     }

-- 
2.27.0.windows.1

[stahta01]

Thanks for the patch. I am trying to build the wxPython 4.0.x package now.

On, my second failure; neither it nor the first one looks like one your patch would fix; likely I have not made it far enough for your patch to be needed.

Tim S.

[jeremyd2019]

Sorry, my patches shouldn't have any effect on building wxPython, only on projects using pywxrc.

[stahta01]

I have finally gotten wxPython 4.0.7 to build in #11496.

Now, to see if I can build KiCAD using it; be a while till I can try to do that my internet connection is not good enough download the KICAD files will have to go to public library to do the download.

Tim S.

[stahta01]

I tried building KiCad 6 and found a issue with OCE package See PR #11508

KiCAD 6 uses opencascade instead of OCE like KiCAD 5.

Tim S.

[stahta01]

I have decide to update KiCAD to version 5.1.12 before making it use wxPython 4.0.7 and wxWidgets 3.0.5. I have gotten clang64 to build; it required the removal of the internal manifest file and the fonts was off on my monitor at least.

Tim S.

[stahta01]

I just thought of a big minus on moving KiCAD back to wxWidgets 3.0 and that is worse high DPI support for monitors. It might be better to drop wxPython use by KiCAD instead.

Tim S.

[stahta01]

PR upgrading KiCAD to 5.1.12 #11512

[lazka]

It might be better to drop wxPython use by KiCAD instead.

fine with me

[stahta01]

Okay. I will likely do that after I get a windows copy of KiCAD 5 to test by pacman package against.

Since, I am just learning KiCAD I have no ideas are normal warnings/issues with the KICAD project I want to build I am guessing it was made with KiCAD version 3; and, it has issues working with KiCAD 5.

I also found Phoenix snapshot-builds; so, its last work was done 2022-03-08; so, I think waiting a week to see if another snapshoot build is done is worth waiting.

Tim S.

[jeremyd2019]

I have decide to update KiCAD to version 5.1.12 before making it use wxPython 4.0.7 and wxWidgets 3.0.5. I have gotten clang64 to build; it required the removal of the internal manifest file and the fonts was off on my monitor at least.

Perhaps the 'internal manifest file' was telling Windows it was high dpi aware?

[driver1998]

On wxWidgets 3.0.x, the built-in manifest does not announce high dpi aware, you would need to supply your own, like FileZilla did.

Of course one should also test if there are UI bugs in high DPI...

[stahta01]

I have decide, once again, to stop trying to get wxPython to work with wxWidgets 3.1.6. And, I have decided to back port manifest files and code from KICAD version 6. And, I have decided to stay using Open Cascade Community Edition (OCE) instead of going to Open Cascade because Open Cascade does not build for clang64.

My next step, is figuring out how to back port manifest files and code from KICAD version 6 to version 5.1.12.

Tim S

[stahta01]

I have after a few tries gotten the manifest code changes to build; now, I need to test the results and see if it works very well.

Tim S.

[stahta01]

After all my RC file work, I have decided there was very little chance KICAD 5 was using HIgh DPI support of wxWidgets 3.1 and it looks like KiCAD 5.1 was designed for wxWidgets 3.0 and had wxWidgets 3.1 fixes added to it. Therefore, I decided to revert KiCAD 5.1.12 to wxWidgets 3.0. This requires wxPython4.0 to be created to maintain wxPython scripting support. Once I test all 4 packages, I will remove my WIP from the PR.

Possible future changes to KiCAD include:

1. run-time fixes
2. upgrade to wxWidgets 3.2 when released
3. enable High DPI monitor support after wxWidgets 3.2
4. change to opencascade when it builds for clang64
5. upgrade to KiCAD version 6.0.x after switching to opencascade

Tim S.

[stahta01]

I have finished the work so the package can be merged.

Tim S.