msys2 / MINGW-packages

Package scripts for MinGW-w64 targets to build under MSYS2.
https://packages.msys2.org
BSD 3-Clause "New" or "Revised" License
2.28k stars 1.21k forks source link

wxPython and Kicad packages request for comment #11424

Closed stahta01 closed 2 years ago

stahta01 commented 2 years ago

Kicad depends on wxPython and wxWidgets3.1 packages.

And, wxPython depends on wxWidgets3.1 package.

I was thinking that removing the depends on the wxWidgets3.1 package would remove the wxPython as being a block when wxWidgets3.1 package version is updated.

It looks like it could be a lot of work on both wxPython and Kicad packages; but, I do not wish to start effort till someone says the changes are likely to be accepted.

Tim S.

jeremyd2019 commented 2 years ago

wxPython depends on wxWidgets, it's a wrapper of it. Are you talking about building wxPython's vendored wxWidgets copy and packaging that into wxPython? That would remove the block, but doesn't seem like a very good idea to have two copies of wxWidgets3.1 running around to worry about maintaining/patching.

stahta01 commented 2 years ago

Yes, I am talking about using the wxWidgets version that wxPython uses in the "ext" folder of the tar file. So, there will be two version of wxWidgets the one inside wxPython and the one in the wxWidgets package(s).

So, you raised the first serious negative to the suggestion I posted.

A positive there would likely be a slight decrease in run-time problems chance of happening because of the miss match between wxPython internal wxWidgets and MSys2 system wxWidgets source code being a tiny bit difference.

Tim S.

lazka commented 2 years ago

Would that even work, having two different wxwidget builds in the same process?

What's the underlying problem here that you are trying to solve?

stahta01 commented 2 years ago

Would that even work, having two different wxwidget builds in the same process?

What's the underlying problem here that you are trying to solve?

We currently have two different wxWidgets in the wxPython build progress; my change will be to have only one wxWidgets version in the wxPython package build. But, if I succeed we will have two different sets wxWidgets binaries libraries instead of one set. The wxPython set of binaries will have to have something different about them to prevent conflicts with the ones from the wxWidgets package.

Edit: Main problem prevent wxPython being a blocker from wxWidgets version upgrade. Very minor Secondary problem; prevent version miss match of wxPython wxWidgets and system wxWidgets causing low risk run-time problems. Great version miss match seems to cause compile time problems; but, it is possible, I have no idea how likely, that minor changes can cause run-time issues. Since, I do not use KiCAD or wxPython I would not likely see them. I hope to start using KiCAD in the near future; but, since KiCAD is reducing MSys2 build support maybe I will not be using the KiCAD msys2 package.

Tim S.

jeremyd2019 commented 2 years ago

Would that even work, having two different wxwidget builds in the same process?

I am not familiar with kicad, but assuming it's a C++ wxWidgets program that exposes wxWidgets objects to python extensions via wxPython, I expect having wxPython using a different version of wxWidgets to break pretty badly.

If it managed to keep the extensions out of process, and serialized OS-level window handles (HWND on Windows) to the extension process, it might just work.

jeremyd2019 commented 2 years ago

What's the underlying problem here that you are trying to solve?

I believe the underlying problem that we can't update our wxWidgets3.1 package until we get a version of wxPython that supports that version of wxWidgets3.1

lazka commented 2 years ago

Would packaging/upgrading to a development version of wxpython help?

jeremyd2019 commented 2 years ago

No. They haven't added support for 3.1.6 yet upstream, even in git. (upstream development has slowed the last couple of years, to the point that the next release appears to be code-named "Rumors of my death are only slightly exaggerated")

stahta01 commented 2 years ago

wxPython is developed using a version of wxWidgets that is frozen in the wxPython repo and updated rarely. If the msys2 system wxWidgets package is a large amount difference from the wxPython frozen wxWidgets version you get build errors. If it is close enough you do not get build errors; if it builds okay you can get run-time issues but, I have no idea if this really is likely to happen; but, the wxPython people warn of it.

Tim S.

jeremyd2019 commented 2 years ago

They may warn of it, but I think every Linux distro that packages wxPython de-vendors it anyway :wink:

stahta01 commented 2 years ago

They may warn of it, but I think every Linux distro that packages wxPython de-vendors it anyway 😉

Which Linux distro do you know of that uses wxWidgets 3.1 base build of wxPython?

One or two is enough, I am trying find package/build directions to help understand the wxPython build options.

I have decided to work on building wxPython using an Msys2 command line without using an MSys2 package because that looks like easier to find directions.

Looks like the feedback is currently undecided with a slight lean to no to my idea.

Tim S.

lazka commented 2 years ago

It looks like Kicad could maybe be built without Python support, so if this all holds back wxwidgets we could think about dropping wxpython.

jeremyd2019 commented 2 years ago

Oh, I guess they are all using wxWidgets 3.0 still (since I think 3.1 is considered a development release?)

stahta01 commented 2 years ago

Last time I suggested that the answer was no to dropping the wxPython support; and, since KiCAD is reducing support of MSys2 build in version 6.0.x I think it would not be worth doing at this time.

Tim S.

stahta01 commented 2 years ago

Oh, I guess they are all using wxWidgets 3.0 still (since I think 3.1 is considered a development release?)

Yep, all the ones I have found have used wxWidgets 3.0, too.

Tim S.

stahta01 commented 2 years ago

We could also see if building the wxPython package against wxWidgets 3.0 is a good way to get rid of the wxWidgets 3.1 package version [upgrade blocking isssue], good or bad idea?

Tim S.

jeremyd2019 commented 2 years ago

You would probably have to downgrade to wxPython 4.0.7

lazka commented 2 years ago

fine with me, that would bring us more in line with Linux distros.

stahta01 commented 2 years ago

You would probably have to downgrade to wxPython 4.0.7

I figured that might be needed and is a negative; likely also have to downgrade KiCAD to wxWidgets 3.0.5.

Tim S.

lazka commented 2 years ago

Last time I suggested that the answer was no to dropping the wxPython support; and, since KiCAD is reducing support of MSys2 build in version 6.0.x I think it would not be worth doing at this time.

I don't remember what I (??) answered, but I remember wanting to keep providing them with a dev env.. but as you said they have moved to MSVC now and dropped mingw support, so this is less important now.

I figured that might be needed and is a negative; likely also have to downgrade KiCAD to wxWidgets 3.0.5.

fine with me

jeremyd2019 commented 2 years ago

fine with me too, with the caveat that it may require care to find what patches need backporting from 4.1.x.

stahta01 commented 2 years ago

I will work on building KiCAD using wxPython built using wxWidgets 3.0 version at first I will try building without MSys2 packages because the wxPython directions is best for that. Then, I will work on an wxPython 4.0 MSys2 package using wxWidgets 3.0 package. Likely based on the ArchLinux and the MSys2 current packages. And, after I get KiCAD 5.1.12 to build I will submit that msys2 package; it will likely be a while my internet connection is slow and I have a paying gig right now that will take up time.

I see either using the folder "mingw-w64-wxPython" or using a new folder "mingw-w64-wxPython4.0" as a path for my wxPython change to wxWidgets 3.0 based wxPython.

Any preference?

Tim S.

jeremyd2019 commented 2 years ago

fine with me too, with the caveat that it may require care to find what patches need backporting from 4.1.x.

Here are the patches I use on wxPython 4.0.7 (which I contributed and were accepted into wxPython 4.1). The second one is only really relevant when building for python 2.7 (so might reasonably be omitted from a package here).

0001-pywxrc-fix-embedded-binary-resources-for-python3.patch

From b78720badf38829ea9b39339dd7c0bc93852c4ef Mon Sep 17 00:00:00 2001
From: jeremyd2019
Date: Wed, 10 Jun 2020 11:33:55 -0700
Subject: [PATCH 1/2] pywxrc: fix embedded binary resources for python3

Previously, I was getting errors 'No handler found for image type' and 'Unknown image data format' after updating from wxPython 'classic'.
---
 wx/tools/pywxrc.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/wx/tools/pywxrc.py b/wx/tools/pywxrc.py
index da47c9a0..40d963aa 100644
--- a/wx/tools/pywxrc.py
+++ b/wx/tools/pywxrc.py
@@ -216,7 +216,7 @@ def __init_resources():
     __res.Load('%(resourceFilename)s')"""

     FILE_AS_STRING = """\
-    %(filename)s = '''\\
+    %(filename)s = b'''\\
 %(fileData)s'''

 """
@@ -226,7 +226,7 @@ def __init_resources():
 """

     ADD_FILE_TO_MEMFS = """\
-    wx.MemoryFSHandler.AddFile('XRC/%(memoryPath)s/%(filename)s', %(filename)s)
+    wx.MemoryFSHandler.AddFile('XRC/%(memoryPath)s/%(filename)s', memoryview(%(filename)s))
 """

     LOAD_RES_MEMFS = """\
-- 
2.27.0.windows.1

0002-Try-both-old-and-new-buffer-protocols-on-python2.7.patch

From a10e1c0cc9f1bb8504749024b6ab799a03e24d2c Mon Sep 17 00:00:00 2001
From: jeremyd2019
Date: Thu, 2 Jul 2020 15:09:54 -0700
Subject: [PATCH 2/2] Try both old and new buffer protocols on python2.7

While some objects on 2.7 might only implement the old buffer protocol,
some (notably memoryview) only implement the new buffer protocol, so
both need to be tried.
---
 src/wxpybuffer.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/wxpybuffer.h b/src/wxpybuffer.h
index 221a7633..ad464e0f 100644
--- a/src/wxpybuffer.h
+++ b/src/wxpybuffer.h
@@ -31,8 +31,9 @@ public:
         #if PY_MAJOR_VERSION < 3
             // Old buffer protocol
             int rv = PyObject_AsReadBuffer(obj, (const void**)&m_ptr, &m_len);
-            return rv != -1;
-        #else
+            if (rv != -1)
+                return true;
+        #endif
             // New buffer protocol
             Py_buffer view;
             if (PyObject_GetBuffer(obj, &view, PyBUF_SIMPLE) != 0)
@@ -41,7 +42,6 @@ public:
             m_len = view.len;
             PyBuffer_Release(&view);
             return true;
-        #endif
     }

-- 
2.27.0.windows.1
stahta01 commented 2 years ago

Thanks for the patch. I am trying to build the wxPython 4.0.x package now.

On, my second failure; neither it nor the first one looks like one your patch would fix; likely I have not made it far enough for your patch to be needed.

Tim S.

jeremyd2019 commented 2 years ago

Sorry, my patches shouldn't have any effect on building wxPython, only on projects using pywxrc.

stahta01 commented 2 years ago

I have finally gotten wxPython 4.0.7 to build in #11496.

Now, to see if I can build KiCAD using it; be a while till I can try to do that my internet connection is not good enough download the KICAD files will have to go to public library to do the download.

Tim S.

stahta01 commented 2 years ago

I tried building KiCad 6 and found a issue with OCE package See PR #11508

KiCAD 6 uses opencascade instead of OCE like KiCAD 5.

Tim S.

stahta01 commented 2 years ago

I have decide to update KiCAD to version 5.1.12 before making it use wxPython 4.0.7 and wxWidgets 3.0.5. I have gotten clang64 to build; it required the removal of the internal manifest file and the fonts was off on my monitor at least.

Tim S.

stahta01 commented 2 years ago

I just thought of a big minus on moving KiCAD back to wxWidgets 3.0 and that is worse high DPI support for monitors. It might be better to drop wxPython use by KiCAD instead.

Tim S.

stahta01 commented 2 years ago

PR upgrading KiCAD to 5.1.12 #11512

lazka commented 2 years ago

It might be better to drop wxPython use by KiCAD instead.

fine with me

stahta01 commented 2 years ago

Okay. I will likely do that after I get a windows copy of KiCAD 5 to test by pacman package against.

Since, I am just learning KiCAD I have no ideas are normal warnings/issues with the KICAD project I want to build I am guessing it was made with KiCAD version 3; and, it has issues working with KiCAD 5.

I also found Phoenix snapshot-builds; so, its last work was done 2022-03-08; so, I think waiting a week to see if another snapshoot build is done is worth waiting.

Tim S.

jeremyd2019 commented 2 years ago

I have decide to update KiCAD to version 5.1.12 before making it use wxPython 4.0.7 and wxWidgets 3.0.5. I have gotten clang64 to build; it required the removal of the internal manifest file and the fonts was off on my monitor at least.

Perhaps the 'internal manifest file' was telling Windows it was high dpi aware?

driver1998 commented 2 years ago

On wxWidgets 3.0.x, the built-in manifest does not announce high dpi aware, you would need to supply your own, like FileZilla did.

Of course one should also test if there are UI bugs in high DPI...

stahta01 commented 2 years ago

I have decide, once again, to stop trying to get wxPython to work with wxWidgets 3.1.6. And, I have decided to back port manifest files and code from KICAD version 6. And, I have decided to stay using Open Cascade Community Edition (OCE) instead of going to Open Cascade because Open Cascade does not build for clang64.

My next step, is figuring out how to back port manifest files and code from KICAD version 6 to version 5.1.12.

Tim S

stahta01 commented 2 years ago

I have after a few tries gotten the manifest code changes to build; now, I need to test the results and see if it works very well.

Tim S.

stahta01 commented 2 years ago

After all my RC file work, I have decided there was very little chance KICAD 5 was using HIgh DPI support of wxWidgets 3.1 and it looks like KiCAD 5.1 was designed for wxWidgets 3.0 and had wxWidgets 3.1 fixes added to it. Therefore, I decided to revert KiCAD 5.1.12 to wxWidgets 3.0. This requires wxPython4.0 to be created to maintain wxPython scripting support. Once I test all 4 packages, I will remove my WIP from the PR.

Possible future changes to KiCAD include:

  1. run-time fixes
  2. upgrade to wxWidgets 3.2 when released
  3. enable High DPI monitor support after wxWidgets 3.2
  4. change to opencascade when it builds for clang64
  5. upgrade to KiCAD version 6.0.x after switching to opencascade

Tim S.

stahta01 commented 2 years ago

I have finished the work so the package can be merged.

Tim S.