conda_build_config targets an unsupported OSX version

rggjan commented 4 years ago

According to: https://github.com/conda-forge/qt-feedstock/blob/master/recipe/conda_build_config.yaml

Qts lowest supported target platform is 10.13. However, this feedstock currently targets 10.12, 10.12 and even 10.9. We should increase this to 10.13 to make sure there are no unintended consequences in cases where the Qt API is trying to use 10.13 API's.

rggjan commented 4 years ago

This would also make some of the patches unnecessary, like:

ipatch commented 4 years ago

when you say Qts?

what version of Qt are you referring to, ie. major.minor.build ?

and i believe apple allows you to download SDKs independently of the current macOS version you're running so when such issue arises you have the SDK to build against, ie. build time dep. however runtime deps are a separate issue.

I've been reading up on how to setup a conda forge recipe, specifically qt ≥ 5.15 so i could experiment with client side decorators, but haven't done much more than getting conda setup and bookmarking a few links.

rggjan commented 4 years ago

On Qt 5.15's docs it says target platform is at least 10.13, while the only supported build environment is Xcode 11 (10.15 SDK).

This means you have to build against the newest Xcode, but can choose to run on 10.13 still, if desired...

mingwandroid commented 4 years ago

This means you have to build against the newest Xcode, but can choose to run on 10.13 still, if desired...

Whether that actually works or not is a different matter entirely. I want people to not have to update their macOS if that is their choice or something they are unable to do (due to Apple's deprecation policies). I will always try to patch away anything I consider unnecessary in support of this end goal.

rggjan commented 4 years ago

This is not about Apple, but about Qt only supporting OSX 10.13. Apple lets you still build for OSX 10.9, if you so desire. I don't know why Qt raised it's minimum supported OSX version from 10.12 (in Qt 5.12) to 10.13 now, or how much effort it is to patch it to work properly on 10.12 or even 10.9.

I only want to make sure Qt works properly at least on the officially supported versions (10.13, 10.14, 10.15), and isn't broken on all version due to conda-forge building it with an unsupported compiler toolchain...

mingwandroid commented 4 years ago

I don't know why Qt raised it's minimum supported OSX version from 10.12 (in Qt 5.12) to 10.13 now, or how much effort it is to patch it to work properly on 10.12 or even 10

I do know why each macOS SDK update was forced upon Qt. By and large all driven by QtWebEngine, due to chromium being obsessed by shiny new things.

Can you explain in what way it is broken and how fixing this but denying people on older OSes the chance to use Qt at all is better?

mingwandroid commented 4 years ago

You can study each of the patches if you want to better understand the intricities of this.

isuruf commented 4 years ago

Since we are not building 5.15 yet, and only building 5.12 in this feedstock, 10.12 is the correct value

mingwandroid commented 4 years ago

and isn't broken on all version due to conda-forge building it with an unsupported compiler toolchain...

Please, drop the hyperbole. Clang works well on macOS as do our compilers.

mingwandroid commented 4 years ago

We build everything with an "unsupported compiler toolchain" by your definition.

mingwandroid commented 4 years ago

Since we are not building 5.15 yet

Real soon now I hope. 10.14 is the new upstream minimum. I will do what I can to relax this.

rggjan commented 4 years ago

We build everything with an "unsupported compiler toolchain" by your definition.

Sorry, I mainly meant the SDK and not the actual compiler here.

rggjan commented 4 years ago

Since we are not building 5.15 yet, and only building 5.12 in this feedstock, 10.12 is the correct value

To be honest I didn't realize this. Of course 10.12 is perfectly fine for building Qt 5.15. What relevance does the 10.9 here have though ... ? (I'm not very familiar with the conda build system)

isuruf commented 4 years ago

What relevance does the 10.9 here have though ... ?

Nothing. It's not used.

mingwandroid commented 4 years ago

@rggjan, can you please identify the specific issues and limitations you have run into due to us relaxing the macOS operating system requirements? A bug report for each (that to me would be purely informative regarding the choices we've made here) would be appreciated!

rggjan commented 4 years ago

I do know why each macOS SDK update was forced upon Qt. By and large all driven by QtWebEngine, due to chromium being obsessed by shiny new things.

Can you explain in what way it is broken and how fixing this but denying people on older OSes the chance to use Qt at all is better?

No I can't. As I said, I just saw by chance that Qt officially doesn't support OSX <= 10.12 anymore, and wanted to make sure that on conda, this doesn't lead to problems. I assumed there was a reason support for 10.12 was dropped from Qt (and they didn't just do it to annoy people), and was afraid that building it with an unsupported SDK might lead to subtle bugs. So in my mind:

Working Qt on new OSX, non-working Qt on old OSX was better than
(Potentially) broken Qt on new and old OSX

rggjan commented 4 years ago

What relevance does the 10.9 here have though ... ?

Nothing. It's not used.

I see, thanks. Is it possible to remove it then to not confuse other people like me in the future, or will this break the build process somehow?

isuruf commented 4 years ago

Is it possible to remove it then to not confuse other people like me in the future, or will this break the build process somehow?

You'll have to look into recipes using it and fix those. And also fix conda-smithy.

rggjan commented 4 years ago

@rggjan, can you please identify the specific issues and limitations you have run into due to us relaxing the macOS operating system requirements? A bug report for each (that to me would be purely informative regarding the choices we've made here) would be appreciated!

@mingwandroid I'll do that if I find anything. I appreciate your efforts a lot to make it work on older OSX versions, don't get me wrong. Just wanted to avoid people (potentially) debugging weird Qt issues for days (as I did in https://github.com/conda-forge/pyside2-feedstock/issues/46) arising from compiling Qt with an unsupported environment. If you think it's better to keep it running on old versions and fix issues only when they pop up, that's OK for me and we can close this issue.

rggjan commented 4 years ago

Is it possible to remove it then to not confuse other people like me in the future, or will this break the build process somehow?

You'll have to look into recipes using it and fix those. And also fix conda-smithy.

Oh, I see... not a quick fix then :/

let's keep it as is then for now...

mingwandroid commented 4 years ago

Thanks @rggjan, you should appreciate that someone (me) crafted each of those patches to enable our software to have a wider reach and saying "Oh we can update and discard all that stuff" is somewhat offensive. I didn't create them for fun and clearly would prefer not to have to carry them but the world is as it is.

To me this is akin to walking into a dark movie theatre shouting "Fire!" You are trying to avoid problems you've invented and then allude to these vague problems a few times.

Given how C linking works, the software either builds or it does not. If it builds because we've removed some features that are not supported then we patch around any issues those cause. That is the point of those patches.

If people report issues we investigate them and fix them.

mingwandroid commented 4 years ago

And yes, most of these new SDK-requiring updates are in support of new OS fluff and their removal does not negatively impact most Data Science uses, and no, passing a lower SDK target will not work. Patching is necessary. On balance I think it's the right thing to do.

mingwandroid commented 4 years ago

The two things that we must take very good care to avoid here if possible, is someone:

Updating their conda env, getting a new qt and not being able to run their software
Being stuck in a position where because they have Qt installed on an old macOS, the conda solver refuses to update anything else in their environment due to shared inter-dependencies on this old (but still compatible) Qt.

Both are disasters.

rggjan commented 4 years ago

Given how C linking works, the software either builds or it does not. If it builds because we've removed some features that are not supported then we patch around any issues those cause. That is the point of those patches.

Unfortunately, that wasn't the case in https://github.com/conda-forge/pyside2-feedstock/issues/46, where everything built fine and starting a hello world Qt app worked without problems, but using more complex Qt functionality stuff just wasn't working properly.

Debugging and figuring out the issue took me a lot of time, and my only intention was to prevent others from the same fate...

rggjan commented 4 years ago

And yes, most of these new SDK-requiring updates are in support of new OS fluff and their removal does not negatively impact most Data Science uses, and no, passing a lower SDK target will not work. Patching is necessary. On balance I think it's the right thing to do.

Closing the issue then...

mingwandroid commented 4 years ago

In that issue, you do not seem to reach any solid technical conclusions and are again rather offensive regarding my work.

rggjan commented 4 years ago

Sincere apologies if I seemed offensive, that was never the intention. After spending many hours debugging conda-forge/pyside2-feedstock#46, I came to the conclusion that:

using PySide2 on conda-forge is broken on OSX
the issue is fixed when linking python against a newer OSX SDK version

It seemed to me that making PySide2 usable on OSX would be a good thing even without understanding in detail why it's broken when linking to older SDKs (which is unsupported according to Qt docs). Again, didn't mean to offend or not value your work...

mingwandroid commented 4 years ago

but using more complex Qt functionality stuff just wasn't working properly.

File specific bugs with reproduction cases.

using PySide2 on conda-forge is broken on OSX the issue is fixed when linking python against a newer OSX SDK version

I need minimal testcases for these. Please prepare them and I will give this my attention.

Of course the obvious elephant in the room is PySide2 which I've never looked at or built. It's entirely possible (no extremely likely) that all of these problems lie there. Casting shade on the Qt libraries because of this very likely completely unrelated issue seems entirely incorrect in this instance. You are aware that all of the libraries in AD and CF are built against old macOS SDKs? And we do not hunt around on the internet for upstream's documented minimum SDK support level because by-and-large things work. And I defend patching away pure fluff (and dark mode is that) in the goal of broader reach.

If it puts your mind at ease I build and test an absolute tonne of software with each and every Qt release for AD on all 3 OSes. These tests include Qt3dStudio, QtCreator, Spyder, PyQt and Anaconda Navigator and I do not propose recipes to conda-forge that I know to be broken without disclosing that.

rggjan commented 4 years ago

I need minimal testcases for these. Please prepare them and I will give this my attention.

@psavery summarized the test case pretty well here: https://github.com/conda-forge/python-feedstock/pull/371#issuecomment-667196573

While there are many more issues in the more complex project I'm working on (like sliders and text fields not updating, signal/slot connections not firing, ...), this is a minimal example I managed to come up with that shows the issue and demonstrates how the proposed solution fixes it.

Let me know if this is not good enough for some reason, and I can try to come up with more examples, but with the tests I did, once this issue was fixed, all the other issues in my project also went away.

rggjan commented 4 years ago

If it puts your mind at ease I build and test an absolute tonne of software with each and every Qt release for AD on all 3 OSes. These tests include Qt3dStudio, QtCreator, Spyder, PyQt and Anaconda Navigator and I do not propose recipes to conda-forge that I know to be broken without disclosing that.

That does sound pretty extensive and like a lot of work. I really do wonder how this issue only seems to be affecting PySide2...

mingwandroid commented 4 years ago

My guess is this might be due to integrated graphics drivers being a bit dodgy. This might help: https://github.com/conda-forge/python.app-feedstock/pull/14

ipatch commented 4 years ago

wasn't trying to open pandora's box with my initial comment on this issue.

and def not trying to pour salt on a wound here, but @rggjan one thing i picked up on in the link you posted above there is a note in the Qt doc stating Xcode 9 could be used to build against the 10.13 SDK so i don't think having a minimum build env of Xcode 11 running on macOS 10.15 is the only way to build Qt.

For my personal use case, I would like to have a version of Qt ≥ 5.15 so i can experiment with client side decorators as stated above. and my personal setup is a macbook running macos 10.13, and i have no hurry to upgrade to a newer version of macos. i only upgraded to 10.13 because it was the only way i could use a required piece of new hardware, otherwise i would have stayed on 10.12.

Apple seems to roll out new major OS upgrades quite aggressively, ≈ 1 per year so they tend to only support the three latest versions (which if you go to try and see if that prior statement is gospel, you won't, you have to read between the lines when looking at this, as this 3rd party observation suggests) which happen to be 10.13 10.14 10.15, that said though, i would bet if a major security flaw is found, and is present in all major OSes 10.12 included Apple would ship an update for 10.12

But looking at https://support.apple.com/en-us/HT201222 there hasn't been a posted update for 10.12 since 26 Sep 2019 and there hasn't been an update for 10.11 since 2018. So with all that said, it makes sense why Qt would set the minimum supported macOS to 10.13, which at this point if it becomes more work to support 10.12 I'd drop it, that said if 10.12 still works keep it around, but don't go out of your way to keep supporting. I have no idea what the status of analytics gathering this project supports, but if there is any it could be behove the maintainer to look at the numbers to see how many people are using 10.12 to see if it's still worth supporting, and if not, display a message through conda of some sort stating, "10.12 support is going to be dropped sometime within the future, so plan on your upgrade solution, etc etc."

mingwandroid commented 4 years ago

I will always pull Qt compatibility back as far as I can but it's a judgement call when I start dropping features.

so i don't think having a minimum build env of Xcode 11 running on macOS 10.15 is the only way to build Qt

Please, this is software, we can and do patch things that are not considered ideal for the Anconda Distro and conda-forge ecosystems. Even if upstream says "you must use X", they are only referring to a truth about the software as they publish it. We are free to modify that software. Looking around for these compatability clues for problems is not necessarily useful IMHO.

rggjan commented 4 years ago

My guess is this might be due to integrated graphics drivers being a bit dodgy. This might help: conda-forge/python.app-feedstock#14

Thanks for the link, I'd be happy to help testing if this fixes the issue as well (while keeping the old SDK). What would I need to do?

conda-forge / qt-feedstock

conda_build_config targets an unsupported OSX version #159