CFEP 04: proposed guidelines for X11-based software.

[pkgw]

Pursuant to the discussion in cairo-feedstock#23, here is a CFEP that proposes some guidelines for conda-forge packages that use, or can use, X11.

Here is the rendered Markdown file.

[pkgw]

There were comments in the Cairo PR about X11 being a heavyweight dependency. For some reference, if I do a clean Miniconda3 installation on linux-64, install cairo, and then run conda clean -tipsy, the resulting tree is 191 MB. As mentioned in the CFEP, the X11 libraries would add ~15 MB to that.

[ocefpaf]

I don't think that 15 MB is too much nowadays but in light of https://github.com/conda-forge/ncl-feedstock/pull/7#issuecomment-284169855 do you still need X11? Or maybe we can rely on XQuartz being available in the Travis-CI image from now on?

[pkgw]

Well, XQuartz might be on the Travis image, but if the final build outputs still link against the X11 libraries, users won't be able to run the software unless they've also got XQuartz installed because the libraries won't be available.

[ocefpaf]

users won't be able to run the software unless they've also got XQuartz installed because the libraries won't be available.

I assumed that is XQuartz is in the image we should expect it to be available by default, but I don't know the first thing about OS X. If that is not true then shipping our own X11 is the only way to go.

[jakirkham]

@jjhelmus, @mingwandroid, could you both please take a look at this and share your thoughts when you have a chance?

[jakirkham]

Have you tried to build XQuartz, @pkgw? I know that basically modern versions of Mac OS don't ship with it. So it might be nice if we can supply our own.

[pkgw]

No, I haven't tried. I don't know of any reason why conda-forge couldn't provide it, but it's not needed for the case of programs that need X11 to build but have non-X11 user interfaces (either CLI or Cocoa).

[jakirkham]

I understand XQuartz is not needed to build them, but wouldn't it be needed to run them? Or do all X11 programs you have encountered have some alternative interface?

[pkgw]

@jakirkham Sorry for never responding to your questions from before ... although I have to say that I don't know the answers to them. I'm not an OSX user myself. I will see if I can gather info from some some of my colleagues.

[jakirkham]

Thanks @pkgw. Did any of your colleagues try this out since? If so, it would be good to know what they found.

[jakirkham]

For posterity, @jjhelmus, it might be good to include info about the tests you tried here with tk especially for people who were not present at the meeting today.

Also a follow up question, did you have XQuartz installed when you tried your tests? Guessing it would have been pretty obvious if it was using XQuartz. Just curious if you tested without it.

[jjhelmus]

First let me begin that I'm excited about conda-forge being able to provide X11 on all three platforms. Installing XQuartz on macOS has always been a pain for me and having it available through conda would be amazing.

My main concern with including X11 as a required dependency in core packages, mainly tk is that it could potentially provide problems if package with and without these dependencies are mixed. For example a tk with a dependency on X11 in the same environment with a python which was build with a tk using the system X11 (for example from the defaults channel) or vice-versa may result in a broken Python _tkinter module. On macOS this issue may be more pronounce as I think tk and the _tkinter module are linked against the Cocoa and Carbon libraries and not X11. Testing the various combinations of these setups should be able to detect possible issues.

Another option would be to used conda features to provide two versions of tk, one linked against the system X11 libraries and one with conda provided X11 libraries. I don't know enough about how linkages are done to know if other downstream packages (i.e. python) would need to also be build with and without this feature.

[jakirkham]

Seems like there are no issues with tk and X11 based on the testing in PR ( https://github.com/conda-forge/tk-feedstock/pull/17 ). Is that correct?

[jakirkham]

Are there any other blockers that we are aware of that would cause issues with making use of the X11 packages in the stack or was tk the only one?

cc @conda-forge/core

[pkgw]

As far as I can recall, no concrete issues came up with having tk depend on the X11 stuff — just (reasonable) concerns about embedding it that low in the dependency stack, especially since defaults doesn't go that route.

I'm not aware of any problems that have cropped up for packages depending on the X11 packages, which is not to say that none have occurred. I also don't have a sense of how widely they're used — I get the impression they've crept in as dependencies for various packages here and there. Do we have any tools for analyzing the intra-package dependency graph of the whole project? (Or, in this relatively straightforward case, just grep would do, I expect.)

[mingwandroid]

Why would we want Python to depend upon X11 through TK depending on X11? Why for that matter would we want TK to depend on X11?

IMHO people who want X11 should use XQuartz or their system X11. I only think it a good idea to package this for Windows where there is no 'system' solution.

[jakirkham]

I can't speak to Tk specifically. However I can speak to the general value of using conda packages instead of say the system. Not having explicit conda dependencies means spending a non-trivial amount of time explaining to people that they need to install these system dependencies. Also as users have newer and more varied Linux systems, inevitably our system packages and their system packages don't match up. Thus we spend an inordinate amount of time debugging issues related to this. For this reason alone, I'm eager to see X11 become an explicit dependency of packages that need it.

[mingwandroid]

Thus we spend an inordinate amount of time debugging issues related to this

Please show me some evidence of this. I have spent close to 0 time debugging issues related to this. The occasional comment of "please use dnf/apt-get/pacman to install your system X11" is all.

Where do you draw the line? Do you want to provide an entire Linux distribution, because that's the way providing X11 is going, and then there's binary compatibility with defaults. Please reconsider.

[jakirkham]

Here are two examples of people getting confused by missing these dependencies.

1: https://github.com/conda-forge/qt-feedstock/issues/54 2: https://github.com/conda-forge/vtk-feedstock/issues/21

[pkgw]

I initially prepared X11 packages because the main cluster that I did my work on was running RHEL5, which didn't support XInput2 and therefore couldn't use some of the graphical toolkits I wanted. Large clusters often have older libraries and you can't just sudo your way out of things.

[ocefpaf]

@mingwandroid this is a complex issue that we need to discuss in a meeting*. I completely agree that making the core packages, like python, depend on X11, via tk, is troublesome and have some key consequences to the ecosystem. I do not want to go that route!

With that said I am not against packaging X11 stuff so some non-key packages, mostly scientific software that are at the end point of the ecosystem like ferret, ncl, and others, can use them and ship in a "headless" way. I am sure that @pkgw has a similar use in mind.

With that said I don't know the best course of action when it comes to tk. Having tk depend on X11 will force it at the bottom of the ecosystem and we face the issue @mingwandroid raises.

I am not sure if having a special x11-tk is a good idea as well. That could solve the end point packages I mentioned but than we would face trouble when installing those packages along side python, for example. Not sure if there is a "clean" solution to that.

* I added it to the discussion topics a few weeks ago but dropped b/c I would like for @pkgw to the present.

PS: issues conda-forge/qt-feedstock#54 and conda-forge/vtk-feedstock#21 are mostly due to disinformation and not the underlying packages or the use of yum. Please let's not diverge from the topic here.

[pkgw]

@ocefpaf Sorry, real life / day job has prevented me from being very plugged in to things here. Is there a date/time for the next meeting?

[ocefpaf]

@ocefpaf Sorry, real life / day job has prevented me from being very plugged in to things here.

Don't worry. Not your fault.

Is there a date/time for the next meeting?

No but as soon as we have I'll ping you and add this topic again.

[pkgw]

@ocefpaf Yes, please give me an explicit heads-up on here or email. Thanks.

[jakirkham]

Would like us to revisit this proposal. We have seen increasingly consumers of our packages generally expect X11 packages to be installed as conda packages and are frequently surprised that we expect them to install these with a system package manager. It's not uncommon to see comments of workarounds in issues suggesting that users install the X11 conda packages to solve the problem. Similarly maintainers are increasingly comfortable using X11 conda packages (at least where Linux is concerned). A few examples are linked below.

xref: https://github.com/conda-forge/python-graphviz-feedstock/issues/15 xref: https://github.com/conda-forge/graphviz-feedstock/issues/18 xref: https://github.com/conda-forge/cairo-feedstock/pull/32 xref: https://github.com/conda-forge/pyngl-feedstock/pull/3 xref: https://github.com/conda-forge/opencv-feedstock/issues/97

[msarahan]

The best solution here would be to require some metapackage representing X11. It could be fulfilled either with conda packages or with system packages. Conda would need to learn to introspect the system a bit to understand which is the correct one.

Depending strictly on conda X11 packages or system-installed packages specifically is going to be painful either way.

[pkgw]

Depending strictly on conda X11 packages or system-installed packages specifically is going to be painful either way.

This might depend on your definition of "painful". X11 is stagnant these days and the X.org libraries are the de facto standard implementations, so relying strictly on conda libraries to implement the X11 protocols just doesn't seem like a big deal to me, from the standpoint of software robustness. (Obviously I'm not an Anaconda support person, but I don't think I've ever seen a problem that was traced back to conda-forge's X11 packages?)

In terms of bloat, that's a different question (although one that I also tend to be sanguine about).

[msarahan]

@mingwandroid's concern has always been about system compatibility that would be broken or sub-optimal with packaged X11. Most especially, he is concerned with hardware acceleration. I don't see that you're taking his concerns into account.

Using a non-hardware accelerated 3D rendering application is painful. Can you definitively prove that your X11 packages work with speed comparable to system packages? Across all packages in the ecosystem? Do you really even want to think about making that proof? Why not leave people with the option, so you can sidestep that proof and leave it up to the user to troubleshoot?

[pkgw]

@msarahan That's a good point. I haven't investigated hardware accel.

[mingwandroid]

There's also an amount of "this is how it was done before and it's worked well" in the decision to not provide our own X11.

[jakirkham]

Do you know of a good benchmark or benchmarks for your hardware acceleration concerns, @mingwandroid?

[mingwandroid]

glxgears? Not really. I never had time to do any analysis of this.

[asmeurer]

We've also included X11 in the emacs feedstock https://github.com/conda-forge/emacs-feedstock/blob/master/recipe/meta.yaml. It's really convenient that you can install emacs on a headless box, which you might not have root access to install the X11 stuff in. The problem is that when emacs is built with GUI mode enabled, it can't launch at all without the X11 libraries, even if you run it with -nw (https://github.com/conda-forge/emacs-feedstock/issues/11). So I think the alternative would have to be to have a separate --without-x build and have conda somehow install that when libXaw etc. aren't found (what ever happened with the talks on conda tracking system packages?). I don't use Linux myself but I know others do use the package, and I haven't heard any complaints regarding the Xorg libraries.

[mingwandroid]

@asmeurer, this isn't about whether things should link to X11 or not, although we don't provide X11 on defaults we still provide packages that link to your system's X11. For headless scenarios you can still install system X11 libs on them in general and use e.g. xvfb-run when testing etc.

From our perspective this difference should be abstracted so the same recipe can use either CDT X11 packages at build time or conda-forge X11 packages at runtime.

[asmeurer]

although we don't provide X11 on defaults we still provide packages that link to your system's X11.

The conda-forge policy doesn't have to match the defaults policy.

For headless scenarios you can still install system X11 libs on them in general

Not if you don't have sudo access. You could say that you could install them with conda, but then you might as well just depend on the conda packages. The user experience of "if you get error while loading shared libraries then install some package Z" is very bad.

The ideal scenario would be an x11 metapackage that installs nothing if the libx* libraries are installed system-wide and installs the xorg packages they aren't. Is something like this currently possible with conda? Last I heard it isn't but is in the works. Until it is my opinion is that conda-forge packages that depend on x libraries should depend on the respective conda-forge packages. The user experience tradeoff is on the one hand, some users will see an error and have to install a package (either via system or conda) to fix it, and on the other, ( as I understand it) there may be a degradation of performance from the conda-forge xorg packages being used instead of hardware accelerated system ones (is this a known issue for some package or a theoretical one?). IMO it's a better tradeoff to avoid the dylib error. Maybe as a stop-gap empty xorg packages could be put in a channel for people to install in case they want to use the system ones.

[mingwandroid]

The conda-forge policy doesn't have to match the defaults policy

Indeed. conda-forge are obviously free to decide what to do here.

The ideal scenario would be an x11 metapackage that installs nothing if the libx* libraries are installed system-wide and installs the xorg packages they aren't.

No, there's not really any way to do this at present.

is this a known issue for some package or a theoretical one?

It needs investigation, but I think the conda-forge X11 stack will not be accelerated. Someone needs to run lsof while running a graphically intensive X11 app with nvidia or amd drivers installed and see if they turn up or not.

[mingwandroid]

The ideal scenario would be an x11 metapackage that installs nothing if the libx* libraries are installed system-wide and installs the xorg packages they aren'

To build a something against an X11 stack such that it'll work on CentOS6 or above as well as with conda-forge libs you'd need to actually link to the CentOS6 CDT packages to prevent newer symbols getting used. Is that something worth considering?

[jjhelmus]

This CFEP could use a champion to continue to examine it. There has been some discussion on this topic in the context recommending matplotlib vs the matplotlib-base package.

One question that should be answered on this topic in addition to hardware accelerate mentioned above is how to build tk with the conda provided X11 libraries. In conda-forge/tk-feedstock#17 @pkgw mentioned that this might be difficult and/or unsafe.

[pkgw]

Just to be explicit, I don't have the time to push on this topic anymore, so I'm not going to be the one to champion this. I'm more than happy for anyone who's able to work on this to do so and take it in whatever direction seems best given where things are now.

As for tk, it is perfectly fine and easy to build it against conda-provided X11 libraries. Doing so just introduces an X11 dependency to the tk package, which would mean pulling in conda X11 on every Mac and Linux install. What would not be safe, I decided, would be pulling a hack to try to get tk to depend on the Conda X11 headers but not libraries.

[ryanvolz]

Not to necessarily re-invigorate the entirety of this CFEP, but the @conda-forge/gtk3 team has a desire to move from using X11 CDTs to the conda-forge Xorg packages for the gtk3 stack and would appreciate some guidance. This would harmonize gtk3 with the new-ish gtk4 package, which went in with the Xorg package dependencies from the start. It is also in response to periodic reports from downstream consumers who are tripped up by needing to add the X11 CDTs to their own recipe because gtk3 depends on them, e.g. https://github.com/conda-forge/gtk3-feedstock/issues/7 and https://github.com/conda-forge/gtk3-feedstock/issues/16.

Though I initially thought that dealing with that issue was not a high enough cost to warrant switching away from the X11 CDTs, I have now come to think that we are unnecessarily burdening other conda-forge contributors with that requirement. This is partially in light of the fact that, though the gtk3 recipe depends on the X11 CDTs, in actual fact we have been using xorg-libx11 and a few others all along because they are pulled in by gtk3's dependency on cairo, which includes the Xorg packages. So this puts us in a situation where we're stuck between the two approaches and not getting the full benefit of either. This is where global guidance from this CFEP would be helpful, but in lieu of resolving that whole discussion, some feedback on the current case with gtk3 would be helpful.

@pkgw brought up two points in particular in https://github.com/conda-forge/gtk3-feedstock/pull/35 for why sticking with the X11 CDTs might still warrant consideration:

1) "Maintain compatibility with the Anaconda recipes." 2) "Concerns about interfaces with system libraries farther down the stack, namely the OpenGL layer. The worry was that maybe custom conda-forge X11 libraries would be stuck using software GL instead of the best, potentially vendor-specific, library for the host machine."

I don't know how much weight the first point still carries. Guidance here in particular would be much appreciated.

As for the second point, I think this is already resolved by the current state of things. Namely, gtk3 supports OpenGL through its epoxy dependency, which has been and will remain dependent on the OpenGL CDTs, which should allow for hardware GL acceleration on users' systems. Couple that with the fact that we have already been, in effect, using the conda-forge Xorg packages with gtk3 through the cairo dependency and this has not raised any issues in ~1 year, and I think that having gtk3 itself explicitly depend on the Xorg packages would not change the current state of things with respect to GL support.

Are there any other concerns that we have missed? Are there objections to merging https://github.com/conda-forge/gtk3-feedstock/pull/35? Thanks!

[beckermr]

merging as deferred