darlinghq / darling

Darwin/macOS emulation layer for Linux
http://www.darlinghq.org
GNU General Public License v3.0
11.32k stars 434 forks source link

Code from Apple under the APSL is incompatible with the GPL #542

Open ryao opened 5 years ago

ryao commented 5 years ago

This project seems to be using code from Apple that is under the APSL. This is known to be incompatible with the GPL:

https://www.gnu.org/philosophy/apsl.html

Both the userland code and kernel code seem to suffer from this problem. To give a specific example, launchd is under the APSL, yet the project itself claims to be under the GPLv3:

https://github.com/darlinghq/darling/blob/master/src/launchd/src/core.c https://github.com/darlinghq/darling/blob/master/LICENSE

The kernel module is mostly APSL code from XNU, but the module itself reports itself to be GPL while containing files that are marked under the GPL:

https://github.com/darlinghq/darling-newlkm/blob/master/darling/module.c https://github.com/darlinghq/darling-newlkm/blob/master/darling/binfmt.c

Would the project please fix this by opting for something compatible with the APSL like the LGPLv2.1 that the wine project uses for nearly all of its code (the imported faudio code is zlib) and that the GNUStep project uses for libraries?

ryao commented 5 years ago

It occurs to me that you could probably make an XNU compatibility layer module under the GPL that does not use GPL exported symbols, mark any symbols that it creates that the XNU code needs under EXPORT_SYMBOL and then use them from the XNU code in a different kernel module. That is how ZFSOnLinux works, which is in a similar position of having a kernel module containing code derived from a different OS kernel under a GPL-incompatible license. It does not mix GPL and GPL incompatible code in the same kernel module.

After having thought about it more, I am still rather concerned about the project describing itself as being GPLv3 when it includes code from Apple under the incompatible APSL. It gives the wrong idea about the licensing. Also, all new code appears to be under the GPLv3, which seems likely to create situations where libraries used by GPL-incompatible code are implemented under the GPL, which would make it impossible for anyone to distribute binaries while satisfying all licenses.

Linux distributions like software that they put into their repositories to be licensed in a way that allows binary redistribution. The things that I mentioned call that into question. They would be enough to keep most Linux distribution developers from even attempting to include this code in their repositories. Just trying to audit it to see if potential conflicts were avoided seems like a nightmare.

ryao commented 5 years ago

I did a bit more research. The LGPL is not compatible with the APSL when including both in a kernel module:

https://tldrlegal.com/license/gnu-lesser-general-public-license-v2.1-(lgpl-2.1)

It might be easiest to just put the non-APSL parts of the kernel module under the 3-clause BSD license (which is used in parts of the Linux kernel), while putting all dynamic library code under the LGPL (like GNUStep and most GNU projects do) and putting any code that goes into statically linked libraries under a permissive license like the 3-clause BSD license. At the very least, that would enable Linux distribution developers to start packaging this for end users.

bugaevc commented 5 years ago

To give a specific example, launchd is under the ASPL, yet the project itself claims to be under the GPLv3

What we claim is

Source code implemented as part of Darling is released under the terms of GNU General Public License version 3. It is available for download on GitHub.

Like OS X, Darling includes MANY third party components. These are typically licensed under APSL, GPLv2, LGPL, BSD and similar licencies.

It is, of course, possible to distribute GPL code with differently licensed code (see GNU-Darwin mentioned at the GNU page on APSL).

CuriousTommy commented 5 years ago

It is, of course, possible to distribute GPL code with differently licensed code

The question I want to ask is how are you able to distribute APSL and GPL together. Under normal circumstances, the APSL is not a GPL-compatible license.

(see GNU-Darwin mentioned at the GNU page on APSL).

So how does GNU-Darwin manage to have GNU and APSL coexist? It that how Darling is also structured?

LubosD commented 5 years ago

Userspace:

If you take most Linux distributions, you'll find packages that have mutually somewhat incompatible licenses. That doesn't mean they cannot co-exist in the same RPM/DEB repository or that you cannot write a Bash script that uses executables from both worlds.

Darling is more than an application, it is kind of a MacOS distribution.

Kernel:

The kernel stuff is divided into 3 categories:

The GPL code can definitely make calls into APSL/BSD code, because APSL/BSD has no restrictions in this manner. There is just the requirement to release changes done to the APSL code, which we comply with (indeed).

I don't think there are any locations where APSL code would make calls to GPL-only kernel exports. If such instances exist, it'd certainly be good to fix them.

Why we use GPL for our stuff:

The reason for using GPL is to make it as hard as possible for someone to build a business on distributing Darling with proprietary improvements. Basically, if someone would choose to do so, he would be barred from making calls into any GPL'd libraries we produce.

It may or may not be easy, depending on the area of such a proprietary improvement.

This is something we haven't thoroughly debated in the team, but if someone wants to add 2% of work on top of our 98% of work and get paid for it without releasing his work, then he needs to cut a deal with us. Or to put it differently: either the code is to be released, or we want to be "reimbursed".

CuriousTommy commented 5 years ago

@LubosD Thanks for clarifying LubosD, I feel more at ease now that you explained the licensing issue.

This is something we haven't thoroughly debated in the team, but if someone wants to add 2% of work on top of our 98% of work and get paid for it without releasing his work, then he needs to cut a deal with us. Or to put it differently: either the code is to be released, or we want to be "reimbursed".

I have a few (dumb) questions about this:

ryao commented 5 years ago

I understand that you want people distributing binaries to release the code, but what you have done makes it questionable whether it is even legal for people to ship binaries. Putting GPL code into the same kernel module as GPL-incompatible code is an issue.

Using the GPL for all code basically runs the risk of creating a GPL library that is used by non-GPL code. Unless you have some kind of exception (e.g. like the implied one from the exported symbols on Linux when porting things from other platforms), that non-GPL code cannot be shipped using the GPL code. This is a concern for completely non-commercial distributions like Debian and Gentoo when it comes to doing packaging (although Gentoo is more lenient here and has ways of marking packages that cannot be redistributed in binary form).

There are multiple ways of doing what you want. The APSL requires code disclosure, as do the LGPL and CDDL to name a few. The GPL is not the only way to do it and it is not always the best either. There is the more extreme AGPL to force disclosure in situations where network services are running on top of the code. However, when combining code under different licenses, you need to consider whether the licenses can simultaneously be satisfied to ensure that people can actually distribute binaries in the first place. Otherwise, you might as well not make it open source in the first place and would be better off with a look, but don’t touch visible source license. It effectively is that if Linux distributions cannot be certain that binaries are able to be distributed.

I am not a lawyer, but I am concerned about whether the darling code as it is licensed could ever be packaged in a Linux distribution like how wine can. The more pieces you get from Mac OS X and the more pieces you write under the GPL to fill in the gaps, the more likely it is that you could create a problem and the more you create a headache for a distribution maintainer in figuring out whether you actually did. There is already a potential problem in the kernel module. It is even worse with the kernel module because the kernel module claims to only be subject to the GPL when in fact, almost none of it is.

I took a quick look at this out of curiosity after hearing about it on Reddit and it raised so many flags to me as a Linux distribution developer that I was discouraged from even trying it out. If I tried it and liked it, I would probably want to package it, but I do not think that I could package it in a way that did not pose problems for potential users. I am sure that every other distribution developer that has looked at this lately has noticed the same issue. We all want end users to be free to redistribute binaries. Requiring that source code be disclosed is fine, but having licenses that prevent binaries from being distributed is not fine. There needs to be some kind of assurance that the licenses don’t conflict. If you adopted the LGPL for library code and handled the kernel code in a way that did not subject it to a license that is incompatible with its license, then that would have been fine.

ryao commented 5 years ago

I did some more reading (as it has been a long time since I felt a need to think about licensing conflicts and I originally had no knowledge about how the APSL interacts with other licenses aside from the remarks from the FSF). My remark earlier about the LGPL and statically linked software is not quite correct. When everything is open source, you can freely statically link LGPL code with code under other licenses. The distinction is that there needs to be a way to modify the LGPL code and if everything is OSS, you can just recompile everything:

https://stackoverflow.com/questions/10130143/gpl-lgpl-and-static-linking#10179181

The confusion earlier was caused by a simplification of how it interacts with proprietary software. There needs to be a way to modify the LGPL code and relink with it and tdlrlegal simplified that to say that static linking was not allowed.

Anyway, you could just put all library code and the kernel code that the project writes under the LGPL, keep everything else new under the GPL (or even make it AGPL) and appropriately mark the kernel module license (e.g. mark it as APSL+LGPL). That would resolve the issues that I spotted.

ryao commented 5 years ago

One more remark. I am not sure if Darling can be called open source software if the licenses conflict in a way that prevents distribution of binaries. That conflicts with the first and second parts of the open source definition:

https://opensource.org/osd

ahyattdev commented 5 years ago

@ryao why does having multiple licenses prevent distribution? Which one of our licenses prevents redistribution?

Where do you see that we claim the entire kernel module is GPL? This is the only 'LICENSE' file in it: https://github.com/darlinghq/darling-newlkm/blob/master/APPLE_LICENSE

The more pieces you get from Mac OS X and the more pieces you write under the GPL to fill in the gaps, the more likely it is that you could create a problem and the more you create a headache for a distribution maintainer in figuring out whether you actually did.

Figure out whether we actually did what? It's not clear what you mean here.

ryao commented 5 years ago

@ahyattdev You document the licenses that apply to the kernel module in MODULE_LICENSE, such that they are reported by modinfo --field license $MODULE. You can't fill that field with GPL by itself without claiming that the entire module is subject to the GPL, which it is not.

If you have multiple licenses and code gets combined in a way that you cannot satisfy both while distributing binaries, then you cannot distribute binaries. GPL code doesn't play nicely with non-GPL code in general. You need to be very careful about it in kernel modules when porting code under GPL-incompatible licenses from other platforms by ensuring that you don't put code into the .ko that is under the GPL with the GPL-incompatible code and use only EXPORT_SYMBOL defined functions (anything EXPORT_SYMBOL_GPL is not allowed) and public headers.

Figure out whether we actually did what? It's not clear what you mean here.

To keep it simple, the issue in userspace is whether you made any library code subject to the GPL that is used by code under the APSL. In the kernel, it is quite clear that the source licenses could pose a problem because the two are mixed in the same module. There is no documented linking exception that the project made for your kernel code's GPL sources to workaround the GPL's refusal to play nicely with the APSL. If you were to do that, then you might as well go with the LGPL.

Anyway, I can't see a distribution shipping a module like the one Darling uses. As far as I know, no module that any Linux distribution has ever shipped has mixed GPL and GPL-incompatible code in the way that Darling does. Nvidia, ZFSOnLinux, etcetera all avoid doing this because the practice is generally considered to make distributing binaries to be a GPL license violation on the part of the distributors.

facekapow commented 4 years ago

So I've been trying to wrap my head around this issue and I've realized that licensing is a real headache 🤦‍♂️.

From what I understand about this issue, it sounds like the issue is with claiming our code is GPL licensed when it actually contains multiple licenses. This post on Stack Exchange, however, seems to indicate that as long as the requirements for all licenses are followed, it's ok to license the project "as a whole" as GPL, but a notice of the additional licenses must be included (as necessary to comply with the terms of those licenses). The Wikipedia page for the GPL seems to confirm this. Since they're all part of the same binary in the end (and no intermediate libraries are created), there should be no issue with static or dynamic linking licensing issues.

Moreover, the code in the module that implements the API is under the GPL; there don't seem to be any cases where non-GPL code calls into GPL code. The code that uses the kernel module is also GPL licensed (see src/kernel/emulation/linux in the main repo).

Non-GPL and GPL linking

Where there might be certain issues with non-GPL code calling into GPL code is any code that makes BSD/Darwin syscalls (i.e. code that calls into libsystem_kernel, that's the library built by src/kernel/emulation/linux). However, it's possible that this would be exempted under the "system library exception": the syscalls are expected to be provided as part of the "system" (and Darling is acting as the "system" here). So at least for non-GPL software not included with Darling (e.g. apps or libraries users install/use), there's no issue. However, see my next paragraph for why there might still be a possible issue with libsystem_kernel.

Non-GPL and GPL, all distributed together, linking

Where there is certainly an issue is with non-GPL libraries/programs that use GPL libraries that are all distributed as part of Darling. For example, darling-corecrypto. The license says that it's GPLv3, which is fine in terms of the code it uses (it has some APSL headers, but assuming the same terms as for the kernel module, it's ok to mix them in). The possible problem arises because non-GPL code included as part of Darling (e.g. the Security library) is linking with GPL libraries also included as part of Darling (e.g. CoreCrypto) and directly calling GPL functions in them. I'm not 100% sure, but I believe it does not qualify for the "system library exception": while CoreCrypto is expected to be part of the "system" that Security is running on, they are distributed together; see this, particularly:

To prevent unscrupulous distributors from trying to use the System Library exception as a loophole, the GPL says that libraries can only qualify as System Libraries as long as they're not distributed with the program itself. If you distribute the DLLs with the program, they won't be eligible for this exception anymore; then the only way to comply with the GPL would be to provide their source code, which you are unable to do.

It seems the only way the GPL libraries (like libsystem_kernel or CoreCrypto) can fulfill the system library exception requirements is to be distributed as a separate package.


It seems to me that the best route here if we want to stick with the GPL is to just go with the LGPL, which at least explicitly allows dynamic linking while still retaining many of the other restrictions of the GPL. It would still require anyone that modifies Darling itself to distribute their code or otherwise ask for permission, while resolving many of the cloudy licensing issues. In any case, if someone only modifies Darling in terms of linking with and extending its libraries, we can't prevent that because that's exactly what application code does when it runs under Darling, so it's not like the LGPL would introduce any undesired side effects or loopholes.


I am obviously not a lawyer, so if anyone would like to better explain this or correct me, please do so.

bugaevc commented 4 years ago

Where there might be certain issues with non-GPL code calling into GPL code is any code that makes BSD/Darwin syscalls

The possible problem arises because non-GPL code included as part of Darling (e.g. the Security library) is linking with GPL libraries also included as part of Darling (e.g. CoreCrypto) and directly calling GPL functions in them

But isn't the system library exception about linking GPL code to otherwise licensed system libraries? It's indeed hard to understand all this licensing details, but at least the question you've linked to is "Can I link a GPL program with a proprietary system library".

It seems to me that the best route here if we want to stick with the GPL is to just go with the LGPL

I don't think it's possible or desirable to change the license of the entire project. It would probably be possible to only relicense the problematic bits (libsystem_kernel, CoreCrypto) though.

HinTak commented 4 years ago

FWIW, the last time I heard, the linux kernel itself is GPLv2 and v2 only, because Linus disagrees with Richard Stallman on the v3-specific part.

While I think that clearly labelling and separating parts which have different licenses is a good idea, I think the "incompatible" ideas only in the theoretical / fanatical sense.

facekapow commented 4 years ago

But isn't the system library exception about linking GPL code to otherwise licensed system libraries? It's indeed hard to understand all this licensing details, but at least the question you've linked to is "Can I link a GPL program with a proprietary system library".

Ah, you're right; looks like I had it backwards in my mind. In that case, I don't think there's any loophole. The current license (GPLv3) would certainly require programs linking to our implementations to be distributed under the GPL. Even programs that users download on their own and run under Darling would technically be violating the license terms by linking to our GPL implementations and not being GPL themselves.

Of course, this whole problem goes away if we change the license of the affected libraries to LGPL, like you said. This would include libsystem_kernel, libcorecrypto, and the many in-repo frameworks that we have reimplemented. (I might have missed a few other libraries). Notice that I said in-repo because certain libraries that have their own repositories like Cocotron or Foundation have different licenses that are already okay (MIT and LGPL, respectively).


By the way, just to backup my claim that it's perfectly okay to include more permissively-licensed code in a GPL project, I found this Stack Exchange answer that links to a very useful guide on how to include non-GPL code within a GPL project. However, the non-GPL code must still be GPL-compatible, and that's where our issue is.


What still bothers me is that the FSF has stated that the APSL is incompatible with the GPL. I'd like a specific clarification on why this is, though (I don't see it explained anywhere; the FSF just says it's incompatible).

The best explanation I could find was this, from the debian-legal mailing list. The problem seems to be that the APSL imposes further restrictions on covered code which the GPL explicitly forbids in section 10. Most of the APSL is okay, but it seems that section 2.2 is the real issue: it imposes an additional restriction that requires distribution of the source code if the program is used to provide a service. This issue seems to also be backed up by this, especially this part:

For example, the APSL defines "Externally Deploy" in section 1.4 and uses the term in section 2.2(c) such that if you cause the program to provide a service over the internet, then you must make available the APSL source code files. The APSL is here stronger than the GNU GPL.

(emphasis mine)

So effectively the problem is that we can't comply with both licenses at the same time, because the APSL imposes additional requirements that the GPL explicitly forbids.

LubosD commented 4 years ago

The problem is trying to see Darling as a monolithic project, but Darling is more like a macOS (instead of Linux) distribution. Linux distributions also distribute software with mutually incompatible licenses.

That being said, GPL stipulates in section 10:

You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License.

But the spirit of this section is so that people who license our code from us (= "you" in the text) the authors, may not add additional restrictions, so that if someone forks Darling, they may not forbid users from using it commercially.

However, since we're the authors of the GPL'd code, we are absolutely free to say so. However, we rarely mix APSL and GPL in a single subproject. The only case I remember is the LKM.

And f*ck FSF's legal advice. Really. When I contributed to GNUstep, they made me sign a copyright assignment agreement with the FSF, desregarding the fact that this action is null and void under the laws of my country.

HinTak commented 4 years ago

I have a lot of respect for many of the things that Richard Stallman did, so I wouldn't want to be rude - but sometimes FSF is indeed a bit fanatical. They are the software equivalent of religious extremist suicide bombers sometimes.

APSL is not that bad. You'll need to go to Raspbian to see that they are shipping GPL software with proprietary (as in not even available in source form) firmware / boot loader bits. IMHO, licensing in darling just isn't a problem. And one should not try to make non-problems into problems.

HinTak commented 4 years ago

I think a close-enough analogy is Sun/Oracle's zfs. It is a kernel module, with code which is under the weird Sun/Oracle open license, not GPL. It is basically Sun/Oracle's version of APSL. Some people like FSF and even Linus take the extremist view and won't touch it; some says it is okay and, last I heard, ubuntu 20.04 is even shipping with it as a root file system option.

TheBrokenRail commented 4 years ago

I mean Oracle might actually sue you for even thinking about violating their license.

HinTak commented 4 years ago

Whereas fanatics, like Richard Stallman, would sue based on ideology, big corporations like Sun/Apple have multiple people/committees, often with a more objective view, usually only do so based on likely financial gain / losses. That means either (1) you are hurting their business and they need to shut you down, or (2) you are rich enough to cough some money if they succeed.

So in practice, they would sue individuals for (1) - if you talk about bugs/vulnerabilities, or actually causing them to lose sales, or (2) you are making money from this, which they want a cut of.

Zfs on linux indeed satisfies (1) - you are drawing users away from Solaris -on which zfs was a reason to use - to linux.

Darling is not a threat to apple's market share: there are people who use Solaris ONLY BECAUSE of zfs, but most mac os x users are not there because of one or two applications.

atagunov commented 3 years ago

Folks, if this is the issue that prevents me from 'apt-get'-ing your code from Ubuntu central repositories, please do fix it. Even if the issue is imaginary and only exists in the heads of distribution builders. That's bad enough. You want Darling to be easily available. I'm sure there are lots of people like me out there - we will not be downloading Darling as a separate OS. But we may give it a go if's part of regular central repositories. Pls. make it crystal-clear for distro folks. Obvious. No-brainer.

bugaevc commented 3 years ago

Folks, if this is the issue that prevents me from 'apt-get'-ing your code from Ubuntu central repositories, please do fix it.

Why do you think that's the case? Has there been a discussion among Ubuntu maintainers on whether to ship Darling or not, and they decided to not ship it because of concerns over the licences? If so, could you please point us to that discussion?

wdlkmpx commented 3 years ago

I guess GPL3 is the problem, the linux kernel is GPL2, WINE is GPL2.

Apple and maybe the BSD's were using GPL2 software, gcc, bash, but with GPL3 they chose to keep using old versions until Clang and other projects replaced gcc, etc..

There is a very strong opposition to GPL3 outside Linux, even Linus himself seems to oppose it https://www.youtube.com/watch?v=PaKIZ7gJlRU

So relicensing the project to GPL2 may solve the issue.

The project needs traction and maybe a sponsor, maybe you should create a business around darling, just like the CW folks, but macOS may not be more popular than linux, in fact I think it isn't, so overall it's the end, the project will die.

TheBrokenRail commented 3 years ago

The project needs traction and maybe a sponsor, maybe you should create a business around darling, just like the CW folks, but macOS may not be more popular than linux, in fact I think it isn't, so overall it's the end, the project will die.

Well that's pessomistic.

wdlkmpx commented 3 years ago

Well that's pessomistic.

Sorry, I'll be more optimistic. I've no idea how complex this project is, or how hard it is to make it progress... both are UNIX-like systems.

When talking about macOS we're talking about big bucks, the machines themselves, a somewhat low user base, the game industry is with Windows, and projects like WINE cannot happen without corporate sponsors... i.e. CodeWeavers, the contributors themselves being paid through a commercial product based on WINE. And working with Steam and so on... the big players.

But it works... WINE runs basically all the apps I want, I can use Linux as the OS and Windows apps for most tasks. I would use Darling only to test some console apps I cross compile, I don't know macOS apps.

Probably nobody knows how much Linux is used in the desktop, but it runs on all my laptops old and new, and it runs fast on old machines.

In this case, help from macOS users may be the way to go, but licensing is an issue, the BSD's and derivatives tend to antagonize Linux, Google is creating it's own OS to replace Linux: Fuchsia.

So, the business model used by WINE may not be possible.. to some degree, but licensing does prevent contributions from the UNIX world.

The BSD's antagonize Linux and it's a strong dispute, and out of necessity FreeBSD has created a Linux compatibility layer, like darling or WINE, but Linux doesn't need a compatibility layer with FreeBSD.

wdlkmpx commented 3 years ago

Seeing a bit more, Cocoa and stuff... the whole macOS world seems to be too far away from the rest of the UNIX world, the BSDs (including Solaris) and Linux use Xorg or some X server, Linux is now forcing Wayland, and GNOME is destroying the Linux desktop with their mobile GTK UI design and forcing it on everyone not using their DE.

Xorg is MIT but it's dying anyway because Red Hat is now focused on Wayland and aims to destroy the incipient Linux desktop. The future looks bleak.

WINE does build on free software, but they write thousands of code lines replacing whole libraries, building a complete system with their own code, that's insane. They use LGPL v2.1 which is not as strict as GPL.

It's just not feasible to replicate that insane amount of work and magical (reverse) engineering with Darling, it's a delicate flower that has the potential to fade away (immediately), so guys think about choosing a better license, and pull lots of stuff from Apple, see it as technical challenge, 'cause all the money is in Windows with CrossOver or Wine, UNIX-like systems only need Windows apps and nothing else.

13rac1 commented 3 years ago

@wdlkmpx This issue is only about APSL/GPL licensing compatibility. Please stay on topic and refrain from being so negative. The Darling team has put in years of work and thousands of people are interested (see the 7.2K stars.) Your anecdotal experience regarding only needing Windows yourself is not relevant.

As for APSL/GPL: Binaries created with APSL or GPL can be packaged/distributed together, but a GPL application cannot use statically-compiled APSL-licensed libraries. IANAL, it may best to get a lawyer involved to answer these questions precisely.

ahyattdev commented 3 years ago

It's just not feasible to replicate that insane amount of work and magical (reverse) engineering with Darling, it's a delicate flower that has the potential to fade away (immediately), so guys think about choosing a better license, and pull lots of stuff from Apple, see it as technical challenge, 'cause all the money is in Windows with CrossOver or Wine, UNIX-like systems only need Windows apps and nothing else.

@wdlkmpx not really sure what your point is. no need to trash us. why are you even on here if you dislike darling?