search

pret / agbcc

C compiler
104 stars 75 forks source link

Add binutils, agbcc optimizations, reformat, and use a root Makefile. #23

Closed easyaspi314 closed 5 years ago

easyaspi314 commented 5 years ago

binutils and cpp (platform-independent) are the only things we use from devkitPro, so let's leave the enormous, confusing-to-install, GPL-violating toolchain to die.

binutils is currently being cherrypicked to remove components for other CPUs which we don't need. PRs are welcome to trim down this enormous tree. Don't run ./configure directly unless you know the flags.

To test the DKP-free build, do 

$ make -j4
$ make install prefix="<pokeruby root>"

then, in pokeruby/Makefile, change the first line

include $(DEVKITARM)/base_tools

to

PREFIX := $(CURDIR)/tools/binutils/bin/arm-none-eabi-

and 

CPP      := $(PREFIX)cpp

to

CPP      := cpp

Tell me if it works.

I also added -O2 to agbcc's Makefile, forced it to run serially because I tried dependency tracking and it is a mess, and converted build.sh and install.sh to a single Makefile in the root which can be run in parallel (and I suggest you do because you will have to wait for recursive configure scripts) - I silenced output of them because it is blinding.

easyaspi314 commented 5 years ago

Working configurations:

Ubuntu 16.04 x64 on WSL
GCC 5.4.0
Android 8.1 armv7a
Termux
Clang 6.0 (with flags changed)
GCC 8.1
Native NDK binutils
easyaspi314 commented 5 years ago

For anyone who had a build fail, please do this

git pull
rm -r binutils
git checkout binutils
make clean; make -j4
WinterMute commented 5 years ago

devkitARM does not violate the GPL.

People repackaging devkitPro tools as their own product violate the trademarks.

There is no requirement for the buildscripts to be licensed at all.

The reason for this is because some people think repackaging binaries and putting them up on their own site under their own name is somehow good for the community.

This is just cutting off your nose to spite your face rather than engaging in any useful debate.

ketsuban commented 5 years ago

There is no requirement for the buildscripts to be licensed at all.

What about the C runtime files (e.g. gba_crt0.s)? Those end up in everything that's compiled with devkitARM unless you provide your own like pret does. Pretty sure those need to be licensed, but they're in the unlicensed buildscripts repository.

WinterMute commented 5 years ago

gba_crt0.s isn't distributed with your homebrew though.

All those files are compiled and the object files distributed with the toolchain which become part of the particular configuration of binutils/gcc/newlib and support tools known as devkitARM. As such these files therefore become GPLv3 licensed with linking exception as per libgcc. There is no more legal issue with distributing binaries built with devkitARM than there is with any other gcc based toolchain.

Choosing to troll a completely unrelated project you haven't contributed to with what seems to be a deliberately malicious interpretation of devkitPro policy rather than engaging in any civil discussion somewhere sensible is just unpleasant and offensive.

Hundreds of thousands of homebrewers have managed to install devkitARM with perfect ease. The only way it really gets easier than https://devkitpro.org/wiki/devkitPro_pacman is if I come round your house & install it for you.

What on earth would possess you to submit a 2K file, 1.6million line PR to a project that uses devkitARM rather than just asking someone who actually might know why we object to people badly repackaging the tools and causing issues we end up having to solve?

ketsuban commented 5 years ago

All those files are compiled and the object files distributed with the toolchain which become part of the particular configuration of binutils/gcc/newlib and support tools known as devkitARM. As such these files therefore become GPLv3 licensed with linking exception as per libgcc. There is no more legal issue with distributing binaries built with devkitARM than there is with any other gcc based toolchain.

I'm no lawyer, but the idea you can attach a different license to a source file and the object file it compiles to seems dubious. That said, easyaspi apparently knows the GPL better than me so I'll defer to their response if they choose to make one.

Choosing to troll a completely unrelated project you haven't contributed to with what seems to be a deliberately malicious interpretation of devkitPro policy rather than engaging in any civil discussion somewhere sensible is just unpleasant and offensive.

I agree, why did you post on this PR?

I've contributed to this project multiple times (1, 2, 3). I only spoke up here because you showed up, apparently having namesearched with no idea what you were actually commenting on. Nothing under the pret umbrella is "a devkitARM project" in the sense implied by TuxSH on IRC when they asked why pret doesn't use LLVM; agbcc is an adaptation of the GPL'd code for the GCC fork Nintendo released as part of their official Game Boy Advance SDK, and uses devkitARM solely as a convenient source of binutils and cpp.

WinterMute commented 5 years ago

@ketsuban Nobody is claiming a different license for the source files and the object files. devkitARM is provided under the GPL but that license gives you no trademark rights.

The line linked to at the start of this ridiculous PR refers to the scripts that build the toolchains which are, as stated there, provided as a courtesy so people whose host platforms have no binary distribution can build the tools themselves. They're not provided so people can distribute the binaries they build independently. This has been a major issue for several years and we still have people turning up reporting "bugs" in things that aren't devkitARM.

I agree, why did you post on this PR?

I posted on this PR because @easyaspi314 is telling lies about devkitARM and I was informed of this issue by people who understand the GPL and manage to use the tools without making a huge fuss about nothing on entirely unrelated projects.

ketsuban commented 5 years ago

Nobody is claiming a different license for the source files and the object files. devkitARM is provided under the GPL but that license gives you no trademark rights.

These are your words.

gba_crt0.s isn't distributed with your homebrew though. All those files are compiled and the object files distributed with the toolchain [...]. As such these files therefore become GPLv3 licensed with linking exception as per libgcc.

My understanding of this is that you are by claiming gba_crt0.s is not licensed to me (as it's stored in the buildscripts repository, which has no license attached) but gba_crt0.o is licensed to me under the terms of the GPL (and thus poses no legal trouble for anything I make with devkitARM).

It's not the first time you've been made aware of this and you've done nothing to clarify the situation. (The issue has since closed, which is obnoxious - you don't seem to have the kind of throughput that would justify setting up autoclose - and unhelpful, since you complained upthread about people bringing this up in unrelated spaces.) Perhaps the "people who understand the GPL" could post either here or in the issue I just linked to explain clearly and definitively why devkitARM doesn't violate it, but in the meantime it's a bit rich to accuse @easyaspi314 of "telling lies" when you've failed to dispel the apparent legal uncertainty in your product despite people voicing their concerns.

WinterMute commented 5 years ago

The issue you linked asked for the buildscripts to be given a permissive license for the express purpose of repackaging our tools elsewhere. Such a license will not be provided and the issue was closed due to obnoxious trolling that was deleted. Implying an issue you've never discussed with us by linking to an issue which

Calling devkitARM "the enormous, confusing-to-install, GPL-violating toolchain." and linking to a statement referring to a set of bash scripts as "evidence" is clearly a malicious and offensive lie. Submitting a PR with this lie rather than contacting devkitPro staff directly to clarify whatever the "concern" might be is offensive and unacceptable behaviour.

Calling this kind of trolling and outright misrepresentation "voicing concerns" is disingenuous at best.

I suggest using an appropriate venue to ask for clarification of whatever your issue is and refrain from libelling us while you do it. This issue isn't appropriate, nor is the pret discord.

https://devkitpro.org/wiki/Community_Portal

aaaaaa123456789 commented 5 years ago

I just found two cents on the table, so I figured I'd contribute them here.

If there's such a heated debate about whether this is an infringement of copyright, why not just write to the Free Software Foundation (the copyright holders for GCC and friends) and ask? They have a dedicated email for this.

If anyone has done this already, please make their response public and put an end to this nonsense.

easyaspi314 commented 5 years ago

@WinterMute I know you are one of the maintainers, right?

The issues I see are…

I am contacting the FSF right now to see if my speculations are correct. If they are not, I will correct myself, apologize, and try not to mention it again, although I think it would be a good idea to fix a few of these regardless for clarity and prevention of future issues.

easyaspi314 commented 5 years ago

In the meantime, please clean and make sure these changes still build.

mid-kid commented 5 years ago

Is including the entirety of the binutils source code into the agbcc tree really worth it, as opposed to including a tarball and unpacking it through a make rule? Or through a git submodule? In other words, what is more important, being able to modify it easily or upgrade it easily, and if the former, what's the reasoning for not hosting a proper fork under the pret umbrella?

easyaspi314 commented 5 years ago

The whole point is to reduce the size of the install, and the source code of the binutils is over 200 mb, so the space saved by removing binutils is lost by the size of the source tree.

Eventually, binutils will be reduced to something like agbcc, which had autotools and unused stuff removed to reduce it to a reasonable size.

mid-kid commented 5 years ago

I'm not too sure completely cutting off everything with upstream is a good idea, especially since we can benefit from any future updates, additions and fixes. There really wasn't any other choice with agbcc considering it isn't maintained anymore, but we don't exactly have a lot of people with intimate knowledge of binutils to be able to properly take over its maintenance. This isn't my place to say much about this, however. Just mentioning it might be worth considering.

TuxSH commented 5 years ago

@easyaspi314

"Not clearly stating the licenses of files like gba_crt0.s and the buildscripts, which, as of right now, is not under a GPL-compatible license, and...

GCC's linkscripts and crt0s are licensed as GPLv3 with linking exceptions, so unless explicitly otherwise mentioned dkA linkscripts and crt0s are the same and you can use them in proprietary programs.

aaaaaa123456789 commented 5 years ago

@TuxSH Since the devkitPro people kindly deleted my comment in their relevant issue (like anyone with something to hide would do), I'll replicate it here:

screenshot

That's where the GPL violation comments and concerns come from.

aaaaaa123456789 commented 5 years ago

Just so you know, the C standard guarantees that sizeof(char) will be 1 — multiplying by it is unnecessary.

easyaspi314 commented 5 years ago

I reformatted gcc and libc for clarity and removed the ugly _ansi.h wrappers.

From what I heard, while this builds on arm7a and x86_64, as segfaults on aarch64, built with Termux. I do not have a device to test on, so if someone would debug the crash, it would help.

aaaaaa123456789 commented 5 years ago

I don't know why that comment is showing up as outdated, but it isn't.

aaaaaa123456789 commented 5 years ago

Maybe it's a good time to locally set a pre-commit hook to watch out for Termux paths?

easyaspi314 commented 5 years ago

Damn it again. Trying this hook. Hopefully it won't happen again, fingers crossed.

#!/bin/sh

. git-sh-setup  # for die
git-diff-index -p -M --cached HEAD -- | grep '^+' | grep "com.termux" && die Fucking Termux!
aaaaaa123456789 commented 5 years ago

I wonder if this pull should be merged as-is or squashed before merging, if it is ever merged.

The reason is that, throughout its lifetime, the pull has lost two thirds of its code. It started at about a million and a half lines added, but it's at 600k now. Squashing the commits would cause people to not clone all of this useless history, which is massive.

On the other hand, rewriting history is bad and we should all feel bad.

ketsuban commented 5 years ago

Squashing seems correct to me. Rewriting history may be gross, but it's part of Git culture to rebase before pushing, and clearly less wrong to me than a million and a half lines of history nobody cares about.

(That said the correctest answer in my view is to depend on and conveniently provide prebuilt binaries for Nintendo's gcc and binutils, but I'm not going to pretend I have a hope of convincing anyone to drop agbcc.)

aaaaaa123456789 commented 5 years ago

it's part of Git culture to rebase before pushing

Thankfully, no. Plenty of people do things the sane way and merge the remote before pushing.

But this is a special case, and it makes sense to handle special cases in special ways. The full history will always be available in the fork, anyway.

conveniently provide prebuilt binaries

For how many OS/architecture combinations? I've seen both 32-bit and 64-bit Linux, 64-bit Windows, OS X (whatever the hell it runs on nowadays), 64-bit Android...

It's a lot easier to provide a build-once repo that everyone can build for their own setup; this repo does exactly that.

ketsuban commented 5 years ago

Evidently we've seen different recommendations about how to curate your contribution to opensource projects, because the ones I've seen emphasise presenting a clean history.

For how many OS/architecture combinations?

32/64-bit Windows/x86, 64-bit OSX/x86. Linux users are already on an ideal platform for rolling your own binary, we can leave instructions.

aaaaaa123456789 commented 5 years ago

I've seen the recommendations; fortunately not everyone follows them.

As for prebuilt binaries, I'd say that's what GitHub's releases feature is good for.

easyaspi314 commented 5 years ago

I am all for squashing, as .git literally takes up half of the size of the entire repo.

I just want to get everything done first:

  1. Remove most unused code. Our focus is armv4tl-none-eabi. We don't need, or want, code for any other CPU.
  2. Make sure everything is stable. My main target is aarch64/Termux, which apparently crashes because of an invalid FILE *, which is difficult to debug because I lack such a device. I want all sorts of hosts to be able to build and compile an OK Pokeruby, without any complaints.
  3. Kill autotools completely or make it as minimal as possible. It is, by far, the most time consuming part of the build, especially on Cygwin with its emulated fork(), and it is pointless because we are configuring the same thing each time.
  4. Remove all unused parts of libiberty, preferably removing it entirely.

Once we squash, it is difficult to undo history.

aaaaaa123456789 commented 5 years ago

Is ARM support still gone from this fork of agbcc? If this is a tool that hopes to be generally useful, and not a single-purpose compiler for the sake of the decompilations, ARM support and interwork support must be fully functional.