Closed ghaerr closed 1 year ago
jart
commented
2 years ago I support this. It would need to be community maintained for the time being. The biggest thing I recommend doing is creating a shell script in the build folder that rewrites the cc flags so clang is happy. That way you can send PRs quickly. I won't merge PRs that change the binary bootstrap blob for compile.com.
ghaerr
commented
2 years ago I support this. It would need to be community maintained for the time being.
Thank you, I could maintain it, as well as submit an acceptable PR that gets macOS builds fully working.
The biggest thing I recommend doing is creating a shell script in the build folder that rewrites the cc flags so clang is happy.
I did that at first, but then thought it might be better to remove the AddArg("-no-canonical-prefixes") from compile.c. I'll go back to using a shell script.
I won't merge PRs that change the binary bootstrap blob for compile.com.
Ok. Would you consider a PR adding a "build/compiler.mk", to be inserted just before "build/definitions.mk" in the Makefile? This would allow for setting all the CC= etc compiler strings for macOS (and other platforms), and also nicely fixes the problem mentioned above where CC= is used in build/definitions.mk for the SPAWNER := line, which needs the correct CC= definition to come prior.
Seperately, could you comment on the "-x-no-pg" issue seemingly only implemented on your static Linux musl compiler? Was this implemented specially by yourself, as it doesn't work on other builds of Rich Felker's compilers. I don't currently see a workaround for this, we need a way to turn off -no-pg.
Thank you!
jart
commented
2 years ago There's an llvm section in config.mk. Would you be interesting in maintaining that? In other words, you'd put something like make MODE=darwin or make MODE=llvm. We use the -x-no-pg flag because sometimes it's needed to turn that flag off. It's recognized by compile.com. You can do what we used to do. See its predecessor https://github.com/jart/cosmopolitan/blob/c91b3c50068224929cd4d3ddf36c5a10254cbd9c/build/compile
ghaerr
commented
2 years ago Hello @jart,
There's an llvm section in config.mk. Would you be interesting in maintaining that?
Yes, I'd be happy to maintain the macOS port for you. I was thinking that this might be done in two steps: the first, getting Rich Felker's gcc musl-based cross compiler building Cosmopolitan, and then in a second step considering the issues with getting llvm building and linking properly. The advantage of the first step is that the compiled objects should be very close if not identical to the Linux versions, and that the ld.bfd linker can be used, rather than having to deal with Apple's llvm-ld at first.
I have Cosmopolitan building now on macOS, using make MODE=darwin. There are still some small issues when restarting the make, for instance:
make MODE=darwin
♥cosmo
error: "usr/share/zoneinfo/" (o/darwin/libc/time/localtime.o) not defined by direct deps of o/darwin/libc/time/time.a.pkg
o/darwin/libc/calls/syscalls.a.pkgAfter sorting these issues out, I can submit a PR for your consideration.
We use the -x-no-pg flag because sometimes it's needed to turn that flag off. It's recognized by compile.com.
Thanks, I didn't realize the flag was recognized by compile.com. I'm looking into why compile.com isn't turning off the -pg flag in this case, there is some special handling by compile.com looking at the basename of the compiler. In my case, the compiler name is the same, except the path to it is /usr/local/bin, rather than o/thirdparty/gcc/bin.
You can do what we used to do. See its predecessor https://github.com/jart/cosmopolitan/blob/c91b3c50068224929cd4d3ddf36c5a10254cbd9c/build/compile
Got it. I would like to be able to solve this issue without having to resort to using a fancy shell script to run the compiler, since there is already compile.com. There are only couple small issues remaining.
Thank you!
jart
commented
2 years ago We're already using musl-cross-make embedded binaries to build on Linux. If that runs on Darwin then it'll probably help. If you don't want to do the shell script then would you like me to modify compile.com so that it filters the flags appropriately if IsXnu()? I guess what you could do is send me a PR that changes compile.c and then I could submit another change that rebuilds the binary.
ghaerr
commented
2 years ago would you like me to modify compile.com so that it filters the flags appropriately if IsXnu()? I guess what you could do is send me a PR that changes compile.c and then I could submit another change that rebuilds the binary.
That would be great! I will post a minimal PR shortly. I have the darwin build working now.
ghaerr
commented
2 years ago Hello @jart,
Things are looking a lot better regarding using macOS for compilation of Cosmopolitan. In short, the entire repo can be made to compile with just one change to build/definitions.mk (using Rich Felker's x86_64-musl-gcc compiler), although later on the build fails in some of the python tests.
I am going to close #386, as much has been done since April for macOS cross-compilation. Only the following patch is required at this point:
diff --git a/build/definitions.mk b/build/definitions.mk
index 4b4f1cd22..19788da32 100644
--- a/build/definitions.mk
+++ b/build/definitions.mk
@@ -86,6 +86,20 @@ OBJCOPY = o/third_party/gcc/bin/x86_64-pc-linux-gnu-objcopy.exe
OBJDUMP = o/third_party/gcc/bin/x86_64-pc-linux-gnu-objdump.exe
ADDR2LINE = $(PWD)/o/third_party/gcc/bin/x86_64-pc-linux-gnu-addr2line.exe
else
+UNAME := $(shell /usr/bin/uname)
+ifeq ($(UNAME),Darwin)
+AS = x86_64-linux-musl-as
+CC = x86_64-linux-musl-gcc
+CXX = x86_64-linux-musl-g++
+CXXFILT = x86_64-linux-musl-c++filt
+LD = x86_64-linux-musl-ld.bfd
+NM = x86_64-linux-musl-nm
+GCC = x86_64-linux-musl-gcc
+STRIP = x86_64-linux-musl-strip
+OBJCOPY = x86_64-linux-musl-objcopy
+OBJDUMP = x86_64-linux-musl-objdump
+ADDR2LINE = x86_64-linux-musl-addr2line
+else
IGNORE := $(shell build/bootstrap/unbundle.com)
AS = o/third_party/gcc/bin/x86_64-linux-musl-as
CC = o/third_party/gcc/bin/x86_64-linux-musl-gcc
@@ -99,6 +113,7 @@ OBJCOPY = o/third_party/gcc/bin/x86_64-linux-musl-objcopy
OBJDUMP = o/third_party/gcc/bin/x86_64-linux-musl-objdump
ADDR2LINE = $(PWD)/o/third_party/gcc/bin/x86_64-linux-musl-addr2line
endif
+endif
export ADDR2LINE
export LC_ALLWould you like me to post that as a PR?
Other items to note:
Typing make using GNU make v3.81 fails with the following error:
zorg:cosmopolitan greg$ make
♥cosmo
third_party/mbedtls/test/test.mk:408: *** multiple target patterns. Stop.Running build/bootstrap/make.com works.
The build eventually fails with the following python-related test errors:
zorg:cosmopolitan greg$ !build
build/bootstrap/make.com
♥cosmo
...........E....E......
======================================================================
ERROR: test_fieldstorage_multipart_maxline (__main__.CgiTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/zip/.python/test/test_cgi.py", line 324, in test_fieldstorage_multipart_maxline
check('x' * (maxline - 1))
File "/zip/.python/test/test_cgi.py", line 322, in check
self.assertEqual(gen_result(data, environ),
File "/zip/.python/test/test_cgi.py", line 109, in gen_result
form = cgi.FieldStorage(fp=fake_stdin, environ=environ, encoding=encoding)
File "/zip/.python/cgi.py", line 569, in __init__
self.read_multi(environ, keep_blank_values, strict_parsing)
File "/zip/.python/cgi.py", line 744, in read_multi
self.encoding, self.errors, max_num_fields, self.separator)
File "/zip/.python/cgi.py", line 571, in __init__
self.read_single()
File "/zip/.python/cgi.py", line 765, in read_single
self.read_lines()
File "/zip/.python/cgi.py", line 794, in read_lines
self.read_lines_to_outerboundary()
File "/zip/.python/cgi.py", line 870, in read_lines_to_outerboundary
self.__write(odelim + line)
File "/zip/.python/cgi.py", line 802, in __write
self.file = self.make_file()
File "/zip/.python/cgi.py", line 919, in make_file
return tempfile.TemporaryFile("wb+")
File "/zip/.python/tempfile.py", line 597, in TemporaryFile
fd = _os.open(dir, flags2, 0o600)
OverflowError: signed integer is greater than maximumTraceback (most recent call last): File "/zip/.python/test/test_cgi.py", line 240, in test_fieldstorage_readline f = TestReadlineFile(tempfile.TemporaryFile("wb+")) File "/zip/.python/tempfile.py", line 597, in TemporaryFile fd = _os.open(dir, flags2, 0o600) OverflowError: signed integer is greater than maximum
Ran 23 tests in 0.016s
Closing as quite outdated, thanks!
Hello @jart,
[Edits below: some earlier errors were my misunderstanding of build process]
I'm continuing the discussion from https://github.com/jart/cosmopolitan/issues/227#issuecomment-1077806881, but am opening a new issue, so as not to be off-topic. I hope that's ok, and excuse in advance this long report.
I (finally) now have almost the entire repo compiling (and all .com files produced seemingly working) under macOS. I'd like to help contribute to Cosmopolitan but need to be able to build everything under macOS, since I don't have easy everyday access to Linux. And of course, a huge motivation is to play more with blinkenlights.com, which now is working and compiling from scratch on macOS :)
Rather than immediately produce a PR, I thought to get your input as to various issues found during the port. At present, some of the fixes required hacks, which will want to be fixed differently. I will describe each issue in more detail below using a bullet point.
I installed a pre-compiled homebrew version of Rich Felker's musl-cross-make x86_64-linux-musl-gcc first, which installs the compiler in /usr/local/bin on macOS. Using this version could be helpful, if it were ever decided to build the Linux and macOS compilers from the same source and build scripts.
Second, it was found that macOS (Catalina) is running GNU make 3.81, which silently ignores the
$(file ...)function, used to prepare argument lists all over. Installing homebrew version of make 4.3 and then executingexport PATH=/usr/local/opt/make/libexec/gnubin:$PATHsolved that problem.These macOS updates then allowed a small diff and a few other changes to compile most of Cosmopolitan.
Here's the entire diff:
A problem in definitions.mk is that the "SPAWNER := $(shell build/getcompile) -V$(shell build/getccversion $(CC))" line uses one-time assignment which contains $(CC), which ends up using the definitions.mk value, rather than the config.mk value of $(CC). Thus CC= has to be changed in definitions.mk as well.
I'm not sure the options I've selected for macOS (which are copies of the "MODE=tinynowin" mode) are best for macOS.
Compile.com adds "-no-canonical-prefixes" to the CC compile line automatically, with no way to remove it. This causes an
excecvp error: cc1 not found, with seemingly no option to turn prefixes back on.A possible compiler incompatibility was noticed in third_party/mbedtls/zeroize.c, where the
-x-no-pgoption (a non-standard option supposed to turn off -pg?) is being interpreted as a language option and giving the error "no such language -no-pg". Thus, both that and the -fomit-frame-pointer options had to be (temporarily) removed. This is likely incorrect, but I'm not sure what to do.The test/libc/release/emulate.sh was modified to just exit, since all *.com.dbg executables are ELF rather than machO binaries.
With these changes, the cosmopolitan builds all libraries, headers and examples, with all .com files working, up to third_party/chibicc/test/common.chibicc.o, which fails due to chibicc.com.dbg not being executable.
I'd be happy to help get Cosmopolitan firmly building on macOS, if you're willing. You have an amazing project! How would you like to proceed?
Thank you!