search

conda-forge / julia-feedstock

A conda-smithy repository for julia.
BSD 3-Clause "New" or "Revised" License
23 stars 32 forks source link

Attempt to build openblas here #220

Closed mkitti closed 2 years ago

mkitti commented 2 years ago

Checklist

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

mkitti commented 2 years ago

@conda-forge-admin, please rerender.

mkitti commented 2 years ago

I do not really understand which curl it is picking up and trying to use. Curl is available somewhere...

mkitti commented 2 years ago

I've got a bad feeling that cross compilation is not going work.

mkitti commented 2 years ago

This is silly. I'll have to think this through later.

mkitti commented 2 years ago 
make[2]: Entering directory '$SRC_DIR/deps'
Warning: git information unavailable; versioning information limited
/Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/tools/jldownload: line 31: /Users/runner/miniforge3/conda-bld/julia_1661675631923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin/curl: Bad CPU type in executable
/Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/tools/jldownload: line 31: /Users/runner/miniforge3/conda-bld/julia_1661675631923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin/curl: Bad CPU type in executable
Could not find working curl, wget, or fetch.
You need to install one of these to download dependencies.
make[2]: *** [/Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/libwhich.mk:4: /Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/srccache/libwhich-81e9723c0273d78493dc8c8ed570f68d9ce7e89e.tar.gz] Error 1
make[2]: *** Waiting for unfinished jobs....
Could not find working curl, wget, or fetch.
You need to install one of these to download dependencies.
make[2]: *** [/Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/libgit2.mk:106: /Users/runner/miniforge3/conda-bld/julia_1661675631923/work/deps/srccache/cacert-2022-02-01.pem] Error 1
make[2]: Leaving directory '$SRC_DIR/deps'
make[1]: *** [Makefile:60: julia-deps] Error 2
make[1]: Leaving directory '$SRC_DIR'
make: *** [Makefile:47: /Users/runner/miniforge3/conda-bld/julia_1661675631923/work/doc/_build/html/en/index.html] Error 2
ngam commented 2 years ago

I've got a bad feeling that cross compilation is not going work.

We can also try to see if we can build it locally first. That is, natively on arm64. We are likely getting arm64 public CI by the end of the year anyway, at which time, we will likely have to rebuild all these cross-compiled packages

ngam commented 2 years ago

Could not find working curl, wget, or fetch

keyword: "working"

meaning, we need to figure out a way to make it see the non-arm64 curl/wget

ngam commented 2 years ago

Isn't there like a toggle/flag to turn on cross-compiling btw?

mkitti commented 2 years ago

We can cross compile the C portions here via the script here: https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/libjulia/common.jl

mkitti commented 2 years ago

The question is how do we build the Julia system image which requires compiling Julia code itself?

ngam commented 2 years ago

https://discourse.julialang.org/t/cross-compiling-julia-from-source-for-the-raspberry-pi-3b/79301/6

ngam commented 2 years ago

https://github.com/conda-forge/openblas-feedstock/pull/146 I get you want to try to push the frontier by cross-compiling the sys image huh?

ngam commented 2 years ago

Also, remind me, do you have an M1/M2 machine now? If so, you could test this locally by something like (after installing boa in a some env)

conda mambabuild ./recipe -m ./.ci_support/osx_arm64_.yaml --suppress-variables --no-test --clobber-file ./.ci_support/clobber_osx_arm64_.yaml

If it works fine, then we can ask someone from core to upload the package manually. This is an option while we work on cross-compiling it. I am giving it a shot now.

Edit: stuck on

make[2]: Leaving directory '$SRC_DIR/src'
make[2]: Entering directory '$SRC_DIR'
Warning: git information unavailable; versioning information limited
    JULIA usr/lib/julia/corecompiler.ji

but with 99.8% CPU utilization of a process named julia... it's doing something.

Edit 2: it kept being stuck until I force-quit, it produced this message

/bin/sh: line 1: 72389 Killed: 9               /Users/ngam/.Mambaforge-MacOSX-arm64/envs/py310/conda-bld/julia_1661722912766/work/usr/bin/julia -C "generic;armv8.2-a,crypto,fullfp16,lse,rdm" --output-ji /Users/ngam/.Mambaforge-MacOSX-arm64/envs/py310/conda-bld/julia_1661722912766/work/usr/lib/julia/corecompiler.ji.tmp --startup-file=no --warn-overwrite=yes -g0 -O0 compiler/compiler.jl
make[2]: *** [sysimage.mk:61: /Users/ngam/.Mambaforge-MacOSX-arm64/envs/py310/conda-bld/julia_1661722912766/work/usr/lib/julia/corecompiler.ji] Error 137
make[1]: *** [Makefile:82: julia-sysimg-ji] Interrupt: 2
make: *** [Makefile:47: /Users/ngam/.Mambaforge-MacOSX-arm64/envs/py310/conda-bld/julia_1661722912766/work/doc/_build/html/en/index.html] Interrupt: 2
mkitti commented 2 years ago

I do have access to a Mac Studio.

If openblas-ilp64 works, we probably should focus on that.

Is something eating the output of the compilation process?

mkitti commented 2 years ago

I'm going to close this since the openblas-ilp64 seems to be moving forward.

ngam commented 2 years ago

Is something eating the output of the compilation process?

What do you mean? You are referring to it being stuck? I have no idea what's going on, but I will need to look into it more. Building natively on M1 needs a little more work I suppose, and we need to isolate it better (a lot of moving variables on osx)

ngam commented 2 years ago

Don't close it yet. Let me show you how to build it via my channel here

EDIT: oops never mind we cannot get the sys img anytime soon, but 47932ca is the edit to use an alt channel, it should be automated but it doesn't work in my tests...

mkitti commented 2 years ago

Could we attach a Github runner to compile Julia natively on a M1?

ngam commented 2 years ago

Could we attach a Github runner to compile Julia natively on a M1?

We could, but there is no point. The infrastructure for this has been delayed repeatedly sadly (for reference, we have the same issue in tensorflow, qt, and pytorch, and there's been long a talk of providing some CI for these but it never materialized, there was even some serious push and troubleshooting, etc. but the status quo is fine for now I guess...). WE might as well just build it locally ourselves. See my comment above https://github.com/conda-forge/julia-feedstock/pull/220#issuecomment-1229560532. We can then upload that to our personal channels and ask someone from the core team to move it to the main conda-forge channel after inspecting the saved local logs

For sketched cfep on this: https://github.com/conda-forge/cfep/blob/main/cfep-03.md

mkitti commented 2 years ago

Well I just did the following locally:

Just make

git clone https://github.com/JuliaLang/julia.git
cd Julia
make clean
make -j10

Then

$ bin/Julia
julia % ./julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-DEV.1244 (2022-09-01)
 _/ |\__'_|_|_|\__'_|  |  Commit cb5f401aaa (0 days old master)
|__/                   |

julia> sin(2π/3)
0.8660254037844387

julia> sin(π/3)
0.8660254037844386

julia> sin(4π/3)
-0.8660254037844385

julia> sin(π/6)
0.4999999999999999

So at least the master branch builds, but that is still pulling separately built binaries from BinaryBuilder.

make USE_BINARYBUILDER_OPENBLAS=0

Building with USE_BINARYBUILDER_OPENBLAS=0 errors with

 cblas_zsyr2k PASSED THE COLUMN-MAJOR COMPUTATIONAL TESTS (  1764 CALLS)
 cblas_zsyr2k PASSED THE ROW-MAJOR    COMPUTATIONAL TESTS (  1764 CALLS)

 END OF TESTS
make[3]: /Users/kittisopikulm/Documents/src/julia/usr/tools/objconv: No such file or directory
make[3]: *** [../libopenblas64__armv8p-r0.3.20.a.osx.renamed] Error 1
make[2]: *** [shared] Error 2
*** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0' if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. ***
make[1]: *** [scratch/openblas-0b678b19dc03f2a999d6e038814c4c50b9640a4e/build-compiled] Error 1
make: *** [julia-deps] Error 2

It is looking for objconv in julia/usr/tools/objconv. However, objconv is located in julia/usr/bin/objconv. See https://github.com/JuliaLang/julia/issues/46579

make USE_BINARYBUILDER_OPENBLAS=0 USE_BINARYBUILDER_OBJCONV=0 works locally

This works locally, testing for cross compilation here.

ngam commented 2 years ago

Yeah "just make" has been working on the main branch since 1.7. The issue here is that we have to use conda-forge toolings, and we have to use unvendor as much as possible

ngam commented 2 years ago

local build instructions https://github.com/conda-forge/julia-feedstock/pull/220#issuecomment-1229560532

ngam commented 2 years ago

Edit ./.ci_support/osxarm64.yaml to add the unofficial openblas or wait until it is merged and up on the main conda-forge channel (approx 30 mins after merge)

mkitti commented 2 years ago

With USE_BINARYBUILDER_OPENBLAS=0 USE_BINARYBUILDER_OBJCONV=0 I'm forcing openblas and objconv to build from source via https://github.com/JuliaLang/julia/blob/master/deps/openblas.mk https://github.com/JuliaLang/julia/blob/master/deps/objconv.mk

If this gets further, then we can at least compare those make files with the condo-forge build scripts.

mkitti commented 2 years ago

Oh, right this silly problem.

You need to install one of these to download dependencies.
Could not find working curl, wget, or fetch.
mkitti commented 2 years ago

https://github.com/JuliaLang/julia/blob/bb5b98e72a151c41471d8cc14cacb495d647fb7f/deps/tools/jldownload#L8-L10

WGET=$(which wget 2>/dev/null)
CURL=$(which curl 2>/dev/null)
FETCH=$(which fetch 2>/dev/null)

We would have to either trick which here or patch that.

ngam commented 2 years ago

Yeah, cross-compiling isn't fun. Just fyi, we're on our own. Just read this comment again https://github.com/JuliaLang/julia/issues/44517#issuecomment-1119056639, that's why I haven't been all that eager about julia (and the discource forum lol), but I am more than ready to support your efforts and do the best I can to help push things forward, even if it is doomed 😄

mkitti commented 2 years ago

Yeah, cross-compiling isn't fun. Just fyi, we're on our own. Just read this comment again JuliaLang/julia#44517 (comment), that's why I haven't been all that eager about julia (and the discource forum lol), but I am more than ready to support your efforts and do the best I can to help push things forward, even if it is doomed 😄

The process here is not too dissimilar. We usually are not going to build openblas from scratch here at the end of the day. We're going to pull the openblas-ilp64 binaries from the openblas-feedstock.

When they build Julia upstream, they use a build via this script via cross-compilation. https://github.com/JuliaPackaging/Yggdrasil/blob/master/O/OpenBLAS/common.jl.

mkitti commented 2 years ago

From my perspective in both cases, the dependency binaries are being pulled in are both from a distributed Azure Pipeline blackbox in the cloud that runs a build script. Is it really that different?

The biggest one seems that is that the Yggdrasil black box seems to be successfully cross compiling to more platforms than the conda-forge one. https://dev.azure.com/JuliaPackaging/Yggdrasil/_build/results?buildId=21745&view=results The conda-forge build system seems to be require multiple black boxes to cover fewer architectures.

mkitti commented 2 years ago

This reminds me of something though. libjulia is cross compiled via Yggdrasil, which is what we're basically trying to pull off here: https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/libjulia/common.jl

mkitti commented 2 years ago

Closing this in favor of trying the new openblas-ilp64 for macos-arm. Need to figure the jldownload issue.