llvm10 port

[magenbluten]

i ( successfully ) built the llvm10/clang10 port. i noticed however that the compile times from the produced clang++ are huge:

$  cat main.cxx

#include <iostream>

int main(){
    std::cout << "hello" << std::endl;
}

$ command time /raven/bin/clang++ -O2 main.cxx -o main-clang10 -I/raven/include/c++/v1/ -L /raven/lib/ -lc++ -nostdinc++ -lc++abi -lunwind -stdlib=libc++
       11.45 real        11.43 user         0.02 sys

$ command time /raven/toolchain/gcc9/bin/g++ -O2 main.cxx -o main 
        0.31 real         0.26 user         0.05 sys

binary size of clang++ is also huge ( 60MB ).

[jrmarino]

wow. what platform? FreeBSD?

[magenbluten]

yes

$ uname -a 
FreeBSD launchbox 12.1-RELEASE-p3 FreeBSD 12.1-RELEASE-p3 GENERIC  amd64

also, i realized that clang reports FreeBSD 11 compat:

$ /raven/bin/clang++ --version
clang version 10.0.0 
Target: x86_64-raven-freebsd11
Thread model: posix
InstalledDir: /raven/bin

Maybe some compiler flags missing? like -O2 or something? i didn't change anything, just followed the wiki as far as possible.

[magenbluten]

from /raven/ravenports/primary/logs/logs/llvm___standard.log

=> Building llvm:standard (version 10.0.0)
Started : Wednesday,27 MAY 2020 at 09:32:16 UTC
Platform: 11.1-RAVEN FreeBSD 11.1-RAVEN #0 Sat Jul 29 09:00:00 CDT 2017 root@octavia.unreal.systems:/usr/obj/usr/src/sys/GENERIC amd64
Watchdog: Disabled

...

CC="cc"
CFLAGS="-pipe -O2   -I/raven/include"
CPP="cpp"
CPPFLAGS="-I/raven/include"
LDFLAGS="-lpthread"
LIBS=""
CXX="c++"
CXXFLAGS=""

so CXXFLAGS is actually empty. and the platform seems wrong.

[jrmarino]

so CFLAGS was setting getting set by framework (meaning -O2 -pipe should be removed from CFLAGS settings - it's additive, not replacing). But it's a problem that the framework doesn't set these flags for cxx too, so I'm going to fix that and remove -O2 -pipe changes. I guess we didn't see it before because often c++ is compiled with both cflags and cxxflags? Maybe. maybe other c++ ports have same issue.

[magenbluten]

Yes. seems to be a framework issue

CXXFLAGS always empty.

theoretically, for sane build systems

• CFLAGS is for c only
• CXXFLAGS is for c++ only
• CCFLAGS is for both

--------------------------------------------------
--  MAKE_ENV
--------------------------------------------------
HOME=/dev/null
BSD_INSTALL_PROGRAM="install -s -m 555"
BSD_INSTALL_LIB="install -s -m 444"
BSD_INSTALL_SCRIPT="install -m 555"
BSD_INSTALL_DATA="install -m 0644"
BSD_INSTALL_MAN="install -m 444"
PREFIX=/raven
LOCALBASE=/raven
OPSYS="FreeBSD"
CC="cc"
CFLAGS="-pipe -O2   -I/raven/include"
CPP="cpp"
CPPFLAGS="-I/raven/include"
LDFLAGS=""
LIBS=""
CXX="c++"
CXXFLAGS=""
STD_DOCDIR="/raven/share/doc/xmlto"
STD_EXAMPLESDIR="/raven/share/examples/xmlto"
MANPREFIX="/raven/share"
ADA_PROJECT_PATH="/raven/lib/gnat"
F77="gfortran"
FC="gfortran"
PATH=/bin:/usr/bin:/raven/toolchain/gcc9/bin:/raven/toolchain/gcc9/bin:/raven/toolchain/bin:/raven/sbin:/raven/bin

On Wed, May 27, 2020 at 2:36 PM jrmarino notifications@github.com wrote:

so CFLAGS was setting getting set by framework (meaning -O2 -pipe should be removed from CFLAGS settings - it's additive, not replacing). But it's a problem that the framework doesn't set these flags for cxx too, so I'm going to fix that and remove -O2 -pipe changes. I guess we didn't see it before because often c++ is compiled with both cflags and cxxflags? Maybe. maybe other c++ ports have same issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jrmarino/ravensource/issues/209#issuecomment-634630550, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWN3SWZPEWECWGXWOJEVDRTUCMLANCNFSM4NK76YBA .

[jrmarino]

I have fixed it already. It's in ravenports but won't appear until the next release for most people. I'm working on getting that release out today.

[jrmarino]

so you were right:

loki# ls -alh /build/ravenports/release58/packages/All/llvm-s*
-rw-r--r--  1 root  wheel   543M Apr 10 15:34 /build/ravenports/release58/packages/All/llvm-single-standard-10.0.0.tzst
-rw-r--r--  1 root  wheel    73M May 27 12:51 /build/ravenports/release58/packages/All/llvm-single-standard-10.0.0_1.tzst
loki# ls -alh /build/ravenports/release58/packages/All/clang-*1.tzst
-rw-r--r--  1 root  wheel    58M May 27 14:33 /build/ravenports/release58/packages/All/clang-compiler-standard-10.0.0_1.tzst
-rw-r--r--  1 root  wheel   460B May 27 14:33 /build/ravenports/release58/packages/All/clang-complete-standard-10.0.0_1.tzst
-rw-r--r--  1 root  wheel    26M May 27 14:33 /build/ravenports/release58/packages/All/clang-extra-standard-10.0.0_1.tzst

Unfortunately the revision wasn't properly set in clang so I can't see the package size difference there. but I looked up the last release:

clang-compiler-standard-10.0.0.tzst                10-Apr-2020 15:54           374815908

So it looks like it dropped from around 360M to 58M

[kraileth]

This is excellent! I hated how building FileZilla took hours on FreeBSD (due to being built with Clang). Guess that this resolves it. I've always wanted to take a closer look at what is happening (and if we're actually using ccache when compiling with Clang, too).

P.S.: Moin Magenbluten und willkommen in unserer illustren Runde!

[kraileth]

also, i realized that clang reports FreeBSD 11 compat:

$ /raven/bin/clang++ --version
clang version 10.0.0 
Target: x86_64-raven-freebsd11
Thread model: posix
InstalledDir: /raven/bin

Oh, and this is due to Raven using a FreeBSD 11.1 base for the build chroot, BTW. We could update that to 11.3 since 12.0 is out of support now, but I'm not sure if such a move wwould yield any real advantages. Looking forward to a time when all supported FreeBSD versions support evdev, though, as I'd really like to bring our default X11 to using that.

If you'd like to mess with this, have a look at bucket_B4/ravensys-root. The FreeBSD version to use is set by a define at the top of the file.

[jrmarino]

so in this case, we want the oldest release of the base series.
For example, 11.2 can run 11.1, but 11.2 can't run 11.3 (confidently). I have been considering dropping release 11 support for a while. I'm not sure what the best point to do that is (other than 11 series going EOL"

[jrmarino]

in fact, I think I had to move the base release from 11.0 to 11.1 because 11.0 was EOL'd, if memory serves ..

[jrmarino]

(which meant the 11.0 image was no longer available to download and I didn't want to host it myself)

[kraileth]

Weren't there some issues with 11.0 that were gone in 11.1? I think you mentioned that in a mail back then... The 11.1 ISO has also gone away from the original location quite a while ago. I remember that I adjusted the port to use the archive URL instead when I hit that problem on a fresh system.

Anyway: 11.4 is just around the corner and I think dropping 11 now would not be a good move. In fact evdev support in 12.1 is flawed (I need to set a sysctl on my main laptop or the touchpad won't work). It's fixed in HEAD as far as I know. So if it's for this feature we might at least wait for 12.2 to be released before becoming 12-only - if we wish to drop 11 before it is officially EOL'd.

[jrmarino]

The other option is keeping both freebsd 11 and freebsd 12 until 11 is EOL'd. It's not what I originally wanted to do, but maybe it's the best. There are some ports that are freebsd 12 only.

[jrmarino]

I mean building binary packages for both. But that might require adding another variant to ravensys-root (unless the 11 root is just kept locally, unreproducibly)

[magenbluten]

i'm basically new for freebsd. since like 6 months :)

On Wed, May 27, 2020 at 6:02 PM jrmarino notifications@github.com wrote:

I mean building binary packages for both. But that might require adding another variant to ravensys-root (unless the 11 root is just kept locally, unreproducibly)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jrmarino/ravensource/issues/209#issuecomment-634764613, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWN3Q3EM76CZCHW4VBSKLRTU2Q7ANCNFSM4NK76YBA .

[magenbluten]

ach das ist ja lustig :)

frage mich selbst wie ich hier mal wieder gelandet bin :)

On Wed, May 27, 2020 at 4:44 PM kraileth notifications@github.com wrote:

This is excellent! I hated how building FileZilla took hours on FreeBSD (due to being built with Clang). Guess that this resolves it. I've always wanted to take a closer look at what is happening (and if we're actually using ccache when compiling with Clang, too).

P.S.: Moin Magenbluten und willkommen in unserer illustren Runde!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jrmarino/ravensource/issues/209#issuecomment-634709614, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWN3WI5RR65QDHJHJ3JSLRTURLBANCNFSM4NK76YBA .

[jrmarino]

I just built filezilla on my freebsd machine with your verision of clang. Build time dropped from like 75 minutes to 5 minutes. Not bad. 👍

[magenbluten]

nice! glad that i could be of help :)

could you figure out the mystery of the disappearing "CXXFLAGS"? because it hits all ports compiled with c++

[jrmarino]

yes, I updated the framework to set CXXFLAGS so that should be done.

If you truly want bleeding edge of the entire tree (which the build of all ports is not guaranteed) you would cd to the Scripts directory of ravensource and execute "regenerate_conspiracy.sh" with no arguments. That regenerates every build sheet known to ravensource.

If you just care about a single port (which is the normal case for development) then "ravenadm dev buildsheet save" is the way to go.

what I typically do is (using libreoffice as an example, cwd has to be within ravensource tree for this to work):

> eval `ravenadm dev jump libreoffice`

make my changes, followed by

> ravenadm dev buildsheet . save
> ravenadm test libreoffice

[jrmarino]

it should be needless to say that for 99% of users, they just want and need "ravenadm update-ports". They should not be using bleeding edge in production. A new release is usually around the corner so they should wait for those milestones.

[magenbluten]

just for the understanding.

$ ravenadm dev jump foobar

jumps to ravensource containing folder for foobar.

save regenerates the buildsheet and stores it in the corresponding folder.

if i do a ravenadm update-ports will the changes to overwritten?

On Thu, May 28, 2020 at 2:34 PM jrmarino notifications@github.com wrote:

it should be needless to say that for 99% of users, they just want and need "ravenadm update-ports". They should not be using bleeding edge in production. A new release is usually around the corner so they should wait for those milestones.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jrmarino/ravensource/issues/209#issuecomment-635321448, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWN3S72JOJ34DOYXKFMFLRTZK6XANCNFSM4NK76YBA .

[jrmarino]

no,

eval `ravenadm dev jump foobar`

The eval and backticks are critical. yes, "save" writes the compiled buildsheet to ravenports bucket directory. and yes, "update-ports" would overwrite any manual changes.

[jrmarino]

So your work was really helpful. The old sun linker can't build clang with all the static llvm libraries, but it can build your optimized/dynamic version. And the previous linking time was measured in hours (like 45 minutes to compile and 6+ hours to link). Now it builds completely in like 30 minutes. I'm really grateful for your contribution!

[magenbluten]

your welcome! was actually much easier to "debug" than initially thought :)