Failed ABI checks

[kmoore134]

I'm trying to do a build of 10.3-BETA1 package and keep getting failed ABI checks whenever we start a new build. (Which of course requires rebuilding pkgng, and all other packages)

root@bellicose:/synth/world # file bin/sh bin/sh: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 10.2 (1002509), stripped root@bellicose:/synth/world # ./bin/freebsd-version 10.3-BETA1

How is the ABI determined? Anything else I can run to figure out why this fails?

[kmoore134]

BTW, I should have mentioned, the host system is running -CURRENT, but we are building 10.3-BETA1 packages using a 10.3-BETA1 base in /synth/world. I assume this is supported?

[jrmarino]

heh. This is where synth differs from poudriere. synth is using a filtered version of the host (by default). So it looks like you're fixing 10.2 and 10.3. However, that still shouldn't matter wrt to the ABI.

what does pkg -VV | grep ABI show?

in the meantime, I'll tell you where the ABI code is.

[jrmarino]

here is interesting: https://github.com/jrmarino/synth/blob/master/src/portscan-packages.adb#L731

It's calculating it from the uname. What does "uname -mr" show?

[kmoore134]

root@bellicose:/synth/pkg # pkg -vv | grep ABI ABI = "FreeBSD:11:amd64"; ALTABI = "freebsd:11:x86:64"; root@bellicose:/synth/pkg # uname -mr 11.0-CURRENTJAN2016 amd64

[jrmarino]

hmm, that uname should led to the same ABI. the vm I have here is 11.0-CURRENT i386 and that has no issues.

[jrmarino]

what packages is it complaining about? packages that synth built previously?

[kmoore134]

giving it another whirl now. The existing packages its checking originally came from a poudriere build, would that matter?

[jrmarino]

probably yes, give it a clean directory

[jrmarino]

if you are really curiously, use pkg query to find out ABI of any package in that poudriere direcotry

[jrmarino]

pkg  query -F <path to package> '%q' 

I think

[kmoore134]

pkg query -F pcbsd-utils-qt5-1454017176.txz '%q'

freebsd:10:x86:64

[kmoore134]

interesting, is it because its x86:64 and not x86_64?

[jrmarino]

well, there you go

[jrmarino]

no, it's because it's 10 and not 11

[kmoore134]

ok, that begs the question then. Should this work? Or will all 10.x packages always fail ABI check when building on 11.x host?

[kmoore134]

I.E. our build nodes for 10.2 / 10.3 / 11-CURRENT all run -CURRENT. With synth I'm giving it a 10.3 world in 'Directory_system= /synth/world'

[jrmarino]

loaded question. If your 11 host builds a 10 world and installs it, and a synth profile uses that 10 sysroot, you can build 10 packages on 11.

[jrmarino]

but if you use the live system, which is 11, then it's going to be limited to 11 packages. poudriere does the same thing with jails

[kmoore134]

maybe I'm misunderstanding the docs then. Is using the Directory_system= the right way to give it a 10 sysroot?

[jrmarino]

I.E. our build nodes for 10.2 / 10.3 / 11-CURRENT all run -CURRENT. With synth I'm giving it a 10.3 world in 'Directory_system= /synth/world'

ok, logic flaw on my part

[jrmarino]

checking uname isn't going to work

[jrmarino]

kmoore: you are doing it right, synth method of getting abi is logically flawed

[kmoore134]

ok, whew ;) Do you want to ping back when you have a fix? I'll be eager to test it

[jrmarino]

let me think about it a bit. I'll ping you back It will be easy to rebuild with github hash

[jrmarino]

by the way, it does use the uname in the specified sysroot, but I forgot that uname just reads the running kernel

[kmoore134]

BTW, while I've got you doing synth things ;)

It would be cool if we had a 'cleanup' sub-command which only does the umount routines. We have things run via Jenkins, and that would be handy if something crashed / didn't cleanup properly.

[jrmarino]

it already exists. run "synth configure" (hit enter) Any command (other than help and version) will clean up dirty mounts

[kmoore134]

Will that work via script? I.E. in automation for jenkins?

[jrmarino]

for jenkins, you won't be able to use any saved options configuration (unless you script that they are always valid, otherwise synth will stop before it starts). However, I assume you don't override options anyway.

I don't see any command that will exit without doing anything substantial.

However, in theory, the next command that the script sends will clean it up. The only way it would fail is if a process is still running which prevents the umount.

so in other words, if the mounts are dirty and a build command is issued, the mounts will always be clean up before anything else. they don't need a special command.

[jrmarino]

okay Kris, I have something for you to test.

On the makefile line 24, change the hash value to "24bc284" then run "make makesum" then the whole clean/disinstall/install routine. It should work as expected now.

[jrmarino]

I mean "makesum" (edited above)

[kmoore134]

Getting closer now, no ABI errors, but each package fails with the following:

` ========================< phase : check-sanity >========================

make: "/xports/Mk/bsd.port.mk" line 1162: UNAME_r (11.0-CURRENTJAN2016) and OSVERSION (1002509) do not agree on major version number.

===========================================================================`

This is expected, you'll need to also set UNAME_r env variable before starting a builder, i remember doing this back in the day when I was doing manual builds ;

[jrmarino]

there's a workaround right now. You can

/etc/local/etc/synth/<profile>-environment

and set it manually.

It's a freebsd-specific issue. I wouldn't have seen it on dragonfly

[jrmarino]

I think the solution is related to the work I just did. Synth is caching _OSVERSION and it's caching the host version. This is the same logic flaw as uname (the difference being I tried to fix the former but failed and I missed this one)

So the same "file /bin/sh" solution for ABI can be reused to set _OSVERSION correctly.

[jrmarino]

ok, code is reading <sys/param.h> so that's what I need to do as well.

[kmoore134]

Ok, setting UNAME_r=10.3-BETA1 in /usr/local/etc/synth/-environment worked fine.

Do you want to add something to set that environment variable automatically though? Thats what poudriere does, if you set it to the output of the 'freebsd-version' from the new world directory, it should work.

[jrmarino]

no, I'm going to fix it properly.

[kmoore134]

Ok, works for me. I'm going to keep testing with the workaround in the meantime, see if we run into any other issues.

[jrmarino]

okay. I'll have a fix later today. I'm working on it now.

[jrmarino]

This should be the proper fix: f676f03 The only "strange" thing is the _OSRELEASE now looks something like "11.0-SYNTH". The suffix is "-SYNTH" regardless of what it really is. Hopefully this won't cause any issues. I guess we will see.

ARCH, OSVERSION, _OSRELEASE, OPSYS were all reworked.

[kmoore134]

cool! I've got some builds going now, I'll give the fix a whirl shortly

[kmoore134]

Ok, ran some tests here. With the change applied, I'm still getting errors about UNAME_r not matching:

` ========================< phase : check-sanity >========================

make: "/xports/Mk/bsd.port.mk" line 1162: UNAME_r (11.0-CURRENTJAN2016) and OSVERSION (1001000) do not agree on major version number.

`

Additionally I notice some oddness in the configure scripts:

OSVERSION=100000 _OSRELEASE=1.0-SYNTH

Shouldn't _OSRELEASE be 10.0-SYNTH?

Adding UNAME_r manually to environment fixes problems

[jrmarino]

those are the "default" values when something failed.
This worked for me. Can you post part of sys/param.h, the line that defines __FreeBSD_version ?

[kmoore134]

#define __FreeBSD_version 1001000       /* Master, propagated to newvers */

[kmoore134]

This is on a 10.1-RELEASE system base BTW

[jrmarino]

no, I mean the version in the system root.

[jrmarino]

or is that it?

[kmoore134]

yea, that is the file from /synth/world/usr/include/sys/parm.h, where /synth/world is my 10.1-RELEASE base

[jrmarino]

can you find a synth log and and paste the contents of the make.conf according to the log? There's nothing wrong with the input line as far as I can tell.

[kmoore134]

Here's a couple:

http://slexy.org/view/s2hX1CkrER

http://slexy.org/view/s2G9XiLpyn

I think you still need to set UNAME_r properly in the environment. Otherwise the port system will run 'uname -r' manually and see the wrong (host) version. Thats why poudriere has to manually set that variable in each jail / builder.