Gprbuild configure failure

[Earnestly]

I've gone to the trouble of preparing a bootstrap which uses the binary gprbuild (and gprconfig and gprinstall) from the 309M size tarball to build and install xmlada.

Now that I've made xmlada available at version 2.2-743-g8a9536b I went ahead and naively attempted to ./configure gprbuild itself but was met with the following error:

checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking for xmlada... no
configure: error: in `/home/earnest/build/store/gprbuild-git/src/gprbuild':
configure: error: cannot find xmlada
See `config.log' for more details

So I dug into config.log and configure itself only to find that if xmlada is not a directory found in the gprbuild tree it will create a conftest.gpr and attempt to use gprbuild to discover where xmlada is.

But gprbuild is what I'm trying to build.

Why has ./configure being written in such a way to assume the project it's actually building is already extant? Is it possible to explicitly tell the build process where xmlada is?

I'd really like to expose errors and failures building this software in hopes to get it fixed instead of all of the problems being silently fixed downstream.

[Earnestly]

Okay, let's try a new attempt.

Some ancillary information:

• GCC 5.3.0

This time I decided to try bootstrapping gprbuild with the gprbuild blob along with xmlada in the source directory.

Finally ./configure passes but I'm faced with a new error on make all:

% export PATH=$PATH:$HOME/build/store/gprbuild-bootstrap/src/gprbuild/gnat-gpl-2015-x86_64-linux-bin/bin
% make all
gprbuild -p -m  -j0 -XOBJDIR=/home/earnest/build/store/gprbuild-bootstrap/src/gprbuild -XBUILD=production -P./gprbuild -XLIBRARY_TYPE=static
object directory "/home/earnest/build/store/gprbuild-bootstrap/src/gprbuild/obj/" created
object directory "/home/earnest/build/store/gprbuild-bootstrap/src/gprbuild/libobj/static" created for project gpr
library directory "/home/earnest/build/store/gprbuild-bootstrap/src/gprbuild/lib/static" created for project gpr
exec directory "/home/earnest/build/store/gprbuild-bootstrap/src/gprbuild/exe" created for project gprbuild
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprconfig-main.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprbuild-main.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprbind.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprlib.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprclean-main.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprinstall-main.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprslave.adb
gcc -c -gnat12 -gnaty -gnatQ -O2 -gnatpn -gnatws -g1 gprname-main.adb
gprslave.adb:1636:13: "Kill_Process_Tree" is undefined
gprslave.adb:1665:13: "Kill" is undefined

   compilation of gprslave.adb failed

gprbuild: *** compilation phase failed
Makefile:69: recipe for target 'all' failed
make: *** [all] Error 4

[Earnestly]

Is Kill and Kill_Process_Tree perhaps something coming with GNAT 6? (GCC 6.1).

Edit: It appears this might be the case.

[t-14]

Hi. Indeed Kill and Kill_Process_Tree are fairly recent additions (we pushed them to the public in October and November 2015 respectively). If you prefer to stay with 5.3, you can workaround this limitation by just not building gprslave at all - this tool is only needed to support massive build projects using distributed build farms, which is a fun capability but is not strictly necessary.

[Earnestly]

No I'm quite happy to go forward if possible (in fact I'm pretty militant about using latest stable releases with a fairly negative view of backporting efforts). I should be getting GCC 6.1 soon, or if not I can just build it locally.

Although I'm curious now as to how you'd not build gprslave. Naively I'd use sed -i '/gprslave/d' gprbuild but is this the correct way of doing it?

I do have one last question if you don't mind and that is can we build gpr based projects as both production and including debug symbols? (Typically the packaging system is responsible for stripping debug symbols (at the users own choice) and optionally installing them under /usr/lib/debug as a separate package for tools like gdb to use.)

[t-14]

Although I'm curious now as to how you'd not build gprslave.

Only manual way unfortunately, by deleting/commenting out one reference to gprslave in gprbuild.gpr and another one in Makefile.in.

I do have one last question if you don't mind and that is can we build gpr based projects as both production and including debug symbols? (Typically the packaging system is responsible for stripping debug symbols (at the users own choice) and optionally installing them under /usr/lib/debug as a separate package for tools like gdb to use.)

Not at the moment,in fact we have only recently become aware of this capability when users started asking about it. It definitely sounds like something we would like to investigate/support.

[Earnestly]

Although now I'm seeing a new problem:

checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking for xmlada... yes (sources in gprbuild tree)
checking whether gnat can build shared libs... yes
configure: creating ./config.status
config.status: creating Makefile
gprbuild -p -m  -j0 -XOBJDIR=/.../gprbuild -XBUILD=production -P./gprbuild -XLIBRARY_TYPE=static
object directory "/.../gprbuild/obj/" created
object directory "/.../gprbuild/libobj/static" created for project gpr
library directory "/.../gprbuild/lib/static" created for project gpr
exec directory "/.../gprbuild/exe" created for project gprbuild
gprbuild.gpr:1:06: unknown project file: "xmlada.gpr"
gprbuild: "./gprbuild" processing failed

The xmlada source tree is linked into the gprbuild:

lrwxrwxrwx 1 earnest users   59 Apr 26 13:40 xmlada -> ../xmlada/

And ./configure confirms that xmlada is in the gprbuild tree. Are there perhaps any environment variables I'm missing for gprbuild to find xmlada?

All I've exported is PATH so gprbuild (and gpr{config,install}) are available.

export PATH=$PATH:$srcdir/gnat-gpl-2015-x86_64-linux-bin/bin

Any suggestions?

[t-14]

There are several special *_PATH variables and a few other places that gprbuild looks in to find imported projects -

First, the file is searched relative to the directory that contains the current project file.

Then it is searched relative to all the directories specified in the environment variables GPR_PROJECT_PATH_FILE, GPR_PROJECT_PATH and ADA_PROJECT_PATH (in that order) if they exist. The value of GPR_PROJECT_PATH_FILE, when defined, is the path name of a text file that contains project directory path names, one per line. GPR_PROJECT_PATH and ADA_PROJECT_PATH, when defined, contain project directory path names separated by directory separators. ADA_PROJECT_PATH is used for compatibility, it is recommended to use GPR_PROJECT_PATH_FILE or GPR_PROJECT_PATH.

Finally, it is searched relative to the default project directories. Such directories depend on the tool used. The locations searched in the specified order are: prefix/target/lib/gnat if option --target is specified prefix/target/share/gpr if option --target is specified prefix/share/gpr/ prefix/lib/gnat/

[Earnestly]

Thanks, using this informtion I added a call to export GPR_PROJECT_PATH:

export GPR_PROJECT_PATH=$srcdir/xmlada

This appears to help finding the xmlada.gpr and the build commences only to run into these errors:

gprbuild -p -m  -j0 -XOBJDIR=/home/earnest/build/store/gprbuild-bootstrap-git/src/gprbuild -XBUILD=production -P./gprbuild -XLIBRARY_TYPE=static
object directory "/home/earnest/build/store/gprbuild-bootstrap-git/src/xmlada/sax/obj/" created for project xmlada_sax
library directory "/home/earnest/build/store/gprbuild-bootstrap-git/src/xmlada/sax/lib/" created for project xmlada_sax
gprbuild: Exception name: STORAGE_ERROR
Message: s-intman.adb:139 explicit raise
Call stack traceback locations:
0x7874f6 0x7fa7a5e682ce 0x5e07be 0x66bbf0 0x674c04 0x674ee2 0x674e28 0x6768ef 0x674eae 0x674e28 0x6768ef 0x676c8b 0x6546ef 0x6d1a9e 0x6d1f1d 0x40fc51 0x40a875 0x7fa7a58c401a 0x40a8cf

Makefile:69: recipe for target 'all' failed
make: *** [all] Error 4

I have no idea what this STORAGE_ERROR is referring to as there's no error message or diagnostic information outside of a handful of "traceback" locations...

My goal here is to build gprbuild from this repository using the static gprbuild from the 300+M tarball while using xmlada from https://github.com/AdaCore/xmlada. The resulting gprbuild is then to be used to build xmlada and gprbuild proper along with anything else.

Edit: I looked into s-intman.adb:139 and it refers to this piece of code:

         when SIGSEGV =>
            raise Storage_Error;

That doesn't look great...

[t-14]

That doesn't look great...

Arf, no it doesn't, indeed. Quick check, do you have TMPDIR set and is it writable? That's my only idea at the moment. I'll try retracing your steps and see if it works for me.

[Earnestly]

TMPDIR isn't set, however /tmp (if you fallback) is writable.

# $srcdir contains the xmlada, gprbuild and gnat-gpl-2015-x86_64-linux-bin
# extracted source. It's also our $PWD.

cd gprbuild

ln -sf "$srcdir"/xmlada .

export PATH=$PATH:$srcdir/gnat-gpl-2015-x86_64-linux-bin/bin

export GPR_PROJECT_PATH=$srcdir/xmlada

# XXX Need GCC 6.1 for Kill_Process_Tree and Kill to build gprslave
#     So for now we don't build gprslave.
sed -i '/gprslave/d' gprbuild.gpr Makefile.in

./configure --prefix=/usr --libexecdir=/usr/lib
make

Here is the complete environment during this process: https://ptpb.pw/_l6H/sh

[Earnestly]

Just to update, I am now on GCC 6.1.1 (obviating the need to not build gprslave), however, this situation still persists.

[Earnestly]

Quick question, as adacore doesn't provide checksums or offer signed distribution (while over http, I might add) can you confirm that the gnat-gpl-2015-x86_64-linux-bin.tar.gz has the sha1sum of e80af8088dad88e1d88dda59bf20c6c021e3f1bf?

I've downloaded this twice now and my internet isn't so good which makes this quite tedious, also waiting for the tarball to extract itself multiple times can cause disruption to the running system due to the IO load, so if you can confirm my tarball isn't corrupt that will assuage my concerns.

[t-14]

this situation still persists.

By "situation" you mean crash in gprbuild? It's still on my list to investigate.

e8...bf is indeed the correct checksum for this file. Good point about providing checksums for our downloads, I'll pass it along.

[Earnestly]

Yeah, I get the same error, even the same backtrace locations. Unfortunately it doesn't seem to generate a core dump.

Thanks for confirming the checksum.

[quinot]

@Earnestly Thanks for your feedback. The download page in our Libre site now lists the SHA1 for every available file.

[Earnestly]

@quinot Thanks for that, although keep in mind for future that it might be better to use SHA256 or larger. SHA1 is however, still more-or-less safe.

Ideally you'd sign them with gpg, which would be the best solution.

[t-14]

@Earnestly - just to let you know that I tried with gprbuild GPL 2015 for x86_64-linux and unfortunately I can't reproduce your issue.

My cookbook:

1. Get gnat-gpl-2015 and remove everything but gpr* exes in the bin directory (let's call this gprbuild-boot).
2. set path to gprbuild-boot/bin:gnat-install/bin:...
3. Get xmlada-master, inside do "./configure" followed by "make static" and "make install-static prefix=_xmladainstall"
4. set GPR_PROJECT_PATH=_xmladainstall/lib/gnat (note "lib/gnat" part, important!)
5. Get gprbuild-master, inside do "./configure" followed by "make", that does it for me.

If you are still getting crashes, in a few days we should release GPL 2016 (and we intend to have gprbuild as a standalone download there), so perhaps worth giving it another try at that point.

[Earnestly]

It's interesting to see your steps include building xmlada itself. I made the assumption that as gprbuild's ./configure had mechanisms to detect an in-tree xmlada that it would build them too, or use their source. This is how, for example, gcc builds in libraries such as mpfr, cloog, mpc and isl.

I'll try explicitly building xmlada and pointing at that like you have. I have a feeling this was the missing step since my very first attempt might have used an already installed xmlada.

Edit: Thank you for including a standalone gprbuild.

---

Success! Almost...

This appears to work for the build however I have a few questions about it:

During the build I saw two lines refering to clang instead of gcc:

clang -c gprbuild_dummies.c
clang -c gpr_imports.c

Normally this isn't an issue (besides making clang a build dependency) but is there a reason to hardware clang instead of using the CC environment?

The next question is about why make install doesn't seem to fully honour prefix=. I am using make prefix="$pkgdir"/usr install where "$pkgdir" is defined as the (absolute) destination directory. It works for most of the install target but fails at the end with:

mkdir -p /lib/gprbuild
mkdir: cannot create directory ‘/lib/gprbuild’: Permission denied
Makefile:107: recipe for target 'install.bin' failed
make: [install.bin] Error 1 (ignored)
/usr/bin/install -c /home/earnest/build/store/gprbuild-bootstrap-git/src/gprbuild/exe/gprbind /lib/gprbuild/
/usr/bin/install: target '/lib/gprbuild/' is not a directory: No such file or directory
Makefile:107: recipe for target 'install.bin' failed

This is because on line 37 of the Makefile I have:

libexecdir=/lib

By using ./configure --libexecdir=/lib (as we don't use /libexec) I've managed to trick configure into removing the ${exec_prefix} which it should normally preserve. I have no idea why this is happening since no other project employing autotools has this issue.

As a workaround I've used ./configure --libexecdir=/usr/lib so that anything which hardwires this path is correct, and then overriding the variable during install with:

make prefix="$pkgdir"/usr libexecdir="$pkgdir"/usr/lib install.data install.bin

Lastly, is there any reasons for the various gpr* programs to have an executable stack? [1] [2] (See scanelf -pleq)

Otherwise, thank you for your effort and attempts to reproduce my problems.

Edit: Also I apologise for using these issues as mailing lists, combining several problems into a single thread.

[t-14]

It's interesting to see your steps include building xmlada itself. I made the assumption that as gprbuild's ./configure had mechanisms to detect an in-tree xmlada that it would build them too, or use their source.

It's conceivable, but is not the case currently. Normally xmlada is preinstalled in our distributions, so we didn't spend much energy on supporting the other case.

Note that it still doesn't crash for me if I put the xmlada source dir in GPR_PROJECT_PATH, just returns an error. But probably not worth spending more time on this.

Normally this isn't an issue (besides making clang a build dependency) but is there a reason to hardware clang instead of using the CC environment?

Nothing is hardwired actually, gprbuild (or rather its gprconfig component) is simply scanning the path, finding the tools that are registered in its knowledge base. You can try running gprconfig without arguments, it will display all toolchains it has found. Probably your clang is just earlier on the path so picked up for the C toolchain.

The next question is about why make install doesn't seem to fully honour prefix

Some cleanup is definitely needed in the makefile. We only use it in a very specific fashion in our builds, so it can certainly use some polishing.

Lastly, is there any reasons for the various gpr* programs to have an executable stack?

Executable stack is needed to implement some corner cases of the Ada semantics. It's not needed in 99.9% of cases but currently the compiler doesn't annotate the objects that don't need it. This looks like something that may be worth investigating, I'll pass it along to the compiler team.

Edit: Also I apologise for using these issues as mailing lists, combining several problems into a single thread.

Works for me. We've touched on issues that aren't necessarily related to gprbuild but some are definitely worth looking into. Although for gprbuild issues it's probably better to keep things separated indeed.

[Earnestly]

Nothing is hardwired actually, gprbuild (or rather its gprconfig component) is simply scanning the path, finding the tools that are registered in its knowledge base.

Oh this is somewhat problematic, does gprbuild have an ability to honour the concept of environments such as CC (along with CFLAGS, LDFLAGS, etc.) [ideally the same ones]? I know at least the Makefile for this has GPROPTS but many projects using gprbuild don't express anything like it.

[t-14]

You must do so explicitly. You can either do it on the level of the config project or main project file itself, like e.g.

    package Compiler is
      for Driver ("C") use External ("CC", "gcc");
    end Compiler;

or

   for Switches ("C") use External_As_List ("CFLAGS", " ") & My_Own_Switches;

or you can pass the toolchain you want via gprconfig --config to generate the config file you want (see docs for details).

[Earnestly]

Oh dear...

How about a compromise: can I create a file containing these general defaults and import them (e.g. include in a Makefile) for each project?

[t-14]

Sorry. Basically it's just not gprbuild's philosophy. The whole idea of it is that you trust the knowledge base to do the right thing. If it doesn't - then either you have a very project-specific use pattern, in which case you encode it in the project; or this is a special scenario (like instrumented debugging/coverage/etc) - then you encode it as a distinct scenario; or finally it is just the wrong default for the platform, in which case you fix it in the knowledge base.

But our experience shows us that relying on something as ephemeral as environment variables to alter the build process is dangerous and difficult to troubleshoot, so while stopping short of forbidding it completely we try nevertheless to make this explicit.

[Earnestly]

Yes, and rolling over PATH to find something which can compile C is "trustworthy". Brilliant.

[t-14]

Well, PATH strikes me as a rather intuitively obvious place to look, wouldn't you agree? Without any special variables that's where configure/make will look, won't they? So while I can't really judge the brilliance of it, at least I don't understand why you'd feign surprise over this choice.

[Earnestly]

No, I would not agree at all. What I am at a loss for is just how much we as an industry ignore our own history and keep making the same mistakes over and over again. I don't expect you to understand this, and I'm not going to detail sane alternatives (to even the use of PATH). I have given up with this; weeks of struggling is not worth it.

[t-14]

Ok, sorry to see you feel so strongly about this, but it's your choice of course. We've made ours and didn't regret it over the course of this project. In any case it's a fundamental aspect of the design and not something that can be changed to suit individual preferences.