Include paths <src/foo.h> not appropriate for installed library?

[markuspf]

I think if GAP is installed and used as a library, users would expect to #include <gap/foo.h>. This is of course somewhat related to sorting out the mess that is including.

[markuspf]

Installing headers for the library seems to be slightly more complicated than I expected. Not only do we need to copy all (most of?) GAP's *.h files, but also the generated gen/config.h.

[markuspf]

Speaking of which, is config.h actually required for a libgap?

[fingolfin]

Resolving these things is part of what is needed for issue #44. Since you list multiple issues, I'll reply to each separately.

First, regarding config.h: This (or rather: something taking its place) is absolutely needed. But this requires work: One should never install a "raw" autoconf config.h directly; it would make life miserable for any other autoconf-using project that might want to link against gap (such as GAP kernel extensions, which actually contain workarounds for this problem right now).

One possible solution then is to rename our config.h to e.g. gap-config.h; and to add a prefix to all the #defines in it (there is a macro for that in the autoconf-archive).

[fingolfin]

Next, regarding #include <src/foo.h>: It would of course be more elegant to have this as gap/foo.h. But then we'd need to worry about handling that while GAP is not installed. E.g. we could rename the src dir to gap -- but that would be ugly, and also confusing, because in the past, it was standard to put GAP sources of packages into gap dir instead of lib (and indeed, many packages still follow that convention).

Another way to do it would be to run sed over the headers as they are installed -- but then packages would not easily be able to be built against both installed and non-installed versions of GAP.

But we could also simply ignore "would look nicer" and go with "is straightforward and safe" and install header as PREFIX/include/gap/src/foo.. Then people who want to link against gap or libgap would add add -IPREFIX/include/gap to their CPPFLAGS and still use include <src/foo.h> in their code.

No, somewhat orthogonal to this, packages currently mostly only include src/compiled.h. This header was meant for C files produced by the GAP compiler originally. But we could add an official "master-header in its place, ideally named gap.h (so we'd have to rename the current gap.h to something else). And then, in an ideal world, we'd install it as PREFIX/include/gap.h, and all other headers as PREFIX/include/gap/foo.h ... But again, that's quite a bit of work. And I am not sure it's worth the trouble... One might argue that it would be nicer

[fingolfin]

Regarding installing config.h, this is the M4 macro I was thinking of : https://www.gnu.org/software/autoconf-archive/ax_prefix_config_h.html

[fingolfin]

Just discussed with @nthiery and wanted to document that here: So if we want to allow 3rd parties to do #include <gap/foo.h>, the easiest way to achieve that would actually be to change our internal includes from #include <src/foo.h> back to #include "foo.h" -- back because that's exactly how the #includes look like in master. But of course there was a reason for that: First, adding the src/ prefix was done to avoid conflicts between our string.h and the string.h defined by POSIX resp. ISO C (I think the GAP one might actually predates that standard... but that's irrelevant). Anyway, since string.h was removed as part of this branch, we could perhaps drop that prefix. But there is still the change from angular brackets. There is just one difference between the two: with quotes, the preprocessor will first look in the same directory as the active file for other includes (so this is exactly what one would want for installed headers, to ensure that they correctly include each other, and not accidentally include headers of the same name but part of some other project).

But in GAP, we use the angular bracket to allow our HPC-GAP kernel to build: For that, we currently use the trick that src contains a full kernel, but hpcgap/src only contains those kernel sources which are different between GAP and HPC-GAP. We then add -Ihpcgap -I. to the CPPFLAGS (in exactly that order), and things work correctly if and only if we use angular brackets for the #includes. (To get rid of the src/ we'd chang this to -Ihpcgap/src -Isrc).

So if we wanted to get rid of the angular brackets, we'd have to come up with another way to handle hpcgap mode. Note that keeping identical files in hpcgap/src is not an option -- we wanted to remove duplicates precisely to avoid code from starting to drift again.

One solution would be to simply wait until the two kernels have been merged, then remove the angular brackets again. But it's unclear how long that merge will take, and if it is not done in time for 4.9, we'd be stuck.

Another option might be to use symlinks liberally in HPC-GAP. E.g. we could symlink hpcgap/src/foo.h to gen/foo.h and likewise src/bar.h to gen/bar.h, and then have only -Igen. The thing I don't like about this is that it is potentially brittle -- we'd have to be careful to add/remove symlinks whenever files are removed or added to hpcgap/src (of course adding should rarely happen, but I'd still want to support it to support e.g. git bisect).

Other ideas?

[fingolfin]

For the symlinks, I am also a bit concerned about (a) how this will work on cygwin, and (b) what happens if we completely remove a header from GAP -- then the symlink stay in gen, which could perhaps cause trouble (or perhaps not - I can't think of any bad consequences, but I am wary nevertheless).

[fingolfin]

Note to self: we could use bash's [[ gen/foo.h -ef src/foo.h]] resp. [[ gen/foo.h -ef hpcgap/src/foo.h]] to verify whether a given symlink points at the right file, and update it if it does not.

Another note: I am not sure whether symlinks would work at all -- this needs to be tested (it will work only if the compiler does not resolve the symlink when it determines the containing directory). If it does not work, we could try hardlinks, but then we really get problems on at least Windows

[fingolfin]

More remarks the rewriting of includes is done by my script which re-creates the hpc-merge branch from time to time, so changing this would OK (though it still requires some manual work to adapt later commits, so I don't want to go back and forth a lot).

And I'd like to mention another alternative for installing the headers into PREFIX/include/gap: We could simply use sed during installation to change them from #include <src/foo.h> to #include "foo.h" on the fly. I cannot think of any problems with that -- am I missing something?

[markuspf]

The only issue I can see with seding is that the include files that are installed are different from the ones in the source tree, which might lead to some confusion.

One could of course put a warning at the top while one is at modifying those files.

If we wanted the symlink solution, then we'd have to automate and verify as much of it as we can, which seems a lot of work, if we could (at least transitionally?) go with just sed.

[fingolfin]

Yeah, the sed solution would be very easy to do -- and once the kernels have been merged, it could be removed and the #includes be reverted.

So I think I tend to just leaving the internal headers as <src/foo.h>, and I won't try to change them to <foo.h>, simply because then the sed call would be more complicated as it would have to avoid transforming stdio.h etc. etc.

[ChrisJefferson]

I agree we should leave as src/foo.h. I think putting the files nested inside /usr/include/gap/src people can just accept, and wonder about ;)

[fingolfin]

The (incomplete) make install I added now actually uses sed to rewrite the #include statements in our installed headers, which are put into $prefix/include/gap.

I'd therefore close this issue, except that some other things were discussed here that perhaps should be migrated to a new issue... if nobody takes care of this until I return from vacation, I'll look into it.

[fingolfin]

See https://github.com/gap-system/gap/pull/1443