Compilation error against PostgreSQL 12rc1

[demitri]

Hello,

I am trying to compile Q3C against PostgreSQL 12RC1 and I get this output:

gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -fPIC  -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/custom/postgresql-12rc1/include/server -I/usr/custom/postgresql-12rc1/include/internal  -D_GNU_SOURCE   -c -o prepare.o prepare.c
gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -fPIC  -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/custom/postgresql-12rc1/include/server -I/usr/custom/postgresql-12rc1/include/internal  -D_GNU_SOURCE   -c -o q3cube.o q3cube.c
gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -fPIC  -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/custom/postgresql-12rc1/include/server -I/usr/custom/postgresql-12rc1/include/internal  -D_GNU_SOURCE   -c -o q3c_poly.o q3c_poly.c
gcc -std=gnu99 prepare.o q3cube.o q3c_poly.o -lm -o prepare
q3cube.o: In function `q3c_dump_prm':
q3cube.c:(.text+0x4a72): undefined reference to `pg_fprintf'
q3cube.c:(.text+0x4a88): undefined reference to `pg_fprintf'
q3cube.c:(.text+0x4a99): undefined reference to `pg_fprintf'
q3cube.c:(.text+0x4ac0): undefined reference to `pg_fprintf'
q3cube.c:(.text+0x4ad5): undefined reference to `pg_fprintf'
q3cube.o:q3cube.c:(.text+0x4af5): more undefined references to `pg_fprintf' follow
collect2: error: ld returned 1 exit status
make: *** [prepare] Error 1

Is it possible that something changed somewhere in v12? I do see a change in the definition of fprint between 11.5 and 12.

11.5 (from installed directory)

% grep -r pg_fprintf
include/internal/port.h:167:extern int  pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2, 3);
include/internal/port.h:180:#define fprintf(...)    pg_fprintf(__VA_ARGS__)
include/internal/port.h:187:#define fprintf         pg_fprintf
include/server/port.h:167:extern int    pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2, 3);
include/server/port.h:180:#define fprintf(...)  pg_fprintf(__VA_ARGS__)
include/server/port.h:187:#define fprintf           pg_fprintf

12RC1:

include/internal/port.h:179:extern int  pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2, 3);
include/internal/port.h:196:#define fprintf         pg_fprintf
include/server/port.h:179:extern int    pg_fprintf(FILE *stream, const char *fmt,...) pg_attribute_printf(2, 3);
include/server/port.h:196:#define fprintf           pg_fprintf
include/server/plpython.h:161:#define fprintf           pg_fprintf

And this may or may not be helpful.

This is on CentOS Linux with gcc v. 4.8.5. The output on macOS 10.14:

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2   -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/local/postgresql-12rc1/include/server -I/usr/local/postgresql-12rc1/include/internal  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -I/usr/local/readline-8.0/include  -c -o prepare.o prepare.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2   -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/local/postgresql-12rc1/include/server -I/usr/local/postgresql-12rc1/include/internal  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -I/usr/local/readline-8.0/include  -c -o q3cube.o q3cube.c
q3cube.c:1328:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long') [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", xbits[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~
q3cube.c:1341:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long') [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", ybits[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~
q3cube.c:1354:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long') [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", xbits1[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~~
q3cube.c:1367:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long') [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", ybits1[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~~
q3cube.c:1372:64: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long') [-Wformat]
        Q3C_IPIX_FMT",____xbits,____ybits,____xbits1,____ybits1};\n", hprm->nside);
        ~~~~~~~~~~~~                                                  ^~~~~~~~~~~
5 warnings generated.
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2   -O3 -D_GNU_SOURCE -D__STDC_FORMAT_MACROS -DQ3C_VERSION='"'1.8.0'"' -I. -I./ -I/usr/local/postgresql-12rc1/include/server -I/usr/local/postgresql-12rc1/include/internal  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -I/usr/local/readline-8.0/include  -c -o q3c_poly.o q3c_poly.c
gcc prepare.o q3cube.o q3c_poly.o -lm -o prepare
Undefined symbols for architecture x86_64:
  "_pg_fprintf", referenced from:
      _q3c_dump_prm in q3cube.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [prepare] Error 1

[segasai]

Thanks for reporting. The diff below seems to fix it. But I'll wait applying it before I can test pg12 on travis.

diff --git a/Makefile b/Makefile
index 2119c32..5984c26 100644
--- a/Makefile
+++ b/Makefile
@@ -28,10 +28,10 @@ dump.c: prepare
        ./prepare

 prepare: prepare.o q3cube.o q3c_poly.o
-       $(CC) prepare.o q3cube.o q3c_poly.o $(MYBINLIBS) -o $@
-              
+       $(CC) prepare.o q3cube.o q3c_poly.o $(PG_LDFLAGS) $(LDFLAGS) $(LIBS) $(MYBINLIBS) -o $@
+
 gen_data: gen_data.c
-       $(CC) $< $(CPPFLAGS) $(MYBINLIBS) -o $@
+       $(CC) $< $(CPPFLAGS)$ $(PG_LDFLAGS) $(MYBINLIBS) -o $@

 test: gen_data all
        createdb q3c_test
diff --git a/q3c.c b/q3c.c
index 6f1460c..84033d4 100644
--- a/q3c.c
+++ b/q3c.c
@@ -37,7 +37,7 @@
 #if PG_VERSION_NUM >= 90300
 #include "access/tupmacs.h"
 #endif
-#include "nodes/relation.h"
+//#include "nodes/relation.h"
 #include "utils/selfuncs.h"

[segasai]

I have finally been able to fix the issue with the compilation and PG 12. After the commit 3824f797fdbdf97dbce1782af7c189441d1e8ad5 (and previous ones) it should now compile without problems.

[demitri]

Thanks! I've been using the patch above for over a week with no problems.

[segasai]

The problem was also that on ubuntu-based systems (like the travis ones), some libraries were moved, so my initial patch didn't actually work, so I had to figure out how to deal with that.

[demitri]

Ah- I was installing it on CentOS, which did work for me.

[demitri]

Hi - I am having a problem with the current Makefile and the 1.8.1 release, getting the same error:

Undefined symbols for architecture x86_64:
  "_pg_fprintf", referenced from:
      _q3c_dump_prm in q3cube.o
ld: symbol(s) not found for architecture x86_64

This is on macOS 10.14 with both PostgreSQL 12.0 and 12.1. Based on the above patch, the Makefile is missing $(LIBS) as shown in the patch above under the line:

prepare: prepare.o q3cube.o q3c_poly.o

Returning the variable fixes the problem.

Also (and it's a minor quibble), in the 1.8.1 release, the version number is still "1.8.0".

-DQ3C_VERSION='"'1.8.0'"'

I haven't had a chance to test this on Linux.

[segasai]

Thanks for the report Demitri. Could you please send a patch/pull request against current master, so I can see if it passes the tests. Since I don't have mac, I can't test the patch myself.

[demitri]

Done, thanks!

[segasai]

Thanks, and you can indeed see that many of tests failed. To develop the pg12 patch I spent a lot of time trying to ensure that PG from 9 to 12 compiles. Since I indeed don't have mac, I don't think I can myself fix your patch (I'll likely break mac in the process). You may want to use the if/then clause I have in the makefile for the 12 vs earlier PG version that I use to assemble different library sets.

[segasai]

Thanks for the working patch. But while looking at it I've got the impression that the only problem on Mac was a not-working version determination, rather than the flags. As the MYBINLIBS should have had the LIBS inside it already. So could you please check the current HEAD ? I think it should work fine.

[demitri]

I grabbed the new version and it compiled without a problem. I haven't done a full test (short of time at the moment), but I would guess it's fine. As an FYI, I do get these warnings (they are not new):

q3cube.c:1328:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long')
      [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", xbits[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~
q3cube.c:1341:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long')
      [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", ybits[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~
q3cube.c:1354:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long')
      [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", xbits1[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~~
q3cube.c:1367:44: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long')
      [-Wformat]
                fprintf(fp, "Q3C_CONST("Q3C_IPIX_FMT")", ybits1[i]);
                                        ~~~~~~~~~~~~     ^~~~~~~~~
q3cube.c:1372:64: warning: format specifies type 'long long' but the argument has type 'q3c_ipix_t' (aka 'long')
      [-Wformat]
        Q3C_IPIX_FMT",____xbits,____ybits,____xbits1,____ybits1};\n", hprm->nside);
        ~~~~~~~~~~~~                                                  ^~~~~~~~~~~
5 warnings generated.

I poked around a bit to see what might be the problem, but didn't find anything obvious. I don't think it's a problem. I didn't see this on CentOS.

Anyway, thanks for the help!

[segasai]

Thanks for checking. (regarding the warnings, I am aware of them, but I don't know an easy fix without hard-coding platform depence; and since it doesn't lead to any issues, I'll skip fixing them)

[segasai]

I also put out the v.1.8.2 version which incorporates the fix