autogen.sh fails - variable 'size' set but not used

[rotten]

Whether I run autogen.sh --with-master or --with-legacy, it runs for a while and then fails with the following:

/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.    -Wall -Werror -Wno-error=deprecated-declarations -Wextra -Wwrite-strings -Wno-unused-parameter -std=gnu99 -D_GNU_SOURCE -D_REENTRANT -g -O2 -MT json_tokener.lo -MD -MP -MF .deps/json_tokener.Tpo -c -o json_tokener.lo json_tokener.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -Wall -Werror -Wno-error=deprecated-declarations -Wextra -Wwrite-strings -Wno-unused-parameter -std=gnu99 -D_GNU_SOURCE -D_REENTRANT -g -O2 -MT json_tokener.lo -MD -MP -MF .deps/json_tokener.Tpo -c json_tokener.c  -fPIC -DPIC -o .libs/json_tokener.o
json_tokener.c: In function 'json_tokener_parse_ex':
json_tokener.c:355:6: error: variable 'size' set but not used [-Werror=unused-but-set-variable]
  int size;
      ^
cc1: all warnings being treated as errors
Makefile:572: recipe for target 'json_tokener.lo' failed
make[1]: *** [json_tokener.lo] Error 1
make[1]: Leaving directory '/home/rotten/src/mongo_fdw/json-c'
Makefile:636: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1

FWIW, I'm on Ubuntu 16.04. I'm using the latest code from the github repo's master branch.

[rotten]

The above is when I answer the "Assume -R"? question with a y.

If I try this combinations of answers:

Reversed (or previously applied) patch detected!  Assume -R? [n] n
Apply anyway? [n] y
Hunk #1 FAILED at 352.
1 out of 1 hunk FAILED -- saving rejects to file json_tokener.c.rej

Other than the FAILED message I included above, it looks like autogen.sh completes successfully.

However, when I try to install the freshly built code I get this:

$  sudo make install
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer -fPIC -shared -o mongo_fdw.so connection.o option.o mongo_wrapper_meta.o mongo_fdw.o mongo_query.o json-c/json_util.o json-c/json_object.o json-c/json_tokener.o json-c/json_object_iterator.o json-c/printbuf.o json-c/linkhash.o json-c/arraylist.o json-c/random_seed.o json-c/debug.o -L/usr/lib/x86_64-linux-gnu -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now  -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,--as-needed  -L/usr/local/lib -lssl -lcrypto -lrt -lmongoc-1.0 -lbson-1.0 
/usr/bin/ld: json-c/json_util.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
json-c/json_util.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
/usr/lib/postgresql/10/lib/pgxs/src/makefiles/../../src/Makefile.shlib:286: recipe for target 'mongo_fdw.so' failed
make: *** [mongo_fdw.so] Error 1

[rotten]

I think I got it to work. I recloned without the "--recursive" flag. I made sure I had these Ubuntu packages installed:

# apt install libbson-1.0-0 libbson-dev
# apt install libmongoc-1.0-0 libmongoc-dev

I ran: $ make -f Makefile.meta

and then: $ sudo make -f Makefile.meta install

I'll do some testing and close this thread if the extension seems to be working.

[rotten]

I've set up the server, user mappings, and a foreign table without any trouble. When I query the foreign table, I can see postgresql connecting to my mongodb cluster successfully. However, no data is returned. I get an empty result set.

I tried setting postgresql log level to debug1 and debug2. No error is evident. Searching this and the parent citus repository does not turn up an obvious way to get debug messages which might explain why no data was being returned.

I know there is data in the table because I can see it with my multicorn based yam_fdw fork (which I was hoping to retire).

[gabbasb]

Please recheck the names in the create foreign table statement. Make sure you are using the correct database name and correct collection name.

[rotten]

The issue causing the tables to have zero rows (with no errors anywhere) had to do with the ssl configuration. Apparently an ssl misconfiguration causes silent failures.

The other challenge is that if you assign "jsonb" to column types, the columns come back empty (with no warning or anything). However if you switch them to "json" column types, they work fine. Since I almost always use materialized views in front of the foreign tables, it wasn't hard to go through and cast the json to jsonb in the materialized view definitions.

Lastly, anywhere that you have a ObjectId() exposed at the column level, you have to use then "NAME" column type (instead of VARCHAR/CHAR/TEXT). Otherwise you just get empty data for that column. Again, with no warnings or any other indication that anything is wrong.

It is simple to cast them to character types in the materialized view definitions also.

I'm closing this ticket since I seem to have everything working now. Perhaps my comments in this issue will help others who are trying this FDW for the first time.

---

Oh - and to elaborate on the driving reason for us to migrate off of my yam_fdw fork - There is a bug in PostgreSQL 9.x which causes the database to crash if you are using the Mulitcorn FDW library and parallel queries at the same time. It isn't something the Multicorn folks can fix. The PostgreSQL team put a patch in shortly after I reported the bug, but the next incremental point release (a couple of months ago) undid the patch causing the crashes again. The bug may be present with other Extensions as well, they said they'd be able to get a permanent fix in PG 11.

My hope is that the Enterprise DB FDW implementation will not be vulnerable to this problem. Also, by moving to PG 10.1, we can continue to press for a permanent fix to the crashing issue. (For our production database it is happening every day. We can't live without parallel queries and we can't live without a MongoDB FDW at this time.)