Support request for pgsql-13.x

[Saketha016]

Hi! As PostgreSQL 13 offers significant improvements in performance and usability, want to know whether any plan to support pgsql-13 version in the future.

[matthewfranglen]

Yeah the thing I've been waiting for is a docker image that has multicorn support. The code should work if you're happy to set that up yourself. You can see the pg-13 branch if you want to see what I was trying.

[matthewfranglen]

The current postgres:13 docker image is debian 10.9 (buster) while postgresql-13-python3-multicorn is in 11 (sid)

[matthewfranglen]

The fdw should work with postgres 13. I'm just not advertising that due to being unable to test it.

[Saketha016]

Yes, I tried with bitnami/postgresql 13 chart, but no luck, maybe I was missing something there...

But Anyhow. Thanks, Matthew for your quick response. Looking forward for an update on this from you. :)

[matthewfranglen]

Since you have been so nice about this I've had another look. I was able to install the required package from sid onto buster (which is a bad idea in general - I would not recommend this for production use). The fdw does work.

For your specific use case I recommend installing multicorn on the virtual machine. You may have to do it from source. See the documentation for details.

If this resolves your issue then please close this ticket. I may be able to help you if you have trouble setting up the fdw.

[Saketha016]

Hello Matthew,

I have used Kubernetes to set up postgresql-ha 13.2 version. And as you suggested, I tried installing Multicorn from the source, but It was failing to install and showing up errors.

root@postgresql-postgresql-ha-postgresql-1:/opt# pip --version
pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)
root@postgresql-postgresql-ha-postgresql-1:/opt# 
root@postgresql-postgresql-ha-postgresql-1:/opt# python --version
Python 3.7.3
root@postgresql-postgresql-ha-postgresql-1:/opt# 
root@postgresql-postgresql-ha-postgresql-1:/opt# psql --version
psql (PostgreSQL) 13.2
root@postgresql-postgresql-ha-postgresql-1:/opt# 
root@postgresql-postgresql-ha-postgresql-1:/opt# git clone git://github.com/Kozea/Multicorn.git
Cloning into 'Multicorn'...
remote: Enumerating objects: 15733, done.
remote: Total 15733 (delta 0), reused 0 (delta 0), pack-reused 15733
Receiving objects: 100% (15733/15733), 5.93 MiB | 1.26 MiB/s, done.
Resolving deltas: 100% (7607/7607), done.
root@postgresql-postgresql-ha-postgresql-1:/opt# 
root@postgresql-postgresql-ha-postgresql-1:/opt# cd Multicorn/
root@postgresql-postgresql-ha-postgresql-1:/opt/Multicorn# 
root@postgresql-postgresql-ha-postgresql-1:/opt/Multicorn# make && make install
Python version is 3.7
[ -d sql ] || mkdir sql
[ -d src ] || mkdir src
touch directories.stamp
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -s -fPIC -I/usr/include/python3.7m -I/usr/include/python3.7m  -I. -I./ -I/opt/bitnami/postgresql/include/server -I/opt/bitnami/postgresql/include/internal  -I/opt/bitnami/common/include -I/opt/bitnami/repmgr/include -I/opt/bitnami/postgresql/include -D_GNU_SOURCE -I/usr/include/libxml2   -c -o src/errors.o src/errors.c
src/errors.c: In function ‘reportException’:
src/errors.c:71:6: error: too many arguments to function ‘errstart’
  if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN))
      ^~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:149:13: note: declared here
 extern bool errstart(int elevel, const char *domain);
             ^~~~~~~~
src/errors.c:73:7: error: too many arguments to function ‘errstart’
   if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN))
       ^~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:149:13: note: declared here
 extern bool errstart(int elevel, const char *domain);
             ^~~~~~~~
src/errors.c:84:2: error: too few arguments to function ‘errfinish’
  errfinish(0);
  ^~~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:150:13: note: declared here
 extern void errfinish(const char *filename, int lineno, const char *funcname);
             ^~~~~~~~~
make: *** [<builtin>: src/errors.o] Error 1
root@postgresql-postgresql-ha-postgresql-1:/opt/Multicorn# 

Couldn't figure out what's wrong in the make file. Appreciate your help !!

[matthewfranglen]

You could try installing it with pgxn: https://pgxn.github.io/pgxnclient/install.html

pip install pgxnclient
pgxn install multicorn

[Saketha016]

Sorry Matthew, even "pgxn install multicorn" didn't work:

root@postgresql-postgresql-ha-postgresql-1:/opt/Multicorn# pgxn install multicorn
INFO: best version: multicorn 1.4.0
INFO: saving /tmp/tmpsrg6ilo1/multicorn-1.4.0.zip
INFO: unpacking: /tmp/tmpsrg6ilo1/multicorn-1.4.0.zip
INFO: building extension
Python version is 3.7
[ -d sql ] || mkdir sql
[ -d src ] || mkdir src
touch directories.stamp
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -s -fPIC -I/usr/include/python3.7m -I/usr/include/python3.7m  -I. -I./ -I/opt/bitnami/postgresql/include/server -I/opt/bitnami/postgresql/include/internal  -I/opt/bitnami/common/include -I/opt/bitnami/repmgr/include -I/opt/bitnami/postgresql/include -D_GNU_SOURCE -I/usr/include/libxml2   -c -o src/errors.o src/errors.c
src/errors.c: In function ‘reportException’:
src/errors.c:71:6: error: too many arguments to function ‘errstart’
  if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN))
      ^~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:149:13: note: declared here
 extern bool errstart(int elevel, const char *domain);
             ^~~~~~~~
src/errors.c:73:7: error: too many arguments to function ‘errstart’
   if (errstart(severity, __FILE__, __LINE__, PG_FUNCNAME_MACRO, TEXTDOMAIN))
       ^~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:149:13: note: declared here
 extern bool errstart(int elevel, const char *domain);
             ^~~~~~~~
src/errors.c:84:2: error: too few arguments to function ‘errfinish’
  errfinish(0);
  ^~~~~~~~~
In file included from /opt/bitnami/postgresql/include/server/postgres.h:47,
                 from src/multicorn.h:2,
                 from src/errors.c:15:
/opt/bitnami/postgresql/include/server/utils/elog.h:150:13: note: declared here
 extern void errfinish(const char *filename, int lineno, const char *funcname);
             ^~~~~~~~~
make: *** [<builtin>: src/errors.o] Error 1
ERROR: command returned 2: make PG_CONFIG=/opt/bitnami/postgresql/bin/pg_config all
root@postgresql-postgresql-ha-postgresql-1:/opt/Multicorn# 

[matthewfranglen]

:( I'm not sure what you can do. I can't really reproduce that as I don't know what docker image you are using. Maybe you could open a ticket with the multicorn project itself?

https://github.com/Segfault-Inc/Multicorn/issues

... I see you've already done that heh. Good luck!

[Saketha016]

Yes Matthew, I was about to post that here for ur reference. https://github.com/Segfault-Inc/Multicorn/issues/261

There seems to be an issue with the Multicorn setup itself. But anyhow will wait for their response. Thanks for ur time, Matthew.

[luss]

FWIW. I'm a big fan of this extension and I've also been waiting for. pg13. I've running into the same Multicorn roadblock. Does anyone know the core developers of Multicorn?

I'm copying my friend Jan Wieck on this thread. I know he's a multicorn fan and I know he's techie enough to fix whatever Python3/Pg13 multcorn issue is the blocker if we need to fork multicorn so that esfdw may live.

--Denis

PS For those who haven't heard of Jan. He's a bit of a Postgres legend (at least in his own mind) :-). Seriously, Jan is an Emeritus Core team member and is the original author of several cornerstone features of Postgres such as PLpgSQL, Toast, Referential Integrity, ... Jan, as you can see, I'm trying to win you over to help us with public flattery. Please do not let it go to your head. ;-)

On Fri, Apr 9, 2021 at 2:04 PM Saketha016 @.***> wrote:

Yes Matthew, I was about to post that here for ur reference. Segfault-Inc/Multicorn#261 https://github.com/Segfault-Inc/Multicorn/issues/261

There seems to be an issue with the Multicorn setup itself. But anyhow will wait for their response. Thanks for ur time, Matthew.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/matthewfranglen/postgres-elasticsearch-fdw/issues/24#issuecomment-816859610, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMWOHR6IZE23VIKROQC52TTH46RVANCNFSM42TMTV5A .

[luss]

I found the exact same PG-13 incompatibilities that are in the pull request https://github.com/Segfault-Inc/Multicorn/pull/260 With those I can build and install on CentOS 8 and then successfully CREATE EXTENSION multicorn;

Any idea what is holding that up? There is some mention about random errors, but I couldn't find anything specific about what those errors are.

Regards, Jan

On 4/10/21 12:23 AM, Denis Lussier wrote:

FWIW.  I'm a big fan of this extension and I've also been waiting for. pg13.   I've running into the same Multicorn roadblock.   Does anyone know the core developers of Multicorn?

I'm copying my friend Jan Wieck on this thread.  I know he's a multicorn fan and I know he's techie enough to fix whatever Python3/Pg13 multcorn issue is the blocker if we need to fork multicorn so that esfdw may live.

--Denis

PS For those who haven't heard of Jan.  He's a bit of a Postgres legend (at least in his own mind) :-).    Seriously, Jan is an Emeritus Core team member and is the original author of several cornerstone features of Postgres such as PLpgSQL, Toast, Referential Integrity, ...    Jan, as you can see, I'm trying to win you over to help us with public flattery.   Please do not let it go to your head.  ;-)

On Fri, Apr 9, 2021 at 2:04 PM Saketha016 @. @.>> wrote:

Yes Matthew, I was about to post that here for ur reference.
Segfault-Inc/Multicorn#261
<https://github.com/Segfault-Inc/Multicorn/issues/261>

There seems to be an issue with the Multicorn setup itself. But
anyhow will wait for their response. Thanks for ur time, Matthew.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/matthewfranglen/postgres-elasticsearch-fdw/issues/24#issuecomment-816859610>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMWOHR6IZE23VIKROQC52TTH46RVANCNFSM42TMTV5A>.

-- Jan Wieck Postgres User since 1994

[matthewfranglen]

Thanks to the work of @luss, this ticket can be resolved. The recommended way to install this FDW on PostgreSQL 13+ is to use Multicorn 2. Please review the installation instructions for that repo.