matthewfranglen / postgres-elasticsearch-fdw

Postgres to Elastic Search Foreign Data Wrapper
MIT License
108 stars 32 forks source link

Multicorn2 now offers support for pg15 #33

Closed luss closed 2 months ago

luss commented 1 year ago

please now test esfdw on pg15.

matthewfranglen commented 1 year ago

Sorry about the delay! I'll have a look at this now.

matthewfranglen commented 1 year ago

So, I have a PostgreSQL 15beta4 Dockerfile that I can install the foreign data wrapper in using Multicorn 2. When I run the tests I get a very unusual result. The delete test for this fails. The foreign data wrapper delete function is invoked through the statement:

DELETE FROM articles_es_returning
WHERE id = 39357158
RETURNING id, title;

(file)

The schema for the foreign data wrapper indicates that the id column is id. Looking at the Multicorn 1 documentation the requirements for implementing the write api is:

In multicorn, this involves defining which column will be used as a primary key (mandatory) and implementing the following methods at your discretion:

def insert(self, new_values) def update(self, rowid, new_values) def delete(self, rowid)

The error message indicates that the delete method is being invoked with a rowid of 0 instead of 39357158. This leads to the error:

PostgreSQL 14 with Elasticsearch 7: Test delete returning row - FAIL
WARNING:  SEARCH for /article-index row_id 0 returned nothing
ERROR:  DELETE for /article-index/0 failed: NotFoundError(404, '{"_index":"article-index","_type":"_doc","_id":"0","_version":1,"result":"not_found","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103,"_primary_term":1}') 

(code)

It also fails in PostgreSQL 14 but it passes in PostgreSQL 13, both of which use Multicorn 2.

The failing images (PostgreSQL 14 and 15beta4) require clang-11 as one of the apt dependencies in order to compile Multicorn 2. What's interesting is that the PostgreSQL 13 does not require this dependency and I have rebuilt it from scratch to check this.

matthewfranglen commented 2 months ago

0.12.0 uses multicorn 2 to manage the extension for PostgreSQL versions 13, 14, and 15.