chumaky / docker-images

Postgres database with different foreign data wrapper extensions installed. Datero data platform engine image.
https://datero.tech
MIT License
30 stars 11 forks source link

Error server #2

Open Diamondra-Rabenaivo opened 3 months ago

Diamondra-Rabenaivo commented 3 months ago

I've installed the docker image JDBC_FDW and I've run the setup postgres sql Script. It gives me "ERROR: remote server returned an error". I've already copied the jar file in a directory :drivers/postgresql....jar

AmebaBrain commented 3 months ago

In the tests/jdbc folder there are two *.jar files are present. When you do repo checkout please make sure that you have git lfs installed locally. If not, you will checkout only hashes of the files.

$ git lfs ls-files
a221c4106b * tests/jdbc/mysql-connector-j-9.0.0.jar
a2644cbfba * tests/jdbc/postgresql-42.7.3.jar

$ ls -lh tests/jdbc/*.jar
-rw-r--r-- 1 user group 2,5M чер  9 04:15 tests/jdbc/mysql-connector-j-9.0.0.jar
-rw-rw-r-- 1 user group 1,1M бер 14 19:41 tests/jdbc/postgresql-42.7.3.jar

If everything is ok with that, then you just execute docker-compose -f tests/postgres_jdbc_compose.yml up -d from the repo root and you should end up with postgres_jdbc_fdw container up and running. You could start psql from within the container and check that 2 foreign servers have been created using jdbc_fdw extension. And for each of these servers there is a foreign table has been created as well.

$ docker exec -it postgres_jdbc_fdw psql postgres postgres
psql (16.3 (Debian 16.3-1.pgdg120+1))
Type "help" for help.

postgres=# \des
           List of foreign servers
    Name    |  Owner   | Foreign-data wrapper 
------------+----------+----------------------
 rfam       | postgres | jdbc_fdw
 rnacentral | postgres | jdbc_fdw
(2 rows)

postgres=# \det
       List of foreign tables
 Schema |     Table     |   Server   
--------+---------------+------------
 public | rfam_taxonomy | rfam
 public | rnacen_xref   | rnacentral
(2 rows)

These are publicibly available postgres and mysql readonly databases. Because they are available over the public internet they have some throttling mechanism implemented. At the time of my tests, one of the queries works without problem, the other one returns an error.

This works

postgres=# select * from rfam_taxonomy limit 10;

But this returns mentioned by you error

postgres=# select * from rnacen_xref limit 10;
ERROR:  remote server returned an error

But if you will have a look onto the container logs, you will see

Exception in thread "Thread-2" org.postgresql.util.PSQLException: FATAL: too many connections for role "reader"
...
2024-07-29 22:11:32.635 UTC [137] ERROR:  remote server returned an error
2024-07-29 22:11:32.635 UTC [137] STATEMENT:  select * from rnacen_xref limit 10;

I.e. there is public access throttling is happening.

But you are right that jdbc_fdw very often returns this generic remote server returned an error message. Which is not too much useful. Current jdbc_fdw version is 0.4.0. I.e. it's in very raw state. And I was observing a few other situations, when it was returning such error. But in all cases, it's not related to this docker image. It's just this extension current code state.

This is one of the main reasons, why I didn't include it yet into all-inclusive datero engine image.