search

apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.46k stars 3.7k forks source link

Scheduling error to find already existing meta tables when using postgresql #7137

Open JinseonyLee opened 5 years ago

JinseonyLee commented 5 years ago

Description

When using postgresql, I found a case where an error occurred during scheduling that check whether a meta table existed. The reason is that the schema for the db user is only allowed to be public, which is the default value. Therefore, I think it would be better to apply the db table schema as a configuration property.

The error log is shown below. ` Caused by: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: org.postgresql.util.PSQLException: ERROR: relation "druid_config" does not exist Position: 21 [statement:"SELECT payload FROM druid_config WHERE name = :key", located:"SELECT payload FROM druid_config WHERE name = :key", rewritten:"SELECT payload FROM druid_config WHERE name = ?", arguments:{ positional:{}, named:{key:'worker.config'}, finder:[]}] at org.skife.jdbi.v2.SQLStatement.internalExecute(SQLStatement.java:1334) ~[jdbi-2.63.1.jar:2.63.1] at org.skife.jdbi.v2.Query.fold(Query.java:173) ~[jdbi-2.63.1.jar:2.63.1] at org.skife.jdbi.v2.Query.list(Query.java:82) ~[jdbi-2.63.1.jar:2.63.1] at org.skife.jdbi.v2.Query.list(Query.java:75) ~[jdbi-2.63.1.jar:2.63.1] at org.apache.druid.metadata.SQLMetadataConnector.lookupWithHandle(SQLMetadataConnector.java:617) ~[druid-server-0.13.0-incubating.jar:0.13.0-incubating] at org.apache.druid.metadata.SQLMetadataConnector$5.withHandle(SQLMetadataConnector.java:595) ~[druid-server-0.13.0-incubating.jar:0.13.0-incubating] at org.apache.druid.metadata.SQLMetadataConnector$5.withHandle(SQLMetadataConnector.java:591) ~[druid-server-0.13.0-incubating.jar:0.13.0-incubating] at org.skife.jdbi.v2.DBI.withHandle(DBI.java:281) ~[jdbi-2.63.1.jar:2.63.1] ... 13 more

`

egor-ryashin commented 5 years ago

Could you describe steps to reproduce?

JinseonyLee commented 5 years ago

Could you describe steps to reproduce?

When using DB, administrators set the search path for each user because of security.

  1. So first I installed postgresql, created the db and set the search path to the user role. ALTER ROLE dbuser IN DATABASE druiddb SET search_path = druid;
  2. After that, I set postgresql to druid.
  3. Finally, when you start druid, you can see that the above error occurs according to the scheduling. This is because previous versions only read the schema as 'public'.

So I changed the source to read the schema information as a setting value. It was applied to the master branch.

egor-ryashin commented 5 years ago

I guess that can be solved on the database level setting the correct schema, for example:

ALTER TABLE prod_segments
    SET SCHEMA druid;
JinseonyLee commented 5 years ago

I guess that can be solved on the database level setting the correct schema, for example:

ALTER TABLE prod_segments
    SET SCHEMA druid;

I checked the source code of the scheduler and it only worked if it was 'druid'. As you said, it seems to work by changing the schema of every druid meta table one by one. However, because there are many tables in addition to 'prod_segments', I thought it was right to specify a schema for druid access. So I modified the configurable source code.