DB does not exist exception when using tap-postgres

[gagarinfan]

Hi!

I'm struggling with loading data from Postgres (AWS RDS, version 14.10) to Clickhouse (self-hosted, v24.8.4)

While using FULL_TABLE with the setup below, I'm getting an exception that the Database does not exist. The problem is that Clickhouse target is trying to put the data to a database name from Postgres (source) schema some_source_schema instead of meltano from CH section.

Basically I want to replicate:

• document table from some_source_schema from Postgres TO
• document table from meltano database in Clickhouse

version: 1
default_environment: dev
project_id: d5635ec6-c458-487a-8086-4633c6195398
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: git+https://github.com/MeltanoLabs/tap-postgres.git
    config:
      database: some_database
      default_replication_method: FULL_TABLE
      host: <REDACTED>
      port: 5432
      user: meltano
      filter_schemas:
      - some_source_schema
    select:
      - some_source_schema-document.*

  loaders:
  - name: target-clickhouse
    variant: shaped-ai
    pip_url: git+https://github.com/shaped-ai/target-clickhouse.git
    config:
      database: meltano
      host: <REDACTED>
      port: 8123
      username: meltano

Here is the error log I get:

clickhouse_sqlalchemy.exceptions.DatabaseException: Orig exception: Code: 81. DB::Exception: Database some_source_schema does not exist. (UNKNOWN_DATABASE) (version 24.8.4.13 (official build))

When I create the DB in Clickhouse, then I'm getting:

Orig exception: Code: 60. DB::Exception: Table some_source_schema.document does not exist. Maybe you meant meltano.document?

Interestingly, table in meltano database is properly created with columns matching source (Postgres)

[techtangents]

I'm getting the same issue.

[edgarrmondragon]

From the Meltano Slack:

Setting default_target_schema seems to fix this.

https://meltano.slack.com/archives/C069CQNHDNF/p1728702228614369