[bitnami/clickhouse] Support CLICKHOUSE_DB in clickhouse container

snicoll commented 4 weeks ago

Name and Version

bitnami/clickhouse:latest

What architecture are you using?

arm64

What steps will reproduce the bug?

Setting 'CLICKHOUSE_DB=mydatabase' should create a mydatabase. That's what the regular image does but, for some reason, the bitnami version does not.

Looking at the doc, it seems that it is intentional.

What is the expected behavior?

mydatabase should be created.

What do you see instead?

reactor.core.Exceptions$ReactiveException: com.clickhouse.client.ClickHouseException: Code: 81. DB::Exception: Database mydatabase does not exist. (UNKNOWN_DATABASE) (version 24.3.12.75 (official build))

    at reactor.core.Exceptions.propagate(Exceptions.java:410)
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:144)
    at reactor.core.publisher.Flux.blockFirst(Flux.java:2793)
    at org.springframework.boot.docker.compose.service.connection.clickhouse.ClickHouseR2dbcDockerComposeConnectionDetailsFactoryIntegrationTests.checkDatabaseAccess(ClickHouseR2dbcDockerComposeConnectionDetailsFactoryIntegrationTests.java:66)
    at org.springframework.boot.docker.compose.service.connection.clickhouse.ClickHouseR2dbcDockerComposeConnectionDetailsFactoryIntegrationTests.runWithBitnamiImageCreatesConnectionDetails(ClickHouseR2dbcDockerComposeConnectionDetailsFactoryIntegrationTests.java:49)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:146)
        ... 6 more
Caused by: com.clickhouse.client.ClickHouseException: Code: 81. DB::Exception: Database mydatabase does not exist. (UNKNOWN_DATABASE) (version 24.3.12.75 (official build))

    at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:151)
    at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.IOException: Code: 81. DB::Exception: Database mydatabase does not exist. (UNKNOWN_DATABASE) (version 24.3.12.75 (official build))

    at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:203)
    at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:246)
    at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:201)
    at com.clickhouse.client.AbstractClient.sendAsync(AbstractClient.java:161)
    at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:273)
    ... 4 more

Additional information

I've tried to setup a script that creates the database, but that didn't work either.

I created an initdb folder with the following create-database.sh file (it is executable):

#!/bin/bash
set -e

clickhouse client --password $CLICKHOUSE_ADMIN_PASSWORD -n <<-EOSQL
  CREATE DATABASE mydatabase ENGINE = Memory COMMENT 'Test database';
EOSQL

My docker compose file is as follows:

services:
  database:
    image: 'bitnami/clickhouse:latest'
    ports:
      - '8123'
    environment:
      - 'CLICKHOUSE_USER=myuser'
      - 'CLICKHOUSE_PASSWORD=secret'
      - 'CLICKHOUSE_DB=mydatabase'
    volumes:
      - ./initdb:/docker-entrypoint-initdb.d

On startup, I am getting:

clickhouse 15:27:02.48 INFO  ==> 
2024-10-23T15:27:02.485869793Z clickhouse 15:27:02.48 INFO  ==> Welcome to the Bitnami clickhouse container
2024-10-23T15:27:02.486760543Z clickhouse 15:27:02.48 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
2024-10-23T15:27:02.489340751Z clickhouse 15:27:02.48 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
2024-10-23T15:27:02.489353960Z clickhouse 15:27:02.48 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
2024-10-23T15:27:02.489651085Z clickhouse 15:27:02.48 INFO  ==> 
2024-10-23T15:27:02.491965751Z clickhouse 15:27:02.49 INFO  ==> ** Starting ClickHouse setup **
2024-10-23T15:27:02.510218918Z clickhouse 15:27:02.50 INFO  ==> Starting ClickHouse in background
2024-10-23T15:27:07.515692295Z clickhouse 15:27:07.51 INFO  ==> ClickHouse started successfully
2024-10-23T15:27:07.516232337Z clickhouse 15:27:07.51 INFO  ==> Loading user's custom files from /docker-entrypoint-initdb.d
2024-10-23T15:27:10.556401588Z clickhouse 15:27:10.55 INFO  ==> ** ClickHouse setup finished! **
2024-10-23T15:27:10.556456047Z 
2024-10-23T15:27:10.562726213Z clickhouse 15:27:10.56 INFO  ==> ** Starting ClickHouse **

javsalgar commented 3 weeks ago

Hi!

Could you add some debug output to ensure that the init script is being executed?

snicoll commented 3 weeks ago

Could you add some debug output to ensure that the init script is being executed?

I can do that, but that's just a workaround for the reported issue. My ask is whether not supporting CLICKHOUSE_DB is intentional. If it is, what is the recommended way to create a custom DB on startup?

javsalgar commented 3 weeks ago

Hi,

Yes, in principle it was intentional to avoid adding too much complexity to the initial script. I could also see that the database creation has several options (engines, clustered or not clustered, sharding...) that we weren't sure a default database would cover the user needs. At the moment, the recommended way is to mount your custom init scripts to ensure that the database has the expected settings.

However, if this is something that the community is very interested, it could be possible to change the clickhouse_initialize function to support the CLICKHOUSE_DB param. I will leave the ticket open to see if we have more feedback from the community.

bitnami / containers