ksql-migrations: `--define` substitution not happening.

[dailytabs]

Setup Details broker: confluentinc/cp-enterprise-kafka:6.2.0 ksqldb: confluentinc/cp-ksqldb-server:6.2.0 migrations: confluentinc/ksqldb-cli:0.18.0

Migration File

CREATE STREAM MY_STREAM (
    ...
) WITH (
    KAFKA_TOPIC='${KAFKA_TOPIC}',
    ...
)

Apply Command

ksql-migrations --config-file /var/migrations/ksql-migrations.properties apply --all --define KAFKA_TOPIC=my-topic

Output

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Validating current migration state before applying new migrations
Loading migration files
1 migration file(s) loaded.
Applying migration version 1: Add stuff
/var/migrations/migrations/V000001__Add_stuff.sql contents:
CREATE STREAM MY_STREAM (
    ...
) WITH (
    KAFKA_TOPIC = '${KAFKA_TOPIC}',
    ...
);. Error: io.confluent.ksql.api.client.exception.KsqlClientException: Received 500 response from server: Failed to check if exists for topic: ${KAFKA_TOPIC}
Caused by: The request attempted to perform an operation on an invalid topic.. Error code: 50000
Execution time: 0.9960 seconds

Question Appears as though the substitution is not happening: I expected the console dump to include the substituted text, as well as the error message. Works fine without the var (WITH (KAFKA_TOPIC='my-topic')).

Am I doing something wrong?

[jzaralim]

The --define flag only works when both the ksql-migrations tool and the ksqlDB server are version 0.18 and newer. This is a gap in our documentation, thanks for catching this!

[dailytabs]

Oh, ok!

The confluentinc/cp-ksqldb-server:6.2.0 container's ksql --help showed the define option - I assumed that would be enough.

Thanks again for your help.