confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
118 stars 1.04k forks source link

Document RUN SCRIPT command #1021

Closed miguno closed 6 years ago

miguno commented 6 years ago

We have no documentation of RUN SCRIPT (I checked syntax guide, examples), even though we use it in e.g. the clickstream demo.

RUN SCRIPT allows you to run a list of predefined queries/commands from in a file.

Note: This is related to but different from starting KSQL servers with a --queries-file configuration. In the latter scenario, KSQL servers will run in non-interactive, "headless" configuration and will disable interactive access from the KSQL CLI aka ksql>.

Example usage:

$ cat /path/to/local/application.sql
CREATE STREAM pageviews_copy AS SELECT * FROM pageviews;

$ ksql
ksql> RUN SCRIPT '/path/to/local/application.sql';

Note that, today, RUN SCRIPT will only execute a subset of possible KSQL commands:

1) CS/CT/CTAS/CSAS (= persistent queries) 2) DROP STREAM, DROP TABLE 3) SET

RUN SCRIPT does NOT execute command such as:

4) SHOW TOPICS, SHOW STREAMS, etc. 5) TERMINATE 6) SELECT ... (= non-persistent queries)

miguno commented 6 years ago

Related to https://github.com/confluentinc/ksql/issues/1017.

miguno commented 6 years ago

Documented at https://docs.confluent.io/current/ksql/docs/tutorials/examples.html#running-ksql-statements-from-the-command-line

doctor-pi commented 6 years ago

EDIT: Pls ignore, I discovered there is an open issue already.

Hi,

I'm using version 5.0.0.

For a simple test, try creating a file with a single line: set 'auto.offset.reset'='earliest';

RUN SCRIPT throws:

ksql> RUN SCRIPT 'scripts/set.ksql';io.confluent.ksql.util.KsqlException: Unable to find ddl command factory for statement:class io.confluent.ksql.parser.tree.SetProperty valid statements:[class io.confluent.ksql.parser.tree.DropTable, class io.confluent.ksql.parser.tree.RegisterTopic, class io.confluent.ksql.parser.tree.DropStream, class io.confluent.ksql.parser.tree.CreateTable, class io.confluent.ksql.parser.tree.CreateStream, class io.confluent.ksql.parser.tree.DropTopic]

Pushing the SET out of the file works, for example:

ksql> set 'auto.offset.reset'='earliest'; RUN SCRIPT 'scripts/test.ksql';Successfully changed local property 'auto.offset.reset' from 'null' to 'earliest'

 Message 
---------

---------
rmoff commented 6 years ago

@doctor-pi I've logged a new issue for this: https://github.com/confluentinc/ksql/issues/1861 for