Clickstream Demo: Recovering from install errors

[alexhafner]

After a permission error I had to restart the ksql clickstream installation.

On re-running ksql-clickstream-demo/demo/clickstream-schema.sql, the exception



_io.confluent.ksql.exception.KafkaTopicException: Topic 'EVENTS_PER_MIN' does not conform to the requirements Partitions:1 v 4. Replication: 1 v 1

was raised. At that point, I had no tables

ksql> show tables;

Table Name | Kafka Topic | Format | Windowed

but plenty of leftover topics

ksql> show topics;

 Kafka Topic                 | Registered | Partitions | Partition Replicas
----------------------------------------------------------------------------
 _schemas                    | false      | 1          | 1
 CLICK_USER_SESSIONS         | false      | 1          | 1
 CLICK_USER_SESSIONS_TS      | false      | 1          | 1
 clickstream                 | true       | 1          | 1
 clickstream_codes           | false      | 1          | 1
 CLICKSTREAM_STATUS_CODES_TS | false      | 1          | 1
 clickstream_users           | false      | 1          | 1
 connect-configs             | false      | 1          | 1
 connect-offsets             | false      | 25         | 1
 connect-statuses            | false      | 5          | 1
 CUSTOMER_CLICKSTREAM        | false      | 1          | 1
 ENRICHED_ERROR_CODES        | false      | 1          | 1
 ENRICHED_ERROR_CODES_COUNT  | false      | 1          | 1
 ENRICHED_ERROR_CODES_TS     | false      | 1          | 1
 ERRORS_PER_MIN              | false      | 1          | 1
 ERRORS_PER_MIN_ALERT        | false      | 1          | 1
 ERRORS_PER_MIN_ALERT_TS     | false      | 1          | 1
 ERRORS_PER_MIN_TS           | false      | 1          | 1
 EVENTS_PER_MIN              | false      | 1          | 1
 EVENTS_PER_MIN_MAX_AVG      | false      | 1          | 1
 EVENTS_PER_MIN_MAX_AVG_TS   | false      | 1          | 1
 EVENTS_PER_MIN_TS           | false      | 1          | 1
 ksql__commands              | true       | 1          | 1
 PAGES_PER_MIN               | false      | 1          | 1
 PAGES_PER_MIN_TS            | false      | 1          | 1
  USER_IP_ACTIVITY            | false      | 1          | 1
 USER_IP_ACTIVITY_TS         | false      | 1          | 1

That could be recovered from by removing the topic and restarting the script; however it then ran into a similar error with EVENTS_PER_MIN_TS. This time we attempted to simply increase the partitions to 4.

On the next run of the script, we now seem to fail when the EVENTS_PER_MIN table appears to be dropped - in this case we run into



_io.confluent.ksql.exception.ParseFailedException: Parsing failed on KsqlEngine msg:Cannot add the new data source. Another data source with the same name already exists: KsqlStream name:EVENTS_PERMIN

in the previous step, the script attempts to drop the table; when you execute that manually, you fail with 



------------------------------------------------------------------------------- io.confluent.ksql.util.KsqlException: No topic with name true was registered.

in a secenario where



ksql> show tables;

 Table Name     | Kafka Topic    | Format | Windowed
-----------------------------------------------------
 EVENTS_PER_MIN | EVENTS_PER_MIN | JSON   | true

and

ksql> show topics; 

 Kafka Topic                 | Registered | Partitions | Partition Replicas
----------------------------------------------------------------------------
...
...
 EVENTS_PER_MIN              | false      | 4          | 1
...

Are these messages expected? Is there a better way to recover once you've got a similar failure?

[hjafarpour]

@alexhafner the first failure corrupted the environment since and you could not recover from it. There are few options to clean up your environment and try again.

• You can destroy the confluent platform and restart it again. This way you will have a clean env and you can start the demo.
• You can delete 'ksql__commands' topic in kafka along with the other topics and start again. If non of these work please let me know.

[alexhafner]

I went down the second route to test if we can recover without a full destroy.

One extra step was needed, as the _TS topics autocreate themselves again on deletion.

This is due to the Kafka Connectors, which I've removed using the REST api

curl -X DELETE localhost:8083/connectors/es_sink_EVENTS_PER_MIN_MAX_AVG_TS curl -X DELETE localhost:8083/connectors/es_sink_CLICKSTREAM_STATUS_CODES_TS curl -X DELETE localhost:8083/connectors/es_sink_ENRICHED_ERROR_CODES_TS curl -X DELETE localhost:8083/connectors/es_sink_CLICK_USER_SESSIONS_TS curl -X DELETE localhost:8083/connectors/es_sink_USER_IP_ACTIVITY_TS curl -X DELETE localhost:8083/connectors/es_sink_PAGES_PER_MIN_TS curl -X DELETE localhost:8083/connectors/es_sink_ERRORS_PER_MIN_TS curl -X DELETE localhost:8083/connectors/es_sink_ERRORS_PER_MIN_ALERT_TS curl -X DELETE localhost:8083/connectors/es_sink_EVENTS_PER_MIN_TS After that, all relevant topics could be deleted and the install restarted. The install completed perfectly. Many thanks @hjafarpour !