Closed alexhafner closed 7 years ago
@alexhafner the first failure corrupted the environment since and you could not recover from it. There are few options to clean up your environment and try again.
I went down the second route to test if we can recover without a full destroy.
One extra step was needed, as the _TS topics autocreate themselves again on deletion.
This is due to the Kafka Connectors, which I've removed using the REST api
curl -X DELETE localhost:8083/connectors/es_sink_EVENTS_PER_MIN_MAX_AVG_TS curl -X DELETE localhost:8083/connectors/es_sink_CLICKSTREAM_STATUS_CODES_TS curl -X DELETE localhost:8083/connectors/es_sink_ENRICHED_ERROR_CODES_TS curl -X DELETE localhost:8083/connectors/es_sink_CLICK_USER_SESSIONS_TS curl -X DELETE localhost:8083/connectors/es_sink_USER_IP_ACTIVITY_TS curl -X DELETE localhost:8083/connectors/es_sink_PAGES_PER_MIN_TS curl -X DELETE localhost:8083/connectors/es_sink_ERRORS_PER_MIN_TS curl -X DELETE localhost:8083/connectors/es_sink_ERRORS_PER_MIN_ALERT_TS curl -X DELETE localhost:8083/connectors/es_sink_EVENTS_PER_MIN_TS
After that, all relevant topics could be deleted and the install restarted. The install completed perfectly. Many thanks @hjafarpour !
After a permission error I had to restart the ksql clickstream installation.
On re-running ksql-clickstream-demo/demo/clickstream-schema.sql, the exception
_io.confluent.ksql.exception.KafkaTopicException: Topic 'EVENTS_PER_MIN' does not conform to the requirements Partitions:1 v 4. Replication: 1 v 1
was raised. At that point, I had no tables
but plenty of leftover topics
That could be recovered from by removing the topic and restarting the script; however it then ran into a similar error with EVENTS_PER_MIN_TS. This time we attempted to simply increase the partitions to 4.
On the next run of the script, we now seem to fail when the EVENTS_PER_MIN table appears to be dropped - in this case we run into
_io.confluent.ksql.exception.ParseFailedException: Parsing failed on KsqlEngine msg:Cannot add the new data source. Another data source with the same name already exists: KsqlStream name:EVENTS_PERMIN
in the previous step, the script attempts to drop the table; when you execute that manually, you fail with
------------------------------------------------------------------------------- io.confluent.ksql.util.KsqlException: No topic with name true was registered.
in a secenario where
and
Are these messages expected? Is there a better way to recover once you've got a similar failure?