confluentinc / confluent-cli

Confluent Platform CLI
Other
60 stars 38 forks source link

Need better error messages for invalid connector config #27

Open rmoff opened 7 years ago

rmoff commented 7 years ago

Valid JSON but not in the expected format (i):

> cat ../connector_config/jdbc_sink_poem.json
{
  "topics": "dummy_topic",
  "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
  "name": "poem_sqlite",
  "connection.url": "jdbc:sqlite:/Users/Robin/cp/confluent-3.3.0-SNAPSHOT-20170629/testdb",
  "auto.create": true,
  "auto.evolve": true
}

> ./bin/confluent load jdbc_sink_poem -d ../connector_config/jdbc_sink_poem.json
parse error: Invalid numeric literal at line 2, column 0

Malformed JSON (ii):

> cat ../connector_config/jdbc_sink_poem.json
{ name:"jdbc_sink_poem",config: {
  "topics": "dummy_topic",
  "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
  "name": "poem_sqlite",
  "connection.url": "jdbc:sqlite:/Users/Robin/cp/confluent-3.3.0-SNAPSHOT-20170629/testdb",
  "auto.create": true,
  "auto.evolve": true}
}

> ./bin/confluent load jdbc_sink_poem -d ../connector_config/jdbc_sink_poem.json
Invalid argument '../connector_config/jdbc_sink_poem.json given to 'config'.

In both of these instances, it should be clear what the error is (JSON is invalid / does not match the expected format). For the first example, since the name of the connector is passed in as a commandline argument can we add extra smarts to confluent CLI to actually massage the JSON into the correct syntax, and simply warn the user that the format is not correct?

rmoff commented 7 years ago

Related to the above - if I have malformed JSON, the error is

Missing 'name' property from connectors properties file.

But name is present - it's just the config elsewhere is screwy (in this case a split-line _comment).

{
  "name": "jdbc_source_mysql_foo_01",
  "config": {
  "_comment": "
  ",
  "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
  "_comment": "",
  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "_comment": "",
  "key.converter.schema.registry.url": "http://localhost:8081",
  "_comment": "",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "_comment": "",
  "value.converter.schema.registry.url": "http://localhost:8081",
  "_comment": "",
  "connection.url": "jdbc:mysql://localhost:3306/demo?user=rmoff&password=pw",
  "_comment": "",
  "table.whitelist": "foo",
  "_comment": "",
  "mode": "incrementing",
  "_comment": "",
  "validate.non.null": "false",
  "_comment": "",
  "incrementing.column.name": "c1",
  "_comment": "",
  "topic.prefix": "mysql2-"
  }
}
kevinpthorne commented 5 years ago

I believe I'm currently dealing with this...

{
  "name": "my-jdbc-connector",
  "config": {
    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
    "tasks.max": "1",
    "connection.url": "jdbc:sqlserver://xxx/xxx",
    "connection.user": "xxx",
    "connection.password": "xxx",
    "mode": "incrementing",
    "incrementing.column.name": "id",
    "topic.prefix": "jdbc-",
    "name": "my-jdbc-connector",
    "table.whitelist": "dbo.xxx",
  }
}
$ confluent load my-jdbc-connector -d my-config.json
[...]
Missing 'name' property from connectors properties file.

I would only guess that it's a JDBC driver error? (As SQL Server driver does not come stock) Still investigating here.

Update: Turns out it was that pesky extra , at the end of the table.whitelist entry. Proper error codes are shown now.