Open oscarkaypee opened 3 years ago
Hi,
Thanks for reaching out. Our configuration parameters are case sensitive. We will look deeper to see if we can support for insensitive ones.
I would like to bump this issue. I ran into the same problem- the confluent docker container creates the configs with all lower case, so it's impossible to use this library with them without hacking a workaround.
While i am looking into this, would you mind describing your use case ?
During startup, the confluent docker containers create all the config files, based on env variables (set in the docker compose). You can add the aws-registry variables to the env variables, and confluents scripts will put them in the config files, but they're all in lower case. I'm doing a spike, and looking at using a containerized kafka connect with aws glue. What I would like to do is be able to configure it through the docker compose env variables, and let the config scripts create the config files without having to either alter the library or script. Having it accept either an all lower case, or be case insensitive, would solve the issue.
Hi, having this issue too. Trying to pass on config params using docker-compose environment variables:
kafka-connect:
image: confluentinc/cp-kafka-connect:latest
...
environment:
...
# these 2 values currently rely on a custom fix in the aws library due to a case conversion issue
CONNECT_VALUE_CONVERTER_SCHEMAAUTOREGISTRATIONENABLED: True
CONNECT_VALUE_CONVERTER_AVRORECORDTYPE: "GENERIC_RECORD"
Any updates on this? It would be nice to set the ENV vars for a ECS service.
Hi all,
We're using the latest Glue Schema Registry source with our dockerized Kafka Connect set up which uses environment variables for it's configuration https://docs.confluent.io/platform/current/installation/docker/config-reference.html#kconnect-long-configuration
This in theory means properties such as
value.converter.avroRecordType=GENERIC_RECORD
would be represented asCONNECT_VALUE_CONVERTER_AVRORECORDTYPE: GENERIC_RECORD
in our environment variables for the container, following the Confluent docs above. However, this runs into:[2021-05-24 12:00:31,125] INFO avroRecordType key is not present in the configs {avrorecordtype=GENERIC_RECORD, schemaautoregistrationenabled=true, region=eu-west-1} (com.amazonaws.services.schemaregistry.common.configs.GlueSchemaRegistryConfiguration)
I've tried:
CONNECT_VALUE_CONVERTER_avroRecordType
CONNECT_VALUE_CONVERTER_AVRORECORDTYPE
CONNECT_VALUE_CONVERTER_AVRO_RECORD_TYPE
CONNECT_VALUE_CONVERTER_AVRO__RECORD__TYPE
CONNECT_VALUE_CONVERTER_AVRO___RECORD___TYPE
Which all result in the same lowercase
avrorecordtype
in the configs e.g.{avrorecordtype=GENERIC_RECORD}
Is it possible to support setting these properties via environment variables? Perhaps by ignoring case in the presence check?