search

IBMStreams / streamsx.kafka

Repository for integration with Apache Kafka
https://ibmstreams.github.io/streamsx.kafka/
Apache License 2.0
13 stars 9 forks source link

Expansion of {applicationDir} in vmArg does not work #227

Closed ghost closed 3 years ago

ghost commented 3 years ago

With toolkit 3.1.2, the expansion of {applicationDir} does not work. This feature has been added with #224 . After enabling Kerberos DEBUG (vmArg: "-Dcom.ibm.security.krb5.Krb5Debug=all"), following cause could be found why Kerberos did not work:

[KRB_DBG_CFG] Config:Thread-9:   readKrb5Properties: defaultRealm = null
[KRB_DBG_CFG] Config:Thread-9:   readKrb5Properties: defaultKDC   = null
[KRB_DBG_CFG] Config:Thread-9:   Java config file: {applicationDir}/etc/krb5.conf
[KRB_DBG_CFG] Config:Thread-9:   Failed to load/parse config file: java.io.FileNotFoundException: {applicationDir}/etc/krb5.conf (No such file or directory)

It seems that setting the property in one thread (which is invoking the operator's initialize) did not make the property aware in a different thread, perhaps the thread that loaded the kerberos related classes.

This effect cannot always be observed, in toolkit version 3.2.0, it has also not been observed.

See also

ghost commented 3 years ago

Hard to reproduce. Observed with Streams version 4.3.1.1.

ghost commented 3 years ago

The commit https://github.com/IBMStreams/streamsx.kafka/commit/a96669f56d3b30fc346f973b5d7c39265ec52fe3 performs the application directory resolution in various threads before instantiating a KafkaProducer or KafkaConsumer now. Still not reproducible with the 3.1.2 toolkit and Streams 4.3.1.1.