search

sha3sha3 / UE-EasyKafka

EasyKafka is a Kafka/Redpanda client sub-system for unreal engine. It supports producing and consuming records through blueprint and C++.
https://www.unrealengine.com/marketplace/en-US/product/client-for-kafka
Other
36 stars 8 forks source link

Issue with Kafka Configuration in Unreal Engine for Digital Twin Project #8

Open ramonunibz opened 7 months ago

ramonunibz commented 7 months ago

I am new to Unreal Engine and currently working on a digital twin project where I am using Kafka to log telemetry data messages. I have copied a blueprint for a Kafka consumer, but I am encountering some issues with the configuration.

image

Here are my questions:

Enable SSL Certification: What should I put under "Enable SSL Certification" in the Kafka consumer blueprint? Is it a boolean value or the path to the truststore.pem where I have saved my keys? Also where do i have to put this normally?

Additional Configuration: I need to configure some other parameters similar to what I have done in my Java application. Here is the configuration from my Java application:

final String bootstrapServers;
final String path = "truststore.pem";
commonProps.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, path);

Optional.ofNullable(FileFormat.detect(path))
    .ifPresent(fileFormat -> commonProps.put(SslConfigs.SSL_TRUSTSTORE_TYPE_CONFIG, fileFormat.name()));

bootstrapServers = "127.0.0.1:9094";
commonProps.put(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
commonProps.put(SslConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGORITHM_CONFIG, "");
commonProps.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, SecurityProtocol.SASL_SSL.name);
commonProps.put(SaslConfigs.SASL_MECHANISM, ScramMechanism.SCRAM_SHA_512.mechanismName());
commonProps.put(
    SaslConfigs.SASL_JAAS_CONFIG,
    scramJaasConfig("hono", "hono-secret"));

Could you please guide me on how to configure these parameters in the Kafka consumer blueprint in Unreal Engine?

Thank you in advance for your assistance!

sha3sha3 commented 7 months ago

Hey @ramonunibz What type of auth your are using. Are you for instance using Confluent or a locally hosted version.

ramonunibz commented 7 months ago

About the auth i am using this line to authenticate myself in kafka:

commonProps.put(
    SaslConfigs.SASL_JAAS_CONFIG,
    scramJaasConfig("hono", "hono-secret"));

i thought that under the username and password in the consumer i could just put "hono" and "hono-secret" accordingly.

my instance is hosted locally i believe as Hono is installed locally with Kafka as message broker (https://eclipse.dev/hono/docs/getting-started/) (https://github.com/eclipse/packages/blob/master/charts/hono/README.md).

as for now I have the following under configuration: image and wanted to add the rest of the configuration.

I get the following errors when starting the simulation:

LogKafka: Warning: LogKafka: KafkaConsumer[2888cfac-d5a2c44d] FAIL | [thrd:sasl_ssl://127.0.0.1:9094/bootstrap]: sasl_ssl://127.0.0.1:9094/bootstrap: SSL handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 83ms in state SSL_HANDSHAKE)
LogKafka: Warning: LogKafka: KafkaConsumer[2888cfac-d5a2c44d] FAIL | [thrd:sasl_ssl://127.0.0.1:9094/bootstrap]: sasl_ssl://127.0.0.1:9094/bootstrap: SSL handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 45ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)
LogKafkaException: Error: LogKafkaException: Local: Timed out [-185] | subscribe() timed out!

Note that i have the CA certificates in the truststore.pem