search

operate-first / support

This repo should serve as a central source for users to raise issues/questions/requests for Operate First.
GNU General Public License v3.0
15 stars 25 forks source link

get access to Kafka #144

Closed Gkrumbach07 closed 3 years ago

Gkrumbach07 commented 3 years ago

Description I am working to port my demo onto MOC (more details https://github.com/operate-first/support/issues/126). I need access Kafka which is located on the namespace opf-kafka. I run into the following error which I try to get the broker https://odh-message-bus-kafka-bootstrap-opf-kafka.apps.zero.massopen.cloud. image I assume it is because I do not have the SSL certs that are defined in the docs https://www.operate-first.cloud/users/apps/docs/odh/kafka/README.md. So my question is, what do I need to do to get these certs and integrate them into my namespace?

Additional context https://chat.google.com/room/AAAAAYpDdc8/vaLARlbLzKU gage_krumbach_pubkey.txt

Gkrumbach07 commented 3 years ago

cc @KPostOffice

HumairAK commented 3 years ago

I've sent the ca.key file required -- let us know if it works!

HumairAK commented 3 years ago

@Gkrumbach07 -- has this been resolved?

Gkrumbach07 commented 3 years ago

@HumairAK I cant seem to get ssl certs setup on my Kafka client. Any suggestions?

HumairAK commented 3 years ago

I believe @harshad16 has done this already, maybe he can chime in with the steps he took.

harshad16 commented 3 years ago

i assume you are using kafka-python.https://kafka-python.readthedocs.io/en/master/apidoc/KafkaProducer.html it has arguments for passing the values. try setting this in your kafkaproducer: 

security_protocol=SSL
ssl_cafile="../ca.cert
Gkrumbach07 commented 3 years ago
  1. On your local machine, you need to setup your GPG key pair so that you can receive your ca.cert securely. ( Send your public key)
  2. Once you receive the file (it should be encrypted), you can decrypt it using your GPG private key.
  3. Then you will be left with a ca.crt file that needs to be imported into OpenShift.
  4. In OpenShift, create a new key/value secret.
  5. Name the secret (doesn't really matter what its named), and import your ca.crt. image
  6. Then go to the details for your new secret and Add Secret to Workload. image The mount path must end in a folder with the same name as the secret you made. I followed these docs for that.
  7. Once the secret is added, the container will relaunch with the new volume in place.
  8. I am using kafka-python to run Kafka. Here is an example of how I create a Producer. 
    producer = kafka.KafkaProducer(bootstrap_servers="odh-message-bus-kafka-bootstrap-opf-kafka.apps.zero.massopen.cloud:443", security_protocol="SSL", ssl_cafile="/etc/ca-cert/ca-cert")

    If you were successful, then there should be no errors. Possible errors might be:

    • You used the wrong bootstrap server port. It must be 443 if you are using a route and not directly to the service.
    • ca-cert is not decrypted ( you will be able to tell when you import the ca.crt into OpenShift. If the imported text does not start with -----BEGIN CERTIFICATE-----, then it probably is still encrypted.