Update go-buildpack with confluent-kafka-go library

[bakalao2k]

What version of Cloud Foundry and CF CLI are you using? (i.e. What is the output of running cf curl /v2/info && cf version?

MN-C02SGJ7XG8WP:aws-configs mmcne$ cf curl /v2/info { "name": "Pivotal Application Service", "build": "2.4.13-build.2", "support": "https://support.pivotal.io", "version": 0, "description": "https://docs.pivotal.io/pivotalcf/2-4/pcf-release-notes/runtime-rn.html", "authorization_endpoint": "https://login.sys.mgmt-plat-pcf.ue1.prd.aws.cloud.arity.com", "token_endpoint": "https://uaa.sys.mgmt-plat-pcf.ue1.prd.aws.cloud.arity.com", "min_cli_version": "6.23.0", "min_recommended_cli_version": "6.23.0", "app_ssh_endpoint": "ssh.sys.mgmt-plat-pcf.ue1.prd.aws.cloud.arity.com:2222", "app_ssh_host_key_fingerprint": "4d:13:22:00:67:fa:f3:f2:07:25:1f:e0:13:c9:30:ac", "app_ssh_oauth_client": "ssh-proxy", "doppler_logging_endpoint": "wss://doppler.sys.mgmt-plat-pcf.ue1.prd.aws.cloud.arity.com:443", "api_version": "2.125.0", "osbapi_version": "2.14", "routing_endpoint": "https://api.sys.mgmt-plat-pcf.ue1.prd.aws.cloud.arity.com/routing" }

cf version - cf version 6.42.0+0cba12168.2019-01-10

What version of the buildpack you are using?

v1.8.42

If you were attempting to accomplish a task, what was it you were attempting to do?

Is it possible to include Confluent's Golang client for Apache Kakfa library in the Go buildpack?

https://github.com/confluentinc/confluent-kafka-go#installing-librdkafka

We received a request from a product team to create a custom buildpack to allow their Go application to communicate with Kafka. Our team and our sister team in a different company's position is to avoid creating/supporting custom buildpacks.

If this is something you can do, what would be the turnaround time for including the library in another version of this buildpack?

What did you expect to happen?

To allow Go apps to talk with Kafka. Other products teams will be able to use the Go buildpack for this use case.

What was the actual behavior?

N/A

Please confirm where necessary:

• [ ] I have included a log output
• [ ] My log includes an error message
• [ ] I have included steps for reproduction

Thanks in advance.

Listed below is the request from the requestor:

Are you aware of any means of installing a library on a PCF instance or os this straight up not possible?
Here is the package which we are trying to use, which requires librdkafka: https://github.com/confluentinc/confluent-kafka-go#installing-librdkafka```

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/168736433

The labels on this github issue will be updated when the story is started.

[dfreilich]

@bakalao2k Thanks for raising the issue! We are in the middle of a rewrite for the buildpacks, transitioning them over to cloud native buildpacks, so I assume it wouldn't be prioritized so highly at the moment. (cc @sclevine @shanks3012 )

At the same time, if all that is necessary is the librdkafka library, you should be able to use the apt-buildpack, together with go buildpack, using multi-buildpack functionality to download librdkafka and allow the confluent-kafka-go library to run.

[dfreilich]

Closing this for now; but if that doesn't solve your issue, feel free to reopen, and we'll see how we would prioritize adding it.