confluentinc / confluent-kafka-go

Confluent's Apache Kafka Golang client
Apache License 2.0
4.59k stars 652 forks source link

How to use Golang client on Windows #128

Closed abhirockzz closed 3 years ago

abhirockzz commented 6 years ago

Description

How can I configure windows in order for my Golang client to work ? e.g. where should I place librdkafka client ?

How to reproduce

  1. download librdkafka from https://www.nuget.org/packages/librdkafka.redist/ & unzip its contents
  2. go get -u github.com/confluentinc/confluent-kafka-go/kafka
  3. setup pkg-config for windows
  4. go run main.go results in
Package rdkafka was not found in the pkg-config search path.
Perhaps you should add the directory containing `rdkafka.pc'
to the PKG_CONFIG_PATH environment variable
No package 'rdkafka' found
pkg-config: exit status 1
neptoess commented 4 years ago

@ligoo / @rwsegura , I was getting seg faults and other strange behavior, so I decided to explore generating the .a file myself instead of renaming the .lib from the NuGet package. If anyone is curious, this is a high level overview of how you do it

  1. pacman -S mingw-w64-x86_64-tools-git
  2. gendef librdkafka.dll
  3. dlltool -k -l librdkafka.a -d librdkafka.def

I'm having success with confluent-kafka-go v1.4.2 and librdkafka v1.4.2 this way

ligoo commented 4 years ago

@neptoess This is great news! I will try out your solution as soon as possible. At the moment, I just dev under 1.3.0 on windows and benchmark with 1.4.2 on ubuntu. @rwsegura I guess you did something similar to @neptoess. Thanks for the answer

rwsegura commented 4 years ago

@ligoo - Sorry for the long delay.

I had two .h files that I made changes to were rdkafka.h and rdkafkacpp.h

KabDeveloper commented 4 years ago

@iradul

I am getting this error message:

go get -u gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
go: finding github.com/confluentinc/confluent-kafka-go/kafka/librdkafka latest
go: finding github.com/confluentinc/confluent-kafka-go/kafka latest
go: downloading github.com/confluentinc/confluent-kafka-go v1.4.2
go: extracting github.com/confluentinc/confluent-kafka-go v1.4.2
# runtime/cgo
exec: "x86_64-w64-mingw32-gcc": executable file not found in %PATH%

I followed exactly all the steps and all things goes well, but when I tried this command: go get -u gopkg.in/confluentinc/confluent-kafka-go.v1/kafka I got the error above.

Any fix please ?

neptoess commented 4 years ago

@bcashier If you aren't targeting Cygwin, you can try my steps (which use MinGW to build native Windows programs) https://github.com/confluentinc/confluent-kafka-go/issues/128#issuecomment-568304317 https://github.com/confluentinc/confluent-kafka-go/issues/128#issuecomment-630361012 https://github.com/confluentinc/confluent-kafka-go/issues/128#issuecomment-630417798

KabDeveloper commented 4 years ago

@neptoess Thank you for your answer !

I get away from using Cygwin, and decided to use Mingw64 alone, but unfortunately I am getting this errors:


/tmp/go-build/cgo-gcc-prolog:1121: undefined reference to `__imp_rd_kafka_topic_result_error_string'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x003.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_result_name':
/tmp/go-build/cgo-gcc-prolog:1139: undefined reference to `__imp_rd_kafka_topic_result_name'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x004.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_AdminOptions_new':
/tmp/go-build/cgo-gcc-prolog:53: undefined reference to `__imp_rd_kafka_AdminOptions_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x004.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_AdminOptions_set_operation_timeout':
/tmp/go-build/cgo-gcc-prolog:76: undefined reference to `__imp_rd_kafka_AdminOptions_set_operation_timeout'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x004.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_AdminOptions_set_request_timeout':
/tmp/go-build/cgo-gcc-prolog:99: undefined reference to `__imp_rd_kafka_AdminOptions_set_request_timeout'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x004.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_AdminOptions_set_validate_only':
/tmp/go-build/cgo-gcc-prolog:122: undefined reference to `__imp_rd_kafka_AdminOptions_set_validate_only'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_conf_destroy':
/tmp/go-build/cgo-gcc-prolog:48: undefined reference to `__imp_rd_kafka_conf_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_conf_new':
/tmp/go-build/cgo-gcc-prolog:62: undefined reference to `__imp_rd_kafka_conf_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_conf_set':
/tmp/go-build/cgo-gcc-prolog:85: undefined reference to `__imp_rd_kafka_conf_set'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_conf_set_default_topic_conf':
/tmp/go-build/cgo-gcc-prolog:101: undefined reference to `__imp_rd_kafka_conf_set_default_topic_conf'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_conf_destroy':
/tmp/go-build/cgo-gcc-prolog:113: undefined reference to `__imp_rd_kafka_topic_conf_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_conf_new':
/tmp/go-build/cgo-gcc-prolog:127: undefined reference to `__imp_rd_kafka_topic_conf_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x005.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_conf_set':
/tmp/go-build/cgo-gcc-prolog:150: undefined reference to `__imp_rd_kafka_topic_conf_set'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_assign':
/tmp/go-build/cgo-gcc-prolog:73: undefined reference to `__imp_rd_kafka_assign'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_assignment':
/tmp/go-build/cgo-gcc-prolog:93: undefined reference to `__imp_rd_kafka_assignment'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_commit_queue':
/tmp/go-build/cgo-gcc-prolog:116: undefined reference to `__imp_rd_kafka_commit_queue'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_committed':
/tmp/go-build/cgo-gcc-prolog:138: undefined reference to `__imp_rd_kafka_committed'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_close':
/tmp/go-build/cgo-gcc-prolog:157: undefined reference to `__imp_rd_kafka_consumer_close'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_group_metadata':
/tmp/go-build/cgo-gcc-prolog:175: undefined reference to `__imp_rd_kafka_consumer_group_metadata'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_group_metadata_destroy':
/tmp/go-build/cgo-gcc-prolog:190: undefined reference to `__imp_rd_kafka_consumer_group_metadata_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_group_metadata_new':
/tmp/go-build/cgo-gcc-prolog:205: undefined reference to `__imp_rd_kafka_consumer_group_metadata_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_group_metadata_read':
/tmp/go-build/cgo-gcc-prolog:225: undefined reference to `__imp_rd_kafka_consumer_group_metadata_read'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_consumer_group_metadata_write':
/tmp/go-build/cgo-gcc-prolog:245: undefined reference to `__imp_rd_kafka_consumer_group_metadata_write'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_name':
/tmp/go-build/cgo-gcc-prolog:263: undefined reference to `__imp_rd_kafka_event_name'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_topic_partition_list':
/tmp/go-build/cgo-gcc-prolog:281: undefined reference to `__imp_rd_kafka_event_topic_partition_list'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_offsets_store':
/tmp/go-build/cgo-gcc-prolog:301: undefined reference to `__imp_rd_kafka_offsets_store'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_pause_partitions':
/tmp/go-build/cgo-gcc-prolog:321: undefined reference to `__imp_rd_kafka_pause_partitions'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_poll_set_consumer':
/tmp/go-build/cgo-gcc-prolog:340: undefined reference to `__imp_rd_kafka_poll_set_consumer'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_position':
/tmp/go-build/cgo-gcc-prolog:360: undefined reference to `__imp_rd_kafka_position'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_queue_get_consumer':
/tmp/go-build/cgo-gcc-prolog:378: undefined reference to `__imp_rd_kafka_queue_get_consumer'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_queue_get_main':
/tmp/go-build/cgo-gcc-prolog:396: undefined reference to `__imp_rd_kafka_queue_get_main'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_resume_partitions':
/tmp/go-build/cgo-gcc-prolog:416: undefined reference to `__imp_rd_kafka_resume_partitions'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_seek':
/tmp/go-build/cgo-gcc-prolog:440: undefined reference to `__imp_rd_kafka_seek'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_subscribe':
/tmp/go-build/cgo-gcc-prolog:460: undefined reference to `__imp_rd_kafka_subscribe'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_subscription':
/tmp/go-build/cgo-gcc-prolog:480: undefined reference to `__imp_rd_kafka_subscription'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_partition_list_add':
/tmp/go-build/cgo-gcc-prolog:501: undefined reference to `__imp_rd_kafka_topic_partition_list_add'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_partition_list_destroy':
/tmp/go-build/cgo-gcc-prolog:516: undefined reference to `__imp_rd_kafka_topic_partition_list_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_partition_list_new':
/tmp/go-build/cgo-gcc-prolog:532: undefined reference to `__imp_rd_kafka_topic_partition_list_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x006.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_unsubscribe':
/tmp/go-build/cgo-gcc-prolog:551: undefined reference to `__imp_rd_kafka_unsubscribe'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_code':
/tmp/go-build/cgo-gcc-prolog:52: undefined reference to `__imp_rd_kafka_error_code'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_destroy':
/tmp/go-build/cgo-gcc-prolog:67: undefined reference to `__imp_rd_kafka_error_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_is_fatal':
/tmp/go-build/cgo-gcc-prolog:83: undefined reference to `__imp_rd_kafka_error_is_fatal'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_is_retriable':
/tmp/go-build/cgo-gcc-prolog:102: undefined reference to `__imp_rd_kafka_error_is_retriable'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_string':
/tmp/go-build/cgo-gcc-prolog:120: undefined reference to `__imp_rd_kafka_error_string'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_error_txn_requires_abort':
/tmp/go-build/cgo-gcc-prolog:139: undefined reference to `__imp_rd_kafka_error_txn_requires_abort'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_fatal_error':
/tmp/go-build/cgo-gcc-prolog:160: undefined reference to `__imp_rd_kafka_fatal_error'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x007.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_test_fatal_error':
/tmp/go-build/cgo-gcc-prolog:182: undefined reference to `__imp_rd_kafka_test_fatal_error'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x008.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_get_err_descs':
/tmp/go-build/cgo-gcc-prolog:89: undefined reference to `__imp_rd_kafka_get_err_descs'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x008.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_version_str':
/tmp/go-build/cgo-gcc-prolog:103: undefined reference to `__imp_rd_kafka_version_str'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `chdrs_to_tmphdrs':
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:33: undefined reference to `__imp_rd_kafka_header_get_all'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_rk_queue_poll':
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:53: undefined reference to `__imp_rd_kafka_event_destroy'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:55: undefined reference to `__imp_rd_kafka_queue_poll'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:56: undefined reference to `__imp_rd_kafka_event_type'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:63: undefined reference to `__imp_rd_kafka_event_message_next'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:64: undefined reference to `__imp_rd_kafka_message_timestamp'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:67: undefined reference to `__imp_rd_kafka_message_headers'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/event.go:68: undefined reference to `__imp_rd_kafka_header_cnt'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_config_string':
/tmp/go-build/cgo-gcc-prolog:74: undefined reference to `__imp_rd_kafka_event_config_string'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_error_is_fatal':
/tmp/go-build/cgo-gcc-prolog:93: undefined reference to `__imp_rd_kafka_event_error_is_fatal'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_message_array':
/tmp/go-build/cgo-gcc-prolog:113: undefined reference to `__imp_rd_kafka_event_message_array'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_message_count':
/tmp/go-build/cgo-gcc-prolog:131: undefined reference to `__imp_rd_kafka_event_message_count'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_stats':
/tmp/go-build/cgo-gcc-prolog:149: undefined reference to `__imp_rd_kafka_event_stats'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_topic_partition':
/tmp/go-build/cgo-gcc-prolog:167: undefined reference to `__imp_rd_kafka_event_topic_partition'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x009.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_partition_destroy':
/tmp/go-build/cgo-gcc-prolog:182: undefined reference to `__imp_rd_kafka_topic_partition_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_last_error':
/tmp/go-build/cgo-gcc-prolog:51: undefined reference to `__imp_rd_kafka_last_error'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_name':
/tmp/go-build/cgo-gcc-prolog:69: undefined reference to `__imp_rd_kafka_name'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_oauthbearer_set_token':
/tmp/go-build/cgo-gcc-prolog:95: undefined reference to `__imp_rd_kafka_oauthbearer_set_token'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_oauthbearer_set_token_failure':
/tmp/go-build/cgo-gcc-prolog:115: undefined reference to `__imp_rd_kafka_oauthbearer_set_token_failure'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_set_log_queue':
/tmp/go-build/cgo-gcc-prolog:135: undefined reference to `__imp_rd_kafka_set_log_queue'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_destroy':
/tmp/go-build/cgo-gcc-prolog:150: undefined reference to `__imp_rd_kafka_topic_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_name':
/tmp/go-build/cgo-gcc-prolog:165: undefined reference to `__imp_rd_kafka_topic_name'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x011.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_topic_new':
/tmp/go-build/cgo-gcc-prolog:185: undefined reference to `__imp_rd_kafka_topic_new'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x013.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_version':
/tmp/go-build/cgo-gcc-prolog:51: undefined reference to `__imp_rd_kafka_version'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x014.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_event_log':
/tmp/go-build/cgo-gcc-prolog:55: undefined reference to `__imp_rd_kafka_event_log'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x016.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_get_watermark_offsets':
/tmp/go-build/cgo-gcc-prolog:138: undefined reference to `__imp_rd_kafka_get_watermark_offsets'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x016.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_metadata':
/tmp/go-build/cgo-gcc-prolog:163: undefined reference to `__imp_rd_kafka_metadata'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x016.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_metadata_destroy':
/tmp/go-build/cgo-gcc-prolog:178: undefined reference to `__imp_rd_kafka_metadata_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x016.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_query_watermark_offsets':
/tmp/go-build/cgo-gcc-prolog:201: undefined reference to `__imp_rd_kafka_query_watermark_offsets'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x018.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_offsets_for_times':
/tmp/go-build/cgo-gcc-prolog:73: undefined reference to `__imp_rd_kafka_offsets_for_times'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `tmphdrs_to_chdrs':
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/producer.go:44: undefined reference to `__imp_rd_kafka_headers_new'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/producer.go:46: undefined reference to `__imp_rd_kafka_header_add'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `do_produce':
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/producer.go:99: undefined reference to `__imp_rd_kafka_producev'
C:/Users/Dev/Desktop/one/gorpc/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.4.2/kafka/producer.go:113: undefined reference to `__imp_rd_kafka_headers_destroy'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_abort_transaction':
/tmp/go-build/cgo-gcc-prolog:87: undefined reference to `__imp_rd_kafka_abort_transaction'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_begin_transaction':
/tmp/go-build/cgo-gcc-prolog:105: undefined reference to `__imp_rd_kafka_begin_transaction'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_commit_transaction':
/tmp/go-build/cgo-gcc-prolog:125: undefined reference to `__imp_rd_kafka_commit_transaction'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_init_transactions':
/tmp/go-build/cgo-gcc-prolog:145: undefined reference to `__imp_rd_kafka_init_transactions'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_outq_len':
/tmp/go-build/cgo-gcc-prolog:164: undefined reference to `__imp_rd_kafka_outq_len'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_produce_batch':
/tmp/go-build/cgo-gcc-prolog:188: undefined reference to `__imp_rd_kafka_produce_batch'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_purge':
/tmp/go-build/cgo-gcc-prolog:209: undefined reference to `__imp_rd_kafka_purge'
C:\Users\Dev\AppData\Local\Temp\go-build697083334\b001\_x019.o: In function `_cgo_60ac638a64d2_Cfunc_rd_kafka_send_offsets_to_transaction':
/tmp/go-build/cgo-gcc-prolog:231: undefined reference to `__imp_rd_kafka_send_offsets_to_transaction'
collect2.exe: error: ld returned 1 exit status

I already copied the files from nuget to mingw folders like so:

(Since I was getting error related to:

# gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
In file included from ..\..\one\gorpc\pkg\mod\gopkg.in\confluentinc\confluent-kafka-go.v1@v1.4.2\kafka\00version.go:24:0:
./librdkafka/rdkafka.h:83:47: fatal error: sys/socket.h: No such file or directory
 #include <sys/socket.h> /* for sockaddr, .. */

)

prefix=c:/
libdir=c:/mingw64/lib/
includedir=c:/mingw64/include

Name: librdkafka
Description: The Apache Kafka C/C++ library
Version: 1.4.2
Cflags: -I${includedir}
Libs: -L${libdir} -lrdkafka
Libs.private: -lssl -lcrypto -lcrypto -lz -ldl -lpthread -lrt

Any help on this please ?

Thank you !

neptoess commented 4 years ago

@bcashier , Try with the v1.4.2 NuGet package. Also make sure you look at my other two comments. For v1.4.2, you have to modify rdkafka.h and 00version.go in the pkg directory (the ones go mod downloads)

KabDeveloper commented 4 years ago

@neptoess

I added the line to 00version.go, but in the commented code, or should I put it before the commented code ?


package kafka

/**
 * Copyright 2016-2019 Confluent Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import (
    "fmt"
)

/*
#cgo LDFLAGS: -lrdkafka
#include <librdkafka/rdkafka.h>

//Minimum required librdkafka version. This is checked both during
//build-time and runtime.
//Make sure to keep the MIN_RD_KAFKA_VERSION, MIN_VER_ERRSTR and #error
//defines and strings in sync.
//

#define MIN_RD_KAFKA_VERSION 0x01040000

#ifdef __APPLE__
#define MIN_VER_ERRSTR "confluent-kafka-go requires librdkafka v1.4.0 or later. Install the latest version of librdkafka from Homebrew by running `brew install librdkafka` or `brew upgrade librdkafka`"
#else
#define MIN_VER_ERRSTR "confluent-kafka-go requires librdkafka v1.4.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
#endif

#if RD_KAFKA_VERSION < MIN_RD_KAFKA_VERSION
#ifdef __APPLE__
#error "confluent-kafka-go requires librdkafka v1.4.0 or later. Install the latest version of librdkafka from Homebrew by running `brew install librdkafka` or `brew upgrade librdkafka`"
#else
#error "confluent-kafka-go requires librdkafka v1.4.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
#endif
#endif
*/
import "C"

func versionCheck() error {
    ver, verstr := LibraryVersion()
    if ver < C.MIN_RD_KAFKA_VERSION {
        return newErrorFromString(ErrNotImplemented,
            fmt.Sprintf("%s: librdkafka version %s (0x%x) detected",
                C.MIN_VER_ERRSTR, verstr, ver))
    }
    return nil
}
neptoess commented 4 years ago

@bcashier You have it in the right spot. The errors you're seeing are coming from the linking step, so librdkafka.a is probably where the issue lies. You said you used librdkafka.redist 0.11.4, but I've only tested this with the latest (v1.4.2). I would recommend updating your files to those, but do the gendef / dlltool thing instead of using the .lib from the NuGet package.

KabDeveloper commented 4 years ago

@neptoess

I just switched to the version 1.4.2, and I got this error:

# gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
c:/ming64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lrdkafka
collect2.exe: error: ld returned 1 exit status

And generated the librdkafka.a using your method !

Any advice please ?

Big thanks again for your time :)

neptoess commented 4 years ago

You followed these steps and copied to your mingw64 lib folder? gendef librdkafka.dll dlltool -k -l librdkafka.a -d librdkafka.def

KabDeveloper commented 4 years ago

Yes I just did that sir :)

And the file is generated successfully !


C:\Users\...\librdkafka.redist.1.4.2\runtimes\win-x64\native> gendef librdkafka.dll
 * [librdkafka.dll] Found PE+ image

C:\Users\...\librdkafka.redist.1.4.2\runtimes\win-x64\native> dlltool -k -l librdkafka.a -d librdkafka.def
neptoess commented 4 years ago

ld --verbose | grep SEARCH_DIR Do that and make sure librdkafka.a is in one of those folders (hint: ls FOLDER | grep kafka)

KabDeveloper commented 4 years ago

Since grep method not present on Windows, I just copied all the response from: ld --versbose command


GNU ld (GNU Binutils) 2.25
  Supported emulations:
   i386pe
using internal linker script:
==================================================
/* Default linker script, for normal executables */
/* Copyright (C) 2014 Free Software Foundation, Inc.
   Copying and distribution of this script, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved.  */
OUTPUT_FORMAT(pei-i386)
SEARCH_DIR("=/c/mingw610/prerequisites/i686-binutils-nomulti/i686-w64-mingw32/lib"); SEARCH_DIR("=/c/mingw610/prerequisites/i686-binutils-nomulti/lib"); SEARCH_DIR("=/usr/local/lib"); SEARCH_DIR("=/lib"); SEARCH_DIR("=/usr/lib");
SECTIONS
{
  /* Make the virtual address and file offset synced if the alignment is
     lower than the target page size. */
  . = SIZEOF_HEADERS;
  . = ALIGN(__section_alignment__);
  .text  __image_base__ + ( __section_alignment__ < 0x1000 ? . : __section_alignment__ ) :
  {
     *(.init)
    *(.text)
    *(SORT(.text$*))
     *(.text.*)
     *(.gnu.linkonce.t.*)
    *(.glue_7t)
    *(.glue_7)
     ___CTOR_LIST__ = .; __CTOR_LIST__ = . ;
            LONG (-1);*(.ctors); *(.ctor); *(SORT(.ctors.*));  LONG (0);
     ___DTOR_LIST__ = .; __DTOR_LIST__ = . ;
            LONG (-1); *(.dtors); *(.dtor); *(SORT(.dtors.*));  LONG (0);
     *(.fini)
    /* ??? Why is .gcc_exc here?  */
     *(.gcc_exc)
    PROVIDE (etext = .);
    PROVIDE (_etext = .);
     *(.gcc_except_table)
  }
  /* The Cygwin32 library uses a section to avoid copying certain data
     on fork.  This used to be named ".data".  The linker used
     to include this between __data_start__ and __data_end__, but that
     breaks building the cygwin32 dll.  Instead, we name the section
     ".data_cygwin_nocopy" and explicitly include it after __data_end__. */
  .data BLOCK(__section_alignment__) :
  {
    __data_start__ = . ;
    *(.data)
    *(.data2)
    *(SORT(.data$*))
    *(.jcr)
    __data_end__ = . ;
    *(.data_cygwin_nocopy)
  }
  .rdata BLOCK(__section_alignment__) :
  {
    *(.rdata)
             *(SORT(.rdata$*))
    __rt_psrelocs_start = .;
    *(.rdata_runtime_pseudo_reloc)
    __rt_psrelocs_end = .;
  }
  __rt_psrelocs_size = __rt_psrelocs_end - __rt_psrelocs_start;
  ___RUNTIME_PSEUDO_RELOC_LIST_END__ = .;
  __RUNTIME_PSEUDO_RELOC_LIST_END__ = .;
  ___RUNTIME_PSEUDO_RELOC_LIST__ = . - __rt_psrelocs_size;
  __RUNTIME_PSEUDO_RELOC_LIST__ = . - __rt_psrelocs_size;
  .eh_frame BLOCK(__section_alignment__) :
  {
    *(.eh_frame*)
  }
  .pdata BLOCK(__section_alignment__) :
  {
    *(.pdata)
  }
  .bss BLOCK(__section_alignment__) :
  {
    __bss_start__ = . ;
    *(.bss)
    *(COMMON)
    __bss_end__ = . ;
  }
  .edata BLOCK(__section_alignment__) :
  {
    *(.edata)
  }
  /DISCARD/ :
  {
    *(.debug$S)
    *(.debug$T)
    *(.debug$F)
    *(.drectve)
     *(.note.GNU-stack)
     *(.gnu.lto_*)
  }
  .idata BLOCK(__section_alignment__) :
  {
    /* This cannot currently be handled with grouped sections.
    See pe.em:sort_sections.  */
    SORT(*)(.idata$2)
    SORT(*)(.idata$3)
    /* These zeroes mark the end of the import list.  */
    LONG (0); LONG (0); LONG (0); LONG (0); LONG (0);
    SORT(*)(.idata$4)
    __IAT_start__ = .;
    SORT(*)(.idata$5)
    __IAT_end__ = .;
    SORT(*)(.idata$6)
    SORT(*)(.idata$7)
  }
  .CRT BLOCK(__section_alignment__) :
  {
    ___crt_xc_start__ = . ;
    *(SORT(.CRT$XC*))  /* C initialization */
    ___crt_xc_end__ = . ;
    ___crt_xi_start__ = . ;
    *(SORT(.CRT$XI*))  /* C++ initialization */
    ___crt_xi_end__ = . ;
    ___crt_xl_start__ = . ;
    *(SORT(.CRT$XL*))  /* TLS callbacks */
    /* ___crt_xl_end__ is defined in the TLS Directory support code */
    ___crt_xp_start__ = . ;
    *(SORT(.CRT$XP*))  /* Pre-termination */
    ___crt_xp_end__ = . ;
    ___crt_xt_start__ = . ;
    *(SORT(.CRT$XT*))  /* Termination */
    ___crt_xt_end__ = . ;
  }
  /* Windows TLS expects .tls$AAA to be at the start and .tls$ZZZ to be
     at the end of section.  This is important because _tls_start MUST
     be at the beginning of the section to enable SECREL32 relocations with TLS
     data.  */
  .tls BLOCK(__section_alignment__) :
  {
    ___tls_start__ = . ;
    *(.tls$AAA)
    *(.tls)
    *(.tls$)
    *(SORT(.tls$*))
    *(.tls$ZZZ)
    ___tls_end__ = . ;
  }
  .endjunk BLOCK(__section_alignment__) :
  {
    /* end is deprecated, don't use it */
    PROVIDE (end = .);
    PROVIDE ( _end = .);
     __end__ = .;
  }
  .rsrc BLOCK(__section_alignment__) : SUBALIGN(4)
  {
    *(.rsrc)
    *(.rsrc$*)
  }
  .reloc BLOCK(__section_alignment__) :
  {
    *(.reloc)
  }
  .stab BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.stab)
  }
  .stabstr BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.stabstr)
  }
  /* DWARF debug sections.
     Symbols in the DWARF debugging sections are relative to the beginning
     of the section.  Unlike other targets that fake this by putting the
     section VMA at 0, the PE format will not allow it.  */
  /* DWARF 1.1 and DWARF 2.  */
  .debug_aranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_aranges)
  }
  .zdebug_aranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_aranges)
  }
  .debug_pubnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_pubnames)
  }
  .zdebug_pubnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_pubnames)
  }
  .debug_pubtypes BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_pubtypes)
  }
  .zdebug_pubtypes BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_pubtypes)
  }
  /* DWARF 2.  */
  .debug_info BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_info .gnu.linkonce.wi.*)
  }
  .zdebug_info BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_info .zdebug.gnu.linkonce.wi.*)
  }
  .debug_abbrev BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_abbrev)
  }
  .zdebug_abbrev BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_abbrev)
  }
  .debug_line BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_line)
  }
  .zdebug_line BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_line)
  }
  .debug_frame BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_frame*)
  }
  .zdebug_frame BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_frame*)
  }
  .debug_str BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_str)
  }
  .zdebug_str BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_str)
  }
  .debug_loc BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_loc)
  }
  .zdebug_loc BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_loc)
  }
  .debug_macinfo BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_macinfo)
  }
  .zdebug_macinfo BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_macinfo)
  }
  /* SGI/MIPS DWARF 2 extensions.  */
  .debug_weaknames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_weaknames)
  }
  .zdebug_weaknames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_weaknames)
  }
  .debug_funcnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_funcnames)
  }
  .zdebug_funcnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_funcnames)
  }
  .debug_typenames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_typenames)
  }
  .zdebug_typenames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_typenames)
  }
  .debug_varnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_varnames)
  }
  .zdebug_varnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_varnames)
  }
  .debug_macro BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_macro)
  }
  .zdebug_macro BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_macro)
  }
  /* DWARF 3.  */
  .debug_ranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_ranges)
  }
  .zdebug_ranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_ranges)
  }
  /* DWARF 4.  */
  .debug_types BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_types .gnu.linkonce.wt.*)
  }
  .zdebug_types BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_types .gnu.linkonce.wt.*)
  }
}

==================================================

And something is sure, I don't have SEARCH_DIR folders !

/c/mingw610/ ...

My mingw folder is: C:/ming64 and already added the bin folder inside it to the ENV VARIABLES.

neptoess commented 4 years ago

Highly recommend using msys2 (https://www.msys2.org/) and installing all your MinGW packages through the provided pacman tool. I'm sure you can get your MinGW install working as well, but I haven't worked with it directly, solely through msys2. You can do a quick test C program and try compling with, e.g. gcc -o a.exe mytest.c -lrdkafka

KabDeveloper commented 4 years ago

@neptoess

I followed your method by installing msys64 and it was compiled successfully !

And reproduced all the steps from here: https://github.com/confluentinc/confluent-kafka-go/issues/128#issuecomment-639672311

And also generated my own .a file using dlltool (I used my mingw64 version for GENDEF since it wasn't found in the msys64 version of mingw).

Thank you sir !

neptoess commented 4 years ago

@bcashier For future reference pacman -S mingw-w64-x86_64-tools-git will install gendef image

PriAggarwal commented 4 years ago

Thanks @iradul, I went throught the sames step (some step are missing but it's still quite good)

But on the last one it won't build, any idea?

D:\AMD\ProRender\rprNetworkNext\go\src\rprWorker>go build

github.com/confluentinc/confluent-kafka-go/kafka

In file included from ..\github.com\confluentinc\confluent-kafka-go\kafka\00version.go:24:0: /usr/include/librdkafka/rdkafka.h:83:47: fatal error: sys/socket.h: No such file or directory

include <sys/socket.h> / for sockaddr, .. /

^ compilation terminated.

I followed through alll the steps but I am also having same issue. I can see the file socket.h in sys folder. Were you or anyone able to resolve this issue?

neptoess commented 4 years ago

@PriAggarwal If you’re using cygwin, I would make sure that your PATH has gcc resolve to the cygwin one.

If you would rather use MinGW, you can follow my instructions on this issue.

neptoess commented 4 years ago

For anyone still interested in this, librdkafka v1.4.4 was released, and this version builds under MinGW. To make confluent-kafka-go more friendly, we’ll need to create and bundle kafka/librdkafka/librdkafka_windows.a and kafka/build_windows.go. This will involve a few minor changes to librdkafka. I won’t have time to get to this in the immediate future, but I can help anyone out who’s willing to take on the work (it’s not a ton of code, I would guess maybe 5-10 lines?)

stephenwsun commented 4 years ago

@neptoess I keep getting the following error even after all the steps you provided:

github.com/confluentinc/confluent-kafka-go/kafka C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lrdkafka collect2.exe: error: ld returned 1 exit status

I was able to do all the step without any issues and the librdkafka.a file is in the directory also:

18325@DGUS-SSUN MINGW64 /c/users/18325/.nuget/packages/librdkafka.redist/1.5.0/runtimes/win-x64/native $ ls /mingw64/lib | findstr kafka librdkafka.a

Any help would be appreciated!

neptoess commented 4 years ago

@stephenwsun , Aside from moving over to MSYS2 for your MINGW64 environment, you can try running this command to see where ld is looking for libraries ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012

stephenwsun commented 4 years ago

@neptoess @edenhill thanks for all the help! I was able to get this working with the latest versions (confluent-kafka-go v1.4.2 and librdkafka v1.5.0). These are the steps I took:

Windows

  1. Download the MSYS2 installer (msys2-x86_64-20200720.exe) here. The default installation folder should be C:\msys64 and the rest can be set to defaults.

  2. After installation, the MSYS shell should be running. If not, then launch it.

  3. Run the following command:

    pacman -Syu
  4. Proceed with installation and press enter when prompted to close the MSYS window.

  5. Open MSYS shell again.

  6. Run the following command again:

    pacman -Syu
  7. Then run the following command:

    pacman -S mingw-w64-x86_64-gcc mingw-w64-x86_64-pkg-config 
  8. Edit environment variables, and add the following to your user variables Path:

    Add C:\msys64\mingw64\bin to your path
  9. Open Visual Studio, and open Package Manager.

  10. Install librdkafka.redist v1.5.0 with the following command:

    Install-Package librdkafka.redist -Version 1.5.0
  11. Copy the following file from the NuGet package:

    C:\Users\18325\.nuget\packages\librdkafka.redist\1.5.0\build\native\include\librdkafka\rdkafka.h
  12. And place in librdkafka directory (create the directory if it doesn't exist):

    C:\msys64\mingw64\include\librdkafka\rdkafka.h
  13. Modify the rdkafka.h file and replace the following:

    Line 58:

    #ifdef _MSC_VER

    With:

    #ifdef __MINGW64__

    Line 2025:

    #ifndef _MSC_VER

    With:

    #ifndef __MINGW64__
  14. In the confluent-kafka-go package, modify 00version.go file:

    Above #include <librdkafka/rdkafka.h>, add the following line:

    #cgo LDFLAGS: -lrdkafka
  15. Copy the rdkafka.pc file in the root directory and paste in the following directory:

    C:\msys64\mingw64\lib\pkgconfig
  16. Add the following DLLs from the NuGet package to C:\Windows\System32:

    librdkafka.dll, librdkafkacpp.dll, libzstd.dll, zlib.dll
  17. Open MSYS shell.

  18. Run the following command:

    pacman -S mingw-w64-x86_64-tools-git
  19. Run the following command:

    gendef librdkafka.dll
  20. Run the following command:

    dlltool -k -l librdkafka.a -d librdkafka.def
  21. Run the following command to see where ld is searching for libraries:

    ld --verbose | findstr SEARCH_DIR | tr -s ' ;' \\012
  22. Copy generated librdkafka.a file to your mingw64 lib folder

  23. Use go run . or go build to create executable.

neptoess commented 4 years ago

@stephenwsun , Good job! Glad to see you got it working. I've finally gotten around to throwing some code together and opening a PR (https://github.com/confluentinc/confluent-kafka-go/pull/509), so hopefully we'll get this down to just your first 8 steps on Windows (with a few more packages in the pacman command) soon!

i542873057 commented 4 years ago

@stephenwsun I follow the step by step to the end , i got this error :

ld.exe: skipping incompatible E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../libmingwthrd.a when searching for -lmingwthrd
E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmingwthrd
ld.exe: skipping incompatible E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../libmingw32.a when searching for -lmingw32
E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmingw32
ld.exe: skipping incompatible E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/libgcc.a when searching for -lgcc
E:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lgcc
....

my confluent-kafka-go version is 1.4.2

I don't know which step is wrong.

Any help would be appreciated!

neptoess commented 4 years ago

@i542873057 If this isn't a production application or anything, you can try to use the source branch of this PR (https://github.com/confluentinc/confluent-kafka-go/pull/509). As long as you have MinGW installed and in path, this should work fine with no extra steps (a Windows static library is bundled)

EugenDueck commented 3 years ago

I'm always getting this error:

./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory

Most steps, as outlined by @stephenwsun and others, (but I added a couple probably unnecessary steps in attempts to fix it) work, but something basic must be wrong.

Here is a full log - echo on - of a build on an Azure VM with lots of libraries and tool preinstalled

##[section]Starting: CmdLine
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.164.2
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\8130011a-1bee-43af-931f-aa4114347d4a.cmd""

D:\a\1\s>set basepath=D:\a\1\s 

D:\a\1\s>echo D:\a\1\s 
D:\a\1\s

D:\a\1\s>SET PATH=C:\msys64\mingw64\bin;C:\msys64l;C:\agents\2.175.2\externals\git\cmd;C:\Program Files\Mercurial\;C:\Program Files\MongoDB\Server\4.4\bin;C:\ProgramData\kind;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS\;C:\Program Files\Mercurial\;C:\hostedtoolcache\windows\stack\2.3.3\x64;C:\ProgramData\chocolatey\lib\ghc.8.10.2\tools\ghc-8.10.2\bin;C:\Program Files\dotnet;C:\mysql-5.7.21-winx64\bin;C:\Program Files\R\R-4.0.2\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Rust\.cargo\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\sbt\bin;C:\hostedtoolcache\windows\go\1.14.9\x64\bin;C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;C:\hostedtoolcache\windows\Python\3.7.9\x64;C:\hostedtoolcache\windows\Ruby\2.5.8\x64\bin;C:\Program Files\Java\jdk8u265-b01\bin;C:\aliyun-cli;C:\npm\prefix;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\Program Files\Microsoft MPI\Bin\;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\ProgramData\Chocolatey\bin;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files\Docker;C:\Program Files\PowerShell\7\;C:\Program Files\dotnet\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\nodejs\;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.6.3\bin;C:\Program Files (x86)\sbt\bin;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;c:\tools\php;C:\Program Files\TortoiseSVN\bin;C:\SeleniumWebDrivers\ChromeDriver\;C:\SeleniumWebDrivers\EdgeDriver\;C:\Program Files\CMake\bin;C:\Program Files\Amazon\AWSCLIV2\;C:\Program Files\Amazon\SessionManagerPlugin\bin\;C:\Program Files\Amazon\AWSSAMCLI\bin\;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;C:\Users\VssAdministrator\AppData\Local\Microsoft\WindowsApps 

D:\a\1\s>nuget install librdkafka.redist -Version 1.4.2 
Feeds used:
  https://api.nuget.org/v3/index.json
  C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

Attempting to gather dependency information for package 'librdkafka.redist.1.4.2' with respect to project 'D:\a\1\s', targeting 'Any,Version=v0.0'
Gathering dependency information took 917 ms
Attempting to resolve dependencies for package 'librdkafka.redist.1.4.2' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'librdkafka.redist.1.4.2'
Resolved actions to install package 'librdkafka.redist.1.4.2'
Retrieving package 'librdkafka.redist 1.4.2' from 'nuget.org'.
  GET https://api.nuget.org/v3-flatcontainer/librdkafka.redist/1.4.2/librdkafka.redist.1.4.2.nupkg
  OK https://api.nuget.org/v3-flatcontainer/librdkafka.redist/1.4.2/librdkafka.redist.1.4.2.nupkg 8ms
Installing librdkafka.redist 1.4.2.
Adding package 'librdkafka.redist.1.4.2' to folder 'D:\a\1\s'
Added package 'librdkafka.redist.1.4.2' to folder 'D:\a\1\s'
Successfully installed 'librdkafka.redist 1.4.2' to D:\a\1\s
Executing nuget actions took 6.62 sec

D:\a\1\s>mkdir C:\msys64\mingw64\include\librdkafka 

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\build\native\include\librdkafka\rdkafka.h C:\msys64\mingw64\include\librdkafka 
        1 file(s) copied.

D:\a\1\s>powershell -Command "(gc C:\msys64\mingw64\include\librdkafka\rdkafka.h) -replace '_MSC_VER', '__MINGW64__' | Out-File -encoding ASCII C:\msys64\mingw64\include\librdkafka\rdkafka.h" 

D:\a\1\s>powershell -Command "(gc C:\msys64\mingw64\include\librdkafka\rdkafka.h) -replace '_WIN32', '__MINGW32__' | Out-File -encoding ASCII C:\msys64\mingw64\include\librdkafka\rdkafka.h" 

D:\a\1\s>copy /Y C:\msys64\mingw64\include\librdkafka\rdkafka.h  C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\build\native\include\librdkafka 
        1 file(s) copied.

D:\a\1\s>echo prefix=c:/ 1>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo libdir=c:/cygwin64/lib/ 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo includedir=c:/cygwin64/usr/include 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo.1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Name: librdkafka 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Description: The Apache Kafka C/C++ library 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Version: 0.11.4 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Cflags: -I${includedir} 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Libs: -L${libdir} -lrdkafka 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>echo Libs.private: -lssl -lcrypto -lcrypto -lz -ldl -lpthread -lrt 1>>C:\msys64\mingw64\lib\pkgconfig\rdkafka.pc 

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\librdkafka.dll C:\Windows\System32 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\librdkafkacpp.dll C:\Windows\System32 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\libzstd.dll C:\Windows\System32 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\zlib.dll C:\Windows\System32 
        1 file(s) copied.

D:\a\1\s>C:\msys64\usr\bin\bash -l -c "pacman -S mingw-w64-x86_64-tools-git" 
mkdir: cannot change permissions of '/dev/shm': Permission denied
mkdir: cannot change permissions of '/dev/mqueue': Permission denied
Copying skeleton files.
These files are for the users to personalise their msys2 experience.

They will never be overwritten nor automatically updated.

'./.bashrc' -> '/home/VssAdministrator/.bashrc'
'./.bash_logout' -> '/home/VssAdministrator/.bash_logout'
'./.bash_profile' -> '/home/VssAdministrator/.bash_profile'
'./.inputrc' -> '/home/VssAdministrator/.inputrc'
'./.profile' -> '/home/VssAdministrator/.profile'
'C:\windows\system32\drivers\etc\hosts' -> '/etc/hosts'
'C:\windows\system32\drivers\etc\protocol' -> '/etc/protocols'
'C:\windows\system32\drivers\etc\services' -> '/etc/services'
'C:\windows\system32\drivers\etc\networks' -> '/etc/networks'
warning: mingw-w64-x86_64-tools-git-8.0.0.5903.3310d459-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) mingw-w64-x86_64-tools-git-8.0.0.5903.3310d459-1

Total Download Size:   0.29 MiB
Total Installed Size:  0.89 MiB
Net Upgrade Size:      0.00 MiB

D:\a\1\s>C:\msys64\usr\bin\bash -l -c "pacman -S mingw-w64-x86_64-gcc mingw-w64-x86_64-pkg-config" 
:: Proceed with installation? [Y/n] warning: mingw-w64-x86_64-gcc-10.2.0-3 is up to date -- reinstalling
resolving dependencies...
warning: mingw-w64-x86_64-pkg-config-0.29.2-1 is up to date -- reinstalling
looking for conflicting packages...
:: Proceed with installation? [Y/n]  * [librdkafka.dll] Found PE+ image

Packages (2) mingw-w64-x86_64-gcc-10.2.0-3  mingw-w64-x86_64-pkg-config-0.29.2-1

Total Download Size:    25.41 MiB
Total Installed Size:  137.05 MiB
Net Upgrade Size:        0.00 MiB

D:\a\1\s>cd /D C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native 

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>gendef librdkafka.dll 

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>dlltool -k -l librdkafka.a -d librdkafka.def 

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>copy librdkafka.a C:\msys64\mingw64\lib 
        1 file(s) copied.

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>ld --verbose   | findstr SEARCH_DIR   | tr -s ' ;' \\012 
SEARCH_DIR("=/mingw64/x86_64-w64-mingw32/lib")0\SEARCH_DIR("=/mingw64/lib")0\SEARCH_DIR("=/usr/local/lib")0\SEARCH_DIR("=/lib")0\SEARCH_DIR("=/usr/lib")0

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>where x86_64-w64-mingw32-gcc 
C:\msys64\mingw64\bin\x86_64-w64-mingw32-gcc.exe
C:\ProgramData\Chocolatey\bin\x86_64-w64-mingw32-gcc.exe
C:\Strawberry\c\bin\x86_64-w64-mingw32-gcc.exe

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>go env CC 
gcc

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>set CC=x86_64-w64-mingw32-gcc 

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>go env CC 
x86_64-w64-mingw32-gcc

C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native>cd /D D:\a\1\s 

D:\a\1\s>git clone https://github.com/confluentinc/confluent-kafka-go 
Cloning into 'confluent-kafka-go'...

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\librdkafka.dll D:\a\1\s\confluent-kafka-go 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\librdkafkacpp.dll D:\a\1\s\confluent-kafka-go 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\libzstd.dll D:\a\1\s\confluent-kafka-go 
        1 file(s) copied.

D:\a\1\s>copy C:\Users\VssAdministrator\.nuget\packages\librdkafka.redist\1.4.2\runtimes\win-x64\native\zlib.dll D:\a\1\s\confluent-kafka-go 
        1 file(s) copied.

D:\a\1\s>cd /D confluent-kafka-go 

D:\a\1\s\confluent-kafka-go>powershell -Command "(gc kafka\00version.go) -replace '#include <librdkafka/rdkafka.h>',"""#cgo LDFLAGS: -lrdkafka`n#include <librdkafka/rdkafka.h>""" | Out-File -encoding ASCII kafka\00version.go" 

D:\a\1\s\confluent-kafka-go>go build ./... 
go: downloading gopkg.in/confluentinc/confluent-kafka-go.v1 v1.4.2
go: finding module for package gopkg.in/alecthomas/kingpin.v2
go: downloading github.com/confluentinc/confluent-kafka-go v1.4.2
go: finding module for package github.com/gdamore/tcell
go: downloading gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: downloading github.com/gdamore/tcell v1.4.0
go: found gopkg.in/alecthomas/kingpin.v2 in gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: found github.com/gdamore/tcell in github.com/gdamore/tcell v1.4.0
go: downloading github.com/mattn/go-runewidth v0.0.7
go: downloading golang.org/x/text v0.3.0
go: downloading github.com/lucasb-eyer/go-colorful v1.0.3
go: finding module for package github.com/alecthomas/template
go: finding module for package github.com/alecthomas/units
go: downloading github.com/gdamore/encoding v1.0.0
go: downloading github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
go: downloading github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: found github.com/alecthomas/template in github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: found github.com/alecthomas/units in github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
# kvcap/confluent-kafka-go/kafka
In file included from kafka\00version.go:25:
./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory
   83 | #include <sys/socket.h> /* for sockaddr, .. */
      |          ^~~~~~~~~~~~~~
compilation terminated.
# gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
In file included from C:\Users\VssAdministrator\go\pkg\mod\gopkg.in\confluentinc\confluent-kafka-go.v1@v1.4.2\kafka\00version.go:24:
./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory
   83 | #include <sys/socket.h> /* for sockaddr, .. */
      |          ^~~~~~~~~~~~~~
compilation terminated.
# github.com/confluentinc/confluent-kafka-go/kafka
In file included from C:\Users\VssAdministrator\go\pkg\mod\github.com\confluentinc\confluent-kafka-go@v1.4.2\kafka\00version.go:24:
./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory
   83 | #include <sys/socket.h> /* for sockaddr, .. */
      |          ^~~~~~~~~~~~~~
compilation terminated.
##[error]Cmd.exe exited with code '2'.
##[section]Finishing: CmdLine
neptoess commented 3 years ago

@EugenDueck Not sure why you’re replacing _WIN32 with MINGW32. rdkafka.h should not require any modification as long as it’s from v1.4.2 or newer.

Might also want to go into your go\pkg\mod...\00version.go file and add

cgo windows LDFLAGS: -lrdkafka above the #include for rdkafka.h

EugenDueck commented 3 years ago

Thanks @neptoess

Not sure why you’re replacing _WIN32 with MINGW32. rdkafka.h should not require any modification as long as it’s from v1.4.2 or newer.

I tried this because I got the sys/socket.h not found error, which meant _WIN32 was not set at the time rdkafka.h was included. As the error is still there, however, my replacing it does not seem to have fixed anything.

Might also want to go into your go\pkg\mod...\00version.go file and add #cgo windows LDFLAGS: -lrdkafka above the #include for rdkafka.h

Doing that already - it's (plainly hidden) in this line:

D:\a\1\s\confluent-kafka-go>powershell -Command "(gc kafka\00version.go) -replace '#include <librdkafka/rdkafka.h>',"""#cgo LDFLAGS: -lrdkafka`n#include <librdkafka/rdkafka.h>""" | Out-File -encoding ASCII kafka\00version.go" 
jfeng45 commented 3 years ago

@EugenDueck

I got exactly the same error. The way I fixed it is that I found it didn't use the file "rdkafka.h" in "msys64\mingw64\include\librdkafka" folder, instead it used the "rdkafka.h" in kafka-confluent package ("go\pkg\mod\github.com\confluentinc\confluent-kafka-go@v1.4.2\kafka\librdkafka"). You first find "00version.go", then find the "librdkafka" directory in the same folder, the file "rdkafka.h" is in that folder. I changed this file, then it worked.

The following is the error I got. It tell you that the file it used is "./librdkafka/rdkafka.h", the path is relative to "00version.go".

github.com/confluentinc/confluent-kafka-go/kafka

In file included from D:\code\pkg\mod\github.com\confluentinc\confluent-kafka-go@v1.4.2\kafka\00version.go:24: ./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory 83 | #include <sys/socket.h> / for sockaddr, .. / | ^~~~~~ compilation terminated.

EugenDueck commented 3 years ago

@jfeng45 Thanks - that looks like a promising route to take indeed.

In the meantime, due to having wasted so much time on trying to cross-compile confluent-kafka-go, and another problem I have with it - namely, it silently not delivering all the messages that I send, with no errors in the events channel, and after supposedly successful flushing), I decided to cut my losses. I am now happily using and cross-compiling sarama. But if I will in fact get back to giving confluent-kafka-go another try, this may be the first thing to try!

ws6 commented 3 years ago

@stephenwsun thanks! I got it worked successfully with your instructions. Except I don't have to change your #13 "Modify the rdkafka.h file and replace the following"

FlyingOnion commented 3 years ago

Hi, guys. I'm also trying to build this in Windows. Luckily I do not have much troubles. I have read the responses above, and find that the solutions above is to add dynamic libraries to the system load path (maybe C:\Windows\System32). My solution is described below:

  1. Install MinGW I installed MinGW with scoop (as well as nuget), run scoop install https://raw.githubusercontent.com/ScoopInstaller/Main/ccc24e57b6026f88d5a7b1c11150e83487a016e1/bucket/gcc.json, and gcc 8.1.0 will be installed. ~The latest gcc (9.3.0) remains untried.~ Yes it could build.
  2. Install Nuget, and run nuget install librdkafka.redist. The version of the library is v1.6.0.
  3. Add github.com/confluentinc/confluent-kafka-go v1.5.2 to go.mod, then run go mod tidy.
  4. Add CGO_LDFLAGS. I run this command in git bash, just modify the user home path or use ~ or $whoami. Use similar methods to add this flag in PS or WT.
    CGO_LDFLAGS="-LC:\Users\Administrator\.nuget\packages\librdkafka.redist\1.6.0\runtimes\win-x64\native -lrdkafka" go build
  5. The binary will be successfully generated, but it needs those dynamic libraries to run. Copy dll files in C:\Users\Administrator\.nuget\packages\librdkafka.redist\1.6.0\runtimes\win-x64\native to the path and it should run normally.

main.go

package main

import (
    "fmt"

    "github.com/confluentinc/confluent-kafka-go/kafka"
)

func main() {
    vnum, vstr := kafka.LibraryVersion()
    fmt.Printf("LibraryVersion: %s (0x%x)\n", vstr, vnum)
}

Before copy

$ ./first.exe
C:/Users/Administrator/go/src/example/first/first.exe: error while loading shared libraries: librdkafka.dll: cannot open shared object file: No such file or directory

After copy

$ ./first.exe
LibraryVersion: 1.6.0 (0x10600ff)
AswathyAshokan commented 3 years ago

@edenhill I followed the solution which you suggested here, but during go build getting an undefined reference error. please help on this

go build

gopkg.in/confluentinc/confluent-kafka-go.v1/kafka

C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AdminOptions_destroy': /tmp/go-build/cgo-gcc-prolog:101: undefined reference toimp_rd_kafka_AdminOptions_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AlterConfigs': /tmp/go-build/cgo-gcc-prolog:117: undefined reference toimp_rd_kafka_AlterConfigs' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AlterConfigs_result_resources': /tmp/go-build/cgo-gcc-prolog:133: undefined reference toimp_rd_kafka_AlterConfigs_result_resources' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_is_read_only': /tmp/go-build/cgo-gcc-prolog:152: undefined reference to__imp_rd_kafka_ConfigEntry_is_read_only' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_is_sensitive': /tmp/go-build/cgo-gcc-prolog:171: undefined reference toimp_rd_kafka_ConfigEntry_is_sensitive' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_is_synonym': /tmp/go-build/cgo-gcc-prolog:190: undefined reference toimp_rd_kafka_ConfigEntry_is_synonym' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_name': /tmp/go-build/cgo-gcc-prolog:208: undefined reference toimp_rd_kafka_ConfigEntry_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_source': /tmp/go-build/cgo-gcc-prolog:227: undefined reference toimp_rd_kafka_ConfigEntry_source' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_synonyms': /tmp/go-build/cgo-gcc-prolog:246: undefined reference toimp_rd_kafka_ConfigEntry_synonyms' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigEntry_value': /tmp/go-build/cgo-gcc-prolog:264: undefined reference toimp_rd_kafka_ConfigEntry_value' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_configs': /tmp/go-build/cgo-gcc-prolog:283: undefined reference toimp_rd_kafka_ConfigResource_configs' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_destroy': /tmp/go-build/cgo-gcc-prolog:298: undefined reference toimp_rd_kafka_ConfigResource_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_error': /tmp/go-build/cgo-gcc-prolog:314: undefined reference to__imp_rd_kafka_ConfigResource_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_error_string': /tmp/go-build/cgo-gcc-prolog:332: undefined reference toimp_rd_kafka_ConfigResource_error_string' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_name': /tmp/go-build/cgo-gcc-prolog:350: undefined reference toimp_rd_kafka_ConfigResource_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_new': /tmp/go-build/cgo-gcc-prolog:370: undefined reference to__imp_rd_kafka_ConfigResource_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_set_config': /tmp/go-build/cgo-gcc-prolog:391: undefined reference toimp_rd_kafka_ConfigResource_set_config' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigResource_type': /tmp/go-build/cgo-gcc-prolog:410: undefined reference toimp_rd_kafka_ConfigResource_type' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ConfigSource_name': /tmp/go-build/cgo-gcc-prolog:429: undefined reference to__imp_rd_kafka_ConfigSource_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_CreatePartitions': /tmp/go-build/cgo-gcc-prolog:448: undefined reference toimp_rd_kafka_CreatePartitions' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_CreatePartitions_result_topics': /tmp/go-build/cgo-gcc-prolog:464: undefined reference toimp_rd_kafka_CreatePartitions_result_topics' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_CreateTopics': /tmp/go-build/cgo-gcc-prolog:483: undefined reference toimp_rd_kafka_CreateTopics' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_CreateTopics_result_topics': /tmp/go-build/cgo-gcc-prolog:499: undefined reference toimp_rd_kafka_CreateTopics_result_topics' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DeleteTopic_destroy': /tmp/go-build/cgo-gcc-prolog:514: undefined reference toimp_rd_kafka_DeleteTopic_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DeleteTopic_new': /tmp/go-build/cgo-gcc-prolog:529: undefined reference toimp_rd_kafka_DeleteTopic_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DeleteTopics': /tmp/go-build/cgo-gcc-prolog:548: undefined reference toimp_rd_kafka_DeleteTopics' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DeleteTopics_result_topics': /tmp/go-build/cgo-gcc-prolog:564: undefined reference toimp_rd_kafka_DeleteTopics_result_topics' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DescribeConfigs': /tmp/go-build/cgo-gcc-prolog:583: undefined reference toimp_rd_kafka_DescribeConfigs' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_DescribeConfigs_result_resources': /tmp/go-build/cgo-gcc-prolog:599: undefined reference toimp_rd_kafka_DescribeConfigs_result_resources' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewPartitions_destroy': /tmp/go-build/cgo-gcc-prolog:614: undefined reference toimp_rd_kafka_NewPartitions_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewPartitions_new': /tmp/go-build/cgo-gcc-prolog:632: undefined reference toimp_rd_kafka_NewPartitions_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewPartitions_set_replica_assignment': /tmp/go-build/cgo-gcc-prolog:657: undefined reference toimp_rd_kafka_NewPartitions_set_replica_assignment' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewTopic_destroy': /tmp/go-build/cgo-gcc-prolog:672: undefined reference toimp_rd_kafka_NewTopic_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewTopic_new': /tmp/go-build/cgo-gcc-prolog:691: undefined reference to__imp_rd_kafka_NewTopic_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewTopic_set_config': /tmp/go-build/cgo-gcc-prolog:712: undefined reference toimp_rd_kafka_NewTopic_set_config' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_NewTopic_set_replica_assignment': /tmp/go-build/cgo-gcc-prolog:737: undefined reference toimp_rd_kafka_NewTopic_set_replica_assignment' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_ResourceType_name': /tmp/go-build/cgo-gcc-prolog:756: undefined reference to__imp_rd_kafka_ResourceType_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_clusterid': /tmp/go-build/cgo-gcc-prolog:776: undefined reference toimp_rd_kafka_clusterid' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_conf_set_events': /tmp/go-build/cgo-gcc-prolog:793: undefined reference toimp_rd_kafka_conf_set_events' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_controllerid': /tmp/go-build/cgo-gcc-prolog:811: undefined reference toimp_rd_kafka_controllerid' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_destroy': /tmp/go-build/cgo-gcc-prolog:826: undefined reference toimp_rd_kafka_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_err2str': /tmp/go-build/cgo-gcc-prolog:842: undefined reference toimp_rd_kafka_err2str' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_AlterConfigs_result': /tmp/go-build/cgo-gcc-prolog:860: undefined reference toimp_rd_kafka_event_AlterConfigs_result' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_CreatePartitions_result': /tmp/go-build/cgo-gcc-prolog:878: undefined reference toimp_rd_kafka_event_CreatePartitions_result' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_CreateTopics_result': /tmp/go-build/cgo-gcc-prolog:896: undefined reference toimp_rd_kafka_event_CreateTopics_result' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_DeleteTopics_result': /tmp/go-build/cgo-gcc-prolog:914: undefined reference toimp_rd_kafka_event_DeleteTopics_result' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_DescribeConfigs_result': /tmp/go-build/cgo-gcc-prolog:932: undefined reference toimp_rd_kafka_event_DescribeConfigs_result' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_destroy': /tmp/go-build/cgo-gcc-prolog:947: undefined reference toimp_rd_kafka_event_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_error': /tmp/go-build/cgo-gcc-prolog:963: undefined reference toimp_rd_kafka_event_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_error_string': /tmp/go-build/cgo-gcc-prolog:981: undefined reference toimp_rd_kafka_event_error_string' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_type': /tmp/go-build/cgo-gcc-prolog:1000: undefined reference toimp_rd_kafka_event_type' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_mem_free': /tmp/go-build/cgo-gcc-prolog:1016: undefined reference to__imp_rd_kafka_mem_free' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_new': /tmp/go-build/cgo-gcc-prolog:1035: undefined reference toimp_rd_kafka_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_queue_destroy': /tmp/go-build/cgo-gcc-prolog:1050: undefined reference toimp_rd_kafka_queue_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_queue_new': /tmp/go-build/cgo-gcc-prolog:1065: undefined reference toimp_rd_kafka_queue_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_queue_poll': /tmp/go-build/cgo-gcc-prolog:1085: undefined reference toimp_rd_kafka_queue_poll' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_result_error': /tmp/go-build/cgo-gcc-prolog:1104: undefined reference toimp_rd_kafka_topic_result_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_result_error_string': /tmp/go-build/cgo-gcc-prolog:1122: undefined reference toimp_rd_kafka_topic_result_error_string' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x003.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_result_name': /tmp/go-build/cgo-gcc-prolog:1140: undefined reference toimp_rd_kafka_topic_result_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x004.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AdminOptions_new': /tmp/go-build/cgo-gcc-prolog:54: undefined reference toimp_rd_kafka_AdminOptions_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x004.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AdminOptions_set_operation_timeout': /tmp/go-build/cgo-gcc-prolog:77: undefined reference toimp_rd_kafka_AdminOptions_set_operation_timeout' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x004.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AdminOptions_set_request_timeout': /tmp/go-build/cgo-gcc-prolog:100: undefined reference toimp_rd_kafka_AdminOptions_set_request_timeout' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x004.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_AdminOptions_set_validate_only': /tmp/go-build/cgo-gcc-prolog:123: undefined reference toimp_rd_kafka_AdminOptions_set_validate_only' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_conf_destroy': /tmp/go-build/cgo-gcc-prolog:49: undefined reference toimp_rd_kafka_conf_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_conf_new': /tmp/go-build/cgo-gcc-prolog:63: undefined reference toimp_rd_kafka_conf_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_conf_set': /tmp/go-build/cgo-gcc-prolog:86: undefined reference toimp_rd_kafka_conf_set' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_conf_set_default_topic_conf': /tmp/go-build/cgo-gcc-prolog:102: undefined reference toimp_rd_kafka_conf_set_default_topic_conf' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_conf_destroy': /tmp/go-build/cgo-gcc-prolog:114: undefined reference toimp_rd_kafka_topic_conf_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_conf_new': /tmp/go-build/cgo-gcc-prolog:128: undefined reference toimp_rd_kafka_topic_conf_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x005.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_conf_set': /tmp/go-build/cgo-gcc-prolog:151: undefined reference toimp_rd_kafka_topic_conf_set' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_assign': /tmp/go-build/cgo-gcc-prolog:74: undefined reference toimp_rd_kafka_assign' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_assignment': /tmp/go-build/cgo-gcc-prolog:94: undefined reference toimp_rd_kafka_assignment' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_commit_queue': /tmp/go-build/cgo-gcc-prolog:117: undefined reference to__imp_rd_kafka_commit_queue' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_committed': /tmp/go-build/cgo-gcc-prolog:139: undefined reference toimp_rd_kafka_committed' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_close': /tmp/go-build/cgo-gcc-prolog:158: undefined reference toimp_rd_kafka_consumer_close' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_group_metadata': /tmp/go-build/cgo-gcc-prolog:176: undefined reference toimp_rd_kafka_consumer_group_metadata' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_group_metadata_destroy': /tmp/go-build/cgo-gcc-prolog:191: undefined reference toimp_rd_kafka_consumer_group_metadata_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_group_metadata_new': /tmp/go-build/cgo-gcc-prolog:206: undefined reference toimp_rd_kafka_consumer_group_metadata_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_group_metadata_read': /tmp/go-build/cgo-gcc-prolog:226: undefined reference toimp_rd_kafka_consumer_group_metadata_read' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_consumer_group_metadata_write': /tmp/go-build/cgo-gcc-prolog:246: undefined reference toimp_rd_kafka_consumer_group_metadata_write' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_name': /tmp/go-build/cgo-gcc-prolog:264: undefined reference toimp_rd_kafka_event_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_topic_partition_list': /tmp/go-build/cgo-gcc-prolog:282: undefined reference toimp_rd_kafka_event_topic_partition_list' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_offsets_store': /tmp/go-build/cgo-gcc-prolog:302: undefined reference toimp_rd_kafka_offsets_store' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_pause_partitions': /tmp/go-build/cgo-gcc-prolog:322: undefined reference to__imp_rd_kafka_pause_partitions' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_poll_set_consumer': /tmp/go-build/cgo-gcc-prolog:341: undefined reference toimp_rd_kafka_poll_set_consumer' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_position': /tmp/go-build/cgo-gcc-prolog:361: undefined reference toimp_rd_kafka_position' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_queue_get_consumer': /tmp/go-build/cgo-gcc-prolog:379: undefined reference toimp_rd_kafka_queue_get_consumer' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_queue_get_main': /tmp/go-build/cgo-gcc-prolog:397: undefined reference toimp_rd_kafka_queue_get_main' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_resume_partitions': /tmp/go-build/cgo-gcc-prolog:417: undefined reference to__imp_rd_kafka_resume_partitions' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_seek': /tmp/go-build/cgo-gcc-prolog:441: undefined reference toimp_rd_kafka_seek' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_subscribe': /tmp/go-build/cgo-gcc-prolog:461: undefined reference toimp_rd_kafka_subscribe' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_subscription': /tmp/go-build/cgo-gcc-prolog:481: undefined reference toimp_rd_kafka_subscription' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_partition_list_add': /tmp/go-build/cgo-gcc-prolog:502: undefined reference toimp_rd_kafka_topic_partition_list_add' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_partition_list_destroy': /tmp/go-build/cgo-gcc-prolog:517: undefined reference toimp_rd_kafka_topic_partition_list_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_partition_list_new': /tmp/go-build/cgo-gcc-prolog:533: undefined reference toimp_rd_kafka_topic_partition_list_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x006.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_unsubscribe': /tmp/go-build/cgo-gcc-prolog:552: undefined reference toimp_rd_kafka_unsubscribe' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_code': /tmp/go-build/cgo-gcc-prolog:53: undefined reference toimp_rd_kafka_error_code' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_destroy': /tmp/go-build/cgo-gcc-prolog:68: undefined reference to__imp_rd_kafka_error_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_is_fatal': /tmp/go-build/cgo-gcc-prolog:84: undefined reference toimp_rd_kafka_error_is_fatal' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_is_retriable': /tmp/go-build/cgo-gcc-prolog:103: undefined reference toimp_rd_kafka_error_is_retriable' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_string': /tmp/go-build/cgo-gcc-prolog:121: undefined reference toimp_rd_kafka_error_string' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_error_txn_requires_abort': /tmp/go-build/cgo-gcc-prolog:140: undefined reference toimp_rd_kafka_error_txn_requires_abort' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_fatal_error': /tmp/go-build/cgo-gcc-prolog:161: undefined reference to__imp_rd_kafka_fatal_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x007.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_test_fatal_error': /tmp/go-build/cgo-gcc-prolog:183: undefined reference toimp_rd_kafka_test_fatal_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x008.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_get_err_descs': /tmp/go-build/cgo-gcc-prolog:90: undefined reference toimp_rd_kafka_get_err_descs' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x008.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_version_str': /tmp/go-build/cgo-gcc-prolog:104: undefined reference to__imp_rd_kafka_version_str' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function chdrs_to_tmphdrs': C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:38: undefined reference toimp_rd_kafka_header_get_all' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _rk_queue_poll': C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:53: undefined reference toimp_rd_kafka_event_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:55: undefined reference to `imp_rd_kafka_queue_poll'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:56: undefined reference to __imp_rd_kafka_event_type' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:63: undefined reference toimp_rd_kafka_event_message_next' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:64: undefined reference to __imp_rd_kafka_message_timestamp' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:67: undefined reference toimp_rd_kafka_message_headers' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/event.go:68: undefined reference to `imp_rd_kafka_header_cnt'
C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_config_string': /tmp/go-build/cgo-gcc-prolog:75: undefined reference to
imp_rd_kafka_event_config_string' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_error_is_fatal': /tmp/go-build/cgo-gcc-prolog:94: undefined reference toimp_rd_kafka_event_error_is_fatal' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_message_array': /tmp/go-build/cgo-gcc-prolog:114: undefined reference toimp_rd_kafka_event_message_array' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_message_count': /tmp/go-build/cgo-gcc-prolog:132: undefined reference toimp_rd_kafka_event_message_count' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_stats': /tmp/go-build/cgo-gcc-prolog:150: undefined reference to__imp_rd_kafka_event_stats' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_topic_partition': /tmp/go-build/cgo-gcc-prolog:168: undefined reference toimp_rd_kafka_event_topic_partition' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x009.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_partition_destroy': /tmp/go-build/cgo-gcc-prolog:183: undefined reference toimp_rd_kafka_topic_partition_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_last_error': /tmp/go-build/cgo-gcc-prolog:52: undefined reference to__imp_rd_kafka_last_error' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_name': /tmp/go-build/cgo-gcc-prolog:70: undefined reference toimp_rd_kafka_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_oauthbearer_set_token': /tmp/go-build/cgo-gcc-prolog:96: undefined reference toimp_rd_kafka_oauthbearer_set_token' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_oauthbearer_set_token_failure': /tmp/go-build/cgo-gcc-prolog:116: undefined reference to__imp_rd_kafka_oauthbearer_set_token_failure' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_set_log_queue': /tmp/go-build/cgo-gcc-prolog:136: undefined reference toimp_rd_kafka_set_log_queue' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_destroy': /tmp/go-build/cgo-gcc-prolog:151: undefined reference toimp_rd_kafka_topic_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_name': /tmp/go-build/cgo-gcc-prolog:166: undefined reference toimp_rd_kafka_topic_name' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x011.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_topic_new': /tmp/go-build/cgo-gcc-prolog:186: undefined reference toimp_rd_kafka_topic_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x013.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_version': /tmp/go-build/cgo-gcc-prolog:52: undefined reference toimp_rd_kafka_version' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x014.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_event_log': /tmp/go-build/cgo-gcc-prolog:56: undefined reference toimp_rd_kafka_event_log' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x016.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_get_watermark_offsets': /tmp/go-build/cgo-gcc-prolog:139: undefined reference toimp_rd_kafka_get_watermark_offsets' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x016.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_metadata': /tmp/go-build/cgo-gcc-prolog:164: undefined reference toimp_rd_kafka_metadata' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x016.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_metadata_destroy': /tmp/go-build/cgo-gcc-prolog:179: undefined reference to__imp_rd_kafka_metadata_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x016.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_query_watermark_offsets': /tmp/go-build/cgo-gcc-prolog:202: undefined reference toimp_rd_kafka_query_watermark_offsets' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x018.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_offsets_for_times': /tmp/go-build/cgo-gcc-prolog:74: undefined reference toimp_rd_kafka_offsets_for_times' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function tmphdrs_to_chdrs': C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/producer.go:44: undefined reference to imp_rd_kafka_headers_new' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/producer.go:50: undefined reference to __imp_rd_kafka_header_add' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182\_x019.o: in functiondo_produce': C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/producer.go:99: undefined reference to __imp_rd_kafka_producev' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/v-aswathya/go/pkg/mod/gopkg.in/confluentinc/confluent-kafka-go.v1@v1.5.2/kafka/producer.go:113: undefined reference toimp_rd_kafka_headers_destroy' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_abort_transaction': /tmp/go-build/cgo-gcc-prolog:88: undefined reference toimp_rd_kafka_abort_transaction' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_begin_transaction': /tmp/go-build/cgo-gcc-prolog:106: undefined reference toimp_rd_kafka_begin_transaction' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_commit_transaction': /tmp/go-build/cgo-gcc-prolog:126: undefined reference toimp_rd_kafka_commit_transaction' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_init_transactions': /tmp/go-build/cgo-gcc-prolog:146: undefined reference toimp_rd_kafka_init_transactions' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_outq_len': /tmp/go-build/cgo-gcc-prolog:165: undefined reference toimp_rd_kafka_outq_len' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_produce_batch': /tmp/go-build/cgo-gcc-prolog:189: undefined reference toimp_rd_kafka_produce_batch' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_purge': /tmp/go-build/cgo-gcc-prolog:210: undefined reference toimp_rd_kafka_purge' C:/TDM-GCC-64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: $WORK\b182_x019.o: in function _cgo_e888b84940cb_Cfunc_rd_kafka_send_offsets_to_transaction': /tmp/go-build/cgo-gcc-prolog:232: undefined reference to__imp_rd_kafka_send_offsets_to_transaction' collect2.exe: error: ld returned 1 exit status

neptoess commented 3 years ago

@AswathyAshokan , There's a lot in information in this comment section, but those errors are because you aren't linking to rdkafka.

AswathyAshokan commented 3 years ago

@neptoess How i can link this rdkafka package in golang. Can you give any useful document for this?

neptoess commented 3 years ago

-lrdkafka needs to be passed to the C compiler. The way that happens with Go is through the #cgo directive. As I mentioned in https://github.com/confluentinc/confluent-kafka-go/issues/128#issuecomment-704518971, I currently do this in the 00version.go file

AswathyAshokan commented 3 years ago

@neptoess I added

cgo windows LDFLAGS: -lrdkafka

#include <librdkafka/rdkafka.h>

to 00version.go but still getting the same error.

neptoess commented 3 years ago

I'm not really sure how to provide much further support. You can try making a quick test app in C and seeing if you're able to link to rdkafka. You're using TDM-GCC though. There are a few sets of instructions in this comment thread for how to get everything working under MSYS2. This is just hard because we're using dynamic linking.

Hopefully, at some point in the future, this PR (https://github.com/confluentinc/confluent-kafka-go/pull/564) gets merged, as it would bundle a static rdkafka for Windows. You can point your Go project to the source repo/branch of that PR and see if it works for you.

AswathyAshokan commented 3 years ago

@neptoess I can able to build after taking source code from PR . I think you will merge this PR in nearer future.. Thank You for your support

neptoess commented 3 years ago

For anyone still following this, #564 was merged, so the next release of confluent-kafka-go will include a static bundle for Windows (librdkafka_windows.a). It should work seamlessly for anyone with MinGW gcc installed.

dergyitheron commented 2 years ago

For anyone still struggling to get target windows/amd64 successfully built on linux, I have managed to make it work with the help of this comment.

I'm using Ubuntu 20.04 LTS, I had to apt install mingw-w64 first.

After that all I needed was some env variables and one ldflags parameter to make it static (might not even be necessary but it works for me so I use it)

env \
  GOOS=windows \
  GOARCH=amd64 \
  CGO_ENABLED=1 \
  CC=x86_64-w64-mingw32-gcc \
  CXX=x86_64-w64-mingw32-g++ \
  CGO_CXXFLAGS="-IC:\msys64\mingw64\include" \
go build -ldflags '-extldflags "-static"'

I'm not exactly sure what's happening with all the vars but I believe the Go utilizes mingw-w64 to build all the necessary libraries that aren't available for windows in this package, such as librdkafka.

calipeng commented 1 year ago

@EugenDueck

I got exactly the same error. The way I fixed it is that I found it didn't use the file "rdkafka.h" in "msys64\mingw64\include\librdkafka" folder, instead it used the "rdkafka.h" in kafka-confluent package ("go\pkg\mod\github.com\confluentinc\confluent-kafka-go@v1.4.2\kafka\librdkafka"). You first find "00version.go", then find the "librdkafka" directory in the same folder, the file "rdkafka.h" is in that folder. I changed this file, then it worked.

@jfeng45 How to change the file "rdkafka.h", Can you tell me it in detail