elastic / stream

Test utility for streaming data via udp/tcp/tls/webhook/GCP Pub-Sub/Kafka.
https://container-library.elastic.co/r/observability/stream
Apache License 2.0
8 stars 13 forks source link

Add Azure eventhub support #58

Closed bhapas closed 11 months ago

bhapas commented 1 year ago

Add support for Azure eventhub output.

For Credentials : https://github.com/Azure/azure-sdk-for-go/blob/6b6f76ebe0d2334c83e8b6f89af4fe9d0b1ce631/sdk/azidentity/README.md?plain=1#L156-L187

Test Execution

workspace/stream - (azure-eventhub-output) >
 ./stream log main.go 
-p=azureeventhub 
--azure-event-hub-connection-string="Endpoint=sb://test-eventhub-stream-seis.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SharedAccessKey"

{"level":"debug","ts":"2023-08-28T10:18:02.777+0200","caller":"output/util.go:28","msg":"Connecting...","address":""}
{"level":"info","ts":"2023-08-28T10:18:02.777+0200","caller":"output/util.go:41","msg":"Connected","address":""}
{"level":"debug","ts":"2023-08-28T10:18:02.777+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":1}
{"level":"debug","ts":"2023-08-28T10:18:04.775+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":2}
{"level":"debug","ts":"2023-08-28T10:18:05.101+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":3}
{"level":"debug","ts":"2023-08-28T10:18:05.422+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":4}
{"level":"debug","ts":"2023-08-28T10:18:05.560+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":5}
{"level":"debug","ts":"2023-08-28T10:18:05.699+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":6}
{"level":"debug","ts":"2023-08-28T10:18:05.831+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":7}
{"level":"debug","ts":"2023-08-28T10:18:05.969+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":8}
{"level":"debug","ts":"2023-08-28T10:18:06.108+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":9}
{"level":"debug","ts":"2023-08-28T10:18:06.243+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":10}
{"level":"debug","ts":"2023-08-28T10:18:06.385+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":11}
{"level":"debug","ts":"2023-08-28T10:18:06.547+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":12}
{"level":"debug","ts":"2023-08-28T10:18:06.689+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":13}
{"level":"debug","ts":"2023-08-28T10:18:06.826+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":14}
{"level":"debug","ts":"2023-08-28T10:18:07.028+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":15}
{"level":"debug","ts":"2023-08-28T10:18:07.227+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":16}
{"level":"debug","ts":"2023-08-28T10:18:07.381+0200","caller":"command/log.go:81","msg":"Sending log line.","address":"","log":"main.go","line_number":17}
{"level":"info","ts":"2023-08-28T10:18:07.516+0200","caller":"command/log.go:94","msg":"Log data sent.","address":"","log":"main.go","total_bytes":337,"total_lines":17}
workspace/stream - (azure-eventhub-output) > 

Screenshots

Messages in Azure portal

image
elasticmachine commented 1 year ago

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

#### Build stats * Start Time: 2023-10-16T12:12:58.083+0000 * Duration: 6 min 59 sec #### Test stats :test_tube: | Test | Results | | ------------ | :-----------------------------: | | Failed | 0 | | Passed | 22 | | Skipped | 0 | | Total | 22 |

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with: - `/test` : Re-trigger the build.

marc-gr commented 1 year ago

It looks nice!

I wonder if we could add some tests for it if possible and to add some docs about how to use it.

bhapas commented 12 months ago

Raised https://github.com/Azure/azure-sdk-for-go/issues/21758 for adding support for emulator in Azurite for Event Hub.