elastic / stream

Test utility for streaming data via udp/tcp/tls/webhook/GCP Pub-Sub/Kafka.
https://container-library.elastic.co/r/observability/stream
Apache License 2.0
8 stars 13 forks source link

Add HTTP server output #19

Closed marc-gr closed 3 years ago

marc-gr commented 3 years ago
elasticmachine commented 3 years ago

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS Pipeline View Test View Changes Artifacts preview

Expand to view the summary

#### Build stats * Build Cause: `Pull request #19 updated` * Start Time: 2021-05-28T13:06:20.888+0000 * Duration: 5 min 10 sec * Commit: dcbd5465df136e88e2c4d85824f7c87adacc297f #### Test stats :test_tube: | Test | Results | | ------------ | :-----------------------------: | | Failed | 0 | | Passed | 5 | | Skipped | 0 | | Total | 5 | #### Trends :test_tube: ![Image of Build Times](https://beats-ci.elastic.co/job/Library/job/stream-mbp/job/PR-19/buildTimeGraph/png) ![Image of Tests](https://beats-ci.elastic.co/job/Library/job/stream-mbp/job/PR-19/11/testReport/history/countGraph/png?start=0&end=25)

andrewkroh commented 3 years ago

Not sure if this is related to this output, but it hangs on SIGINT.

{"level":"debug","ts":"2021-05-21T08:47:28.666-0400","caller":"command/log.go:74","msg":"Sending log line.","address":"127.0.0.1:8081","log":"alert2.json","line_number":1}
^C^C^C^C^C^C^CError: waiting to write for too long
marc-gr commented 3 years ago

Not sure if this is related to this output, but it hangs on SIGINT.

{"level":"debug","ts":"2021-05-21T08:47:28.666-0400","caller":"command/log.go:74","msg":"Sending log line.","address":"127.0.0.1:8081","log":"alert2.json","line_number":1}
^C^C^C^C^C^C^CError: waiting to write for too long

Was missing to check the cmd context, this is fixed now 👍

marc-gr commented 3 years ago

I tried it out for testing defender_atp. Worked for me. We should discuss how we want pagination and config to work.

Currently this will just return the given logs in order, one request at a time, so pagination will be defined by the order of the logs you pass to the command.

This ignores the request URL or any other variable in the request as its initial intent is just to send the stream of logs through an http server output.

I'd like to have a more sophisticated tool that could be used to mock a specific server, with some routing and assertions, which I think is more in line with what we discussed in https://github.com/elastic/stream/issues/9, I am unsure if this would belong here or as its own separate tool.

From my testing I think this initial approach might be enough for our current needs, WDYT?

andrewkroh commented 3 years ago

I think all of that could be part of this tool. I think we should do the minimal required for our first set of test cases and then iterate on the development of features that we need to get the test coverage required. Since it's a test tool and primarily for our own consumption it will be fine if we need to break some of the config to refine it.

andrewkroh commented 3 years ago

I noticed the README lists the "outputs". Can you update it to include http-server or remove the list of outputs.