magnusbaeck / logstash-filter-verifier

Apache License 2.0
191 stars 27 forks source link

Number of received events printed in test summary is incorrect if not all events are received #145

Closed jgough closed 2 years ago

jgough commented 2 years ago

With LFV2 in daemon mode, if I have a test that expects 2 outputs but only 1 event is generated, e.g.

input_plugin: "input"
testcases:
  - input:
    - foo
    expected:
      - message: foo
      - message: bar

Then when I run this test I get the following output:

☐ Compare actual event with expected event from test1.yml:
Expected 2 event(s), got 0 instead.

However I expected to see:

☐ Compare actual event with expected event from test1.yml:
Expected 2 event(s), got 1 instead.

Tasty Dockerfile to repro!

# syntax=docker/dockerfile:1.3-labs
FROM golang:1.17 as lfv_builder

RUN git clone https://github.com/magnusbaeck/logstash-filter-verifier.git
RUN apt update && apt install -y protobuf-compiler
RUN useradd -m -s /bin/bash go && chown -R go /go

USER go
WORKDIR /go/logstash-filter-verifier
RUN make && make check test

#------------------------------------------------------#

FROM docker.elastic.co/logstash/logstash:7.10.2

COPY --from=lfv_builder /go/logstash-filter-verifier /usr/bin

RUN <<EOF
mkdir tests
mkdir pipeline/pipeline1

cat <<EOT > /usr/share/logstash/config/pipelines.yml
- pipeline.id: pipeline1
  path.config: "pipeline/pipeline1/*.conf"
EOT

cat <<EOT > /usr/share/logstash/tests/test1.yml
input_plugin: "input"
ignore:
  - "@timestamp"
testcases:
  - input:
    - foo
    expected:
      - message: foo
      - message: bar
EOT

cat <<EOT > /usr/share/logstash/pipeline/pipeline1/01-input.conf
input {
  stdin {
    id => "input"
  }
}

output { stdout {} }
EOT

cat <<EOT > /usr/share/logstash/tests/plugin-mock.yml
- id: test
EOT

cat <<EOT > /usr/share/logstash/run_tests.sh
logstash-filter-verifier daemon start --no-cleanup &
sleep 5
logstash-filter-verifier daemon run --pipeline /usr/share/logstash/config/pipelines.yml --pipeline-base /usr/share/logstash/ --testcase-dir /usr/share/logstash/tests/test1.yml --add-missing-id --plugin-mock /usr/share/logstash/tests/plugin-mock.yml
EOT

EOF

CMD ["/bin/bash", "/usr/share/logstash/run_tests.sh"]

Run with

DOCKER_BUILDKIT=1 docker build --tag test .
docker run --rm -it test