Open mauri870 opened 3 weeks ago
Pinging @elastic/elastic-agent (Team:Elastic-Agent)
Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)
another instance of this failure: https://buildkite.com/elastic/elastic-agent-extended-testing/builds/4262#0192fb96-deec-4c4d-9131-8f9608d6b2a4
Failing test case
TestLogIngestionFleetManaged/Monitoring_logs_are_shipped
Error message
Build
No build logs, this test does not fail when running as part of the the CI.
Actually, TestLogIngestionFleetManaged CI builds are green: https://buildkite.com/organizations/elastic/analytics/suites/elastic-agent-ci/tests/052eba6d-a34c-8f3b-bb58-efe90d9218a7?branch=all+branches.
The easiest way to reproduce is to run the isolated test locally via:
OS
windows/amd64
Stacktrace and notes
Test output
``` Running target: Integration:Single exec: go "list" "-m" Found Elastic Beats dir at /home/mauri870/git/elastic/elastic-agent >>> Creating zip archive of repo to send to remote hosts >>> (windows-amd64-2022-fleet) Starting SSH; connect with `ssh -i /home/mauri870/git/elastic/elastic-agent/.integration-cache/id_rsa windows@34.41.55.30` >>> (windows-amd64-2022-fleet) Connected over SSH >>> (windows-amd64-2022-fleet) Copying repo >>> (windows-amd64-2022-fleet) Running make mage and prepareOnRemote >>> (windows-amd64-2022-fleet) Copying agent build elastic-agent-9.0.0-SNAPSHOT-windows-x86_64.zip >>> (windows-amd64-2022-fleet) Waiting for stack to be ready... >>> (windows-amd64-2022-fleet) Using Stack with Kibana host https://488137a26358480b8928723db8a68c78.us-west2.gcp.elastic-cloud.com, credentials available under .integration-cache >>> (windows-amd64-2022-fleet) Running sudo tests... >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): Running target: Integration:TestOnRemote >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): Running dependency: main.Build.TestBinaries >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): exec: go "build" "-o" "C:\\Users\\windows\\agent\\pkg\\component\\fake\\component\\component.exe" "C:\\Users\\windows\\agent\\pkg\\component\\fake\\component" >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): exec: go "build" "-o" "C:\\Users\\windows\\agent\\internal\\pkg\\agent\\install\\testblocking\\testblocking.exe" "C:\\Users\\windows\\agent\\internal\\pkg\\agent\\install\\testblocking" >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): Running dependency: github.com/elastic/elastic-agent/dev-tools/mage.InstallGoTestTools >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): exec: go "install" "gotest.tools/gotestsum" >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): >> go test: remote-windows-amd64-2022-fleet-sudo.integration Testing >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): >> ARGS: remote-windows-amd64-2022-fleet-sudo.integration Command: gotestsum --no-color -f standard-verbose --junitfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.xml --jsonfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.out.json -- -tags integration -test.shuffle on -test.timeout 2h0m0s -test.run ^(TestLogIngestionFleetManaged)$ github.com/elastic/elastic-agent/testing/integration >>> (windows-amd64-2022-fleet) Test output (sudo) (stderr): exec: gotestsum --no-color -f standard-verbose --junitfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.xml --jsonfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.out.json -- -tags integration -test.shuffle on -test.timeout 2h0m0s -test.run ^(TestLogIngestionFleetManaged)$ github.com/elastic/elastic-agent/testing/integration >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): exec: gotestsum --no-color -f standard-verbose --junitfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.xml --jsonfile build/TEST-go-remote-windows-amd64-2022-fleet-sudo.integration.out.json -- -tags integration -test.shuffle on -test.timeout 2h0m0s -test.run ^(TestLogIngestionFleetManaged)$ github.com/elastic/elastic-agent/testing/integration >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): -test.shuffle 1730385490266613700 >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): === RUN TestLogIngestionFleetManaged >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:66: Enrolling agent in Fleet with a test policy >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:97: Creating enrollment API key... >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:97: Unpacking and installing Elastic Agent >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture_install.go:164: [test TestLogIngestionFleetManaged] Inside fixture install function >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture_install.go:192: [test TestLogIngestionFleetManaged] Inside fixture installNoPkgManager function >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:288: Extracting artifact elastic-agent-9.0.0-SNAPSHOT-windows-x86_64.zip to C:\Users\windows\AppData\Local\Temp\TestLogIngestionFleetManaged995300763 >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:306: Completed extraction of artifact elastic-agent-9.0.0-SNAPSHOT-windows-x86_64.zip to C:\Users\windows\AppData\Local\Temp\TestLogIngestionFleetManaged995300763 >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:906: Components were not modified from the fetched artifact >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:663: >> running binary with: [C:\Users\windows\AppData\Local\Temp\TestLogIngestionFleetManaged995300763\elastic-agent-9.0.0-SNAPSHOT-windows-x86_64\elastic-agent.exe install --force --non-interactive --unprivileged --url https://681f4e7f2b084a01a3528cff05c1001b.fleet.us-west2.gcp.elastic-cloud.com:443 --enrollment-token c2dNRDQ1SUJkbE1rVDhGd1doUUg6RWsya2MwUy1UQ3Voa1dRQ1B3Q0VlZw==] >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:97: >>> Ran Enroll. Output: Unprivileged installation mode enabled; this feature is currently in beta. >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): Installing in non-interactive mode. >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): [ ] Creating gro[====] Service Started [7s] Elastic Agent successfully installed, starting enrollment. >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): [====] Enrolling El[====] Waiting For Enroll... [7s] {"log.level":"info","@timestamp":"2024-10-31T14:38:37.760Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/cmd.(*enrollCmd).enrollWithBackoff","file.name":"cmd/enroll_cmd.go","file.line":520},"message":"Starting enrollment to URL: https://681f4e7f2b084a01a3528cff05c1001b.fleet.us-west2.gcp.elastic-cloud.com:443/","ecs.version":"1.6.0"} >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): {"log.level":"info","@timestamp":"2024-10-31T14:38:40.769Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/cmd.(*enrollCmd).daemonReloadWithBackoff","file.name":"cmd/enroll_cmd.go","file.line":483},"message":"Restarting agent daemon, attempt 0","ecs.version":"1.6.0"} >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): {"log.level":"info","@timestamp":"2024-10-31T14:38:40.774Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/cmd.(*enrollCmd).Execute","file.name":"cmd/enroll_cmd.go","file.line":301},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"} >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): Successfully enrolled the Elastic Agent. >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): [ ==] Enroll Compl[ ==] Done [11s] >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): Elastic Agent has been successfully installed. >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): check.go:64: Agent fleet status: updating >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): check.go:64: Agent fleet status: enrolling >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:105: created policy: 1cab9a58-0c90-49e8-ab08-6e0627383716 >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:663: >> running binary with: [C:\Program Files\Elastic\Agent\elastic-agent.exe status --output json] >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): fixture.go:663: >> running binary with: [C:\Program Files\Elastic\Agent\elastic-agent.exe status --output json] >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): logs_ingestion_test.go:152: Received agent status: >>> (windows-amd64-2022-fleet) Test output (sudo) (stdout): {Info:{ID:8b5b500a-dd1c-47b1-8cc9-2a05fbfcd6b0 Version:9.0.0 Commit:e6f281aa4de18d5ba1aa10aa36da9796c3b22a47 BuildTime:2024-10-31 14:26:29 +0000 UTC Snapshot:true PID:4780 Unprivileged:true} State:2 Message:Running Components:[{ID:beat/metrics-monitoring Name:beat/metrics State:2 Message:Healthy: communicating with pid '3284' Units:[{UnitID:beat/metrics-monitoring UnitType:1 State:2 Message:Healthy Payload:{OsqueryVersion:}} {UnitID:beat/metrics-monitoring-metrics-monitoring-beats UnitType:0 State:2 Message:Healthy Payload:{OsqueryVersion:}}] VersionInfo:{Name:beat-v2-client Version: Meta:{BuildTime:2024-10-30 19:35:32 +0000 UTC Commit:b1c7478458bd1d50155eb1c735f90150d53160a9}}} {ID:filestream-monitoring Name:filestream State:2 Message:Healthy: communicating with pid '5564' Units:[{UnitID:filestream-monitoring UnitType:1 State:2 Message:Healthy Payload:{OsqueryVersion:}} {UnitID:filestream-monitoring-filestream-monitoring-agent UnitType:0 State:2 Message:Healthy Payload:{OsqueryVersion:}}] VersionInfo:{Name:beat-v2-client Version: Meta:{BuildTime:2024-10-30 19:35:32 +0000 UTC Commit:b1c7478458bd1d50155eb1c735f90150d53160a9}}} {ID:http/metrics-monitoring Name:http/metrics State:2 Message:Healthy: communicating with pid '2776' Units:[{UnitID:http/metrics-monitoring UnitType:1 State:2 Message:Healthy Payload:{OsqueryVersion:}} {UnitID:http/metrics-monitoring-metrics-monitoring-agent UnitType:0 State:2 Message:Healthy Payload:{OsqueryVersion:}}] VersionInfo:{Name:beat-v2-client Version: Meta:{BuildTime:2024-10-30 19:35:32 +0000 UTC Commit:b1c7478458bd1d50155eb1c735f90150d53160a9}}}] FleetState:2 FleetMessage:Connected UpgradeDetails: