[8.x](backport #4042) update elastic-agent-libs

mergify[bot] commented 4 weeks ago

What is the problem this PR solves?

fleet-server using an outdated version of elastic-agent-libs

How does this PR solve the problem?

by updating elastic-agent-libs ot its latest version

How to test this PR locally

Ensure mTLS is still working

adjust the IPs/hostnames as needed

build a fleet-server out of this PR

go build

you might need to build an 8.16 agent out of main:

AGENT_PACKAGE_VERSION=8.16.0-SNAPSHOT BEATS_VERSION=8.16.0-SNAPSHOT DEV=true SNAPSHOT=true EXTERNAL=true PLATFORMS="linux/amd64" PACKAGES="tar.gz" mage package

add your fleet server built to the agent package

tar -xf elastic-agent-8.16.0-SNAPSHOT-linux-x86_64.tar.gz
cp path/tp/your/fleet-server ./elastic-agent-8.16.0-SNAPSHOT-linux-x86_64/data/elastic-agent-*/components/fleet-server

create 2 TLS certificates

use elastic-agent-libs/testing/certutil/cmd to create the certificates. Make sure to use elastic-agent-libs with this PR merged or use the PR branch
either make your fleet-server reachable on fleet-server or change -name fleet-server to a valid DNS for your fleet-server.

go run main.go -prefix server -name fleet-server -noip
go run main.go -client -prefix client

you should have:

-rw------- 1 ainsoph ainsoph  312 Oct 24 16:36 client-ca_key.pem
-rw------- 1 ainsoph ainsoph  928 Oct 24 16:36 client-ca.pem
-rw------- 1 ainsoph ainsoph  312 Oct 24 16:36 client-client_key.pem
-rw------- 1 ainsoph ainsoph  895 Oct 24 16:36 client-client.pem

-rw------- 1 ainsoph ainsoph  312 Oct 24 16:37 server-ca_key.pem
-rw------- 1 ainsoph ainsoph  932 Oct 24 16:37 server-ca.pem
-rw------- 1 ainsoph ainsoph  312 Oct 24 16:37 server-fleet-server_key.pem
-rw------- 1 ainsoph ainsoph  928 Oct 24 16:37 server-fleet-server.pem

start an elastic stack (considering elastic-cloud)

add a fleet server with mTLS

elastic-agent install -nf \
--url=https://fleet-server:8220 \
--fleet-server-es=https://es.elastic-cloud.com:443 \
--fleet-server-service-token=a-service-token \
--fleet-server-policy=fleet-server-policy \
--certificate-authorities=/root/certs/server-ca.pem,/root/certs/client-ca.pem,/etc/ssl/certs/ca-certificates.crt \
--fleet-server-cert=/root/certs/server-fleet-server.pem \
--fleet-server-cert-key=/root/certs/server-fleet-server_key.pem \
--elastic-agent-cert=/root/certs/client-client.pem \
--elastic-agent-cert-key=/root/certs/client-client_key.pem \
--fleet-server-client-auth=required \
--fleet-server-port=8220

create a policy with Elastic Defend

add an agent to that policy

elastic-agent install -nf \
--url=https://fleet-server:8220 \
--enrollment-token=a-token \
--certificate-authorities=/root/certs/server-ca.pem,/etc/ssl/certs/ca-certificates.crt \
--elastic-agent-cert=/root/certs/client-client.pem \
--elastic-agent-cert-key=/root/certs/client-client_key.pem

Design Checklist

~~[ ] I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.~~
~~[ ] I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.~~
~~[ ] I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.~~

Checklist

~~[ ] I have commented my code, particularly in hard-to-understand areas~~
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
~~[ ] I have added tests that prove my fix is effective or that my feature works~~
[x] I have added an entry in ./changelog/fragments using the changelog tool

Related issues

N/A
This is an automatic backport of pull request #4042 done by Mergify.

mergify[bot] commented 4 weeks ago

Cherry-pick of 7d77467984f6b34fcb339fbcf59b620dca9e3238 has failed:

On branch mergify/bp/8.x/pr-4042
Your branch is up to date with 'origin/8.x'.

You are currently cherry-picking commit 7d77467.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
    new file:   changelog/fragments/1729782694-Fleet-Server-uses-'ssl.verification_mode:-certificate'-by-default-for-incomming-client-connections.yaml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
    both modified:   NOTICE.txt
    both modified:   go.mod
    both modified:   go.sum
    both modified:   testing/go.mod
    both modified:   testing/go.sum

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

mergify[bot] commented 3 weeks ago

This pull request has not been merged yet. Could you please review and merge it @AndersonQ? 🙏

AndersonQ commented 3 weeks ago

/test

mergify[bot] commented 3 weeks ago

This pull request is now in conflicts. Could you fix it @mergify[bot]? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.x/pr-4042 upstream/mergify/bp/8.x/pr-4042
git merge upstream/8.x
git push upstream mergify/bp/8.x/pr-4042

mergify[bot] commented 2 weeks ago

This pull request has not been merged yet. Could you please review and merge it @AndersonQ? 🙏