[Fleet] Trying to upgrade all fleet servers at once spits "Error upgrading agents: Cannot upgrade agent to XXX because it is higher than the latest fleet server version YYY"

[ipaqmaster]

Kibana version: kibana-8.10.2-1.x86_64

Elasticsearch version: elasticsearch-8.10.2-1.x86_64

Server OS version: Rocky Linux 9.2 - Kernel 5.14.0-284.30.1.el9_2.x86_64

Browser version: Firefox 118.0.2 (64-bit)

Browser OS version: Archlinux 6.5.6-arch2-1

Original install method (e.g. download page, yum, from source, etc.): Official RPM Repository

Describe the bug:

Trying to upgrade all fleet servers at once spits

Error upgrading agents:

Error upgrading agents cannot upgrade agent to XXX because it is higher than the latest fleet server version YYY

Steps to reproduce:

1. Try to upgrade more than one agent at a time to the latest version

Expected behavior:

Kibana should be aware that the multiple agents selected are fleet servers and should not throw this soft UI error when all fleet agents are to be upgraded.

It would also be nice if selecting all agents and upgrading them together could notice the fleet servers are included and won't be a problem.

At the current time it is not possible to upgrade all fleet agents together (Including the fleet servers) - nor upgrading all the fleet servers themselves together because of this version check being called under the wrong conditions. Each fleet server must be upgraded individually from the Fleet dropdown menus.

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

Any additional context:

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jen-huang]

@criamico Did we make changes relevant to this area recently?

[criamico]

@jen-huang I don't recall any changes in 8.10.x that could cause this, this logic that checks the fleet server version separately was added a long time ago.

@ipaqmaster If I understand correctly, the problem arises when there are multiple fleet servers that have different versions and you try to upgrade all of them together?

[ipaqmaster]

Hi

In my experience I had three fleet servers all 'selected' and tried to upgrade them in bulk which threw the error. But their versions were identical.

[ehogeweg]

Hi,

"cannot upgrade agent to 8.12.0 because it is higher than the latest fleet server version 8.7.1"

I see this issue with a single fleet server, upgrading a single agent... The fleet server is at 8.12.0 though so I don't quite understand the message.

[kees-closed]

I also ran into this issue. Still no solution?

[kpollich]

Bumping the priority of this and scheduling it for the near future. cc @nimarezainia

[nimarezainia]

@kees-closed we will get this prioritised. Can you in the meantime give us some detail about your deployment please:

• cloud or on-prem?
• is your fleet server on-prem or part of
• ESS? your workflow was to update all agents - correct?

[kees-closed]

@kees-closed we will get this prioritised. Can you in the meantime give us some detail about your deployment please:

* cloud or on-prem?

* is your fleet server on-prem or part of

* ESS? your workflow was to update all agents - correct?

Hi, my deployment is on-prem, using vanilla ELK (no Kubernetes/ECE). My Fleet server is also on-prem. My action was to update all nodes, see menu below. When I select 'upgrade agent', I get the error shown in my previous comment.

[nimarezainia]

@kees-closed could you first upgrade the agent which is your fleet server? elkalloc05p. Can you let us know if that resolves this issue for you.

While we get around to addressing this.

[kees-closed]

@kees-closed could you first upgrade the agent which is your fleet server? elkalloc05p. Can you let us know if that resolves this issue for you.

While we get around to addressing this.

That works indeed! It also makes sense, didn't think about this update path at first.

[nimarezainia]

Great. for future reference restrictions: https://www.elastic.co/guide/en/fleet/current/upgrade-elastic-agent.html#upgrade-agent-restrictions

[ulab]

I am just getting this warning while trying to update the fleet server from 8.13.4 to 8.14.1