search

burmilla / os

Tiny Linux distro that runs the entire OS as Docker containers
https://burmillaos.org
Apache License 2.0
210 stars 13 forks source link

Upgrade rancher/os 1.5.8 to burmilla/os 2.0.1 to get latest docker version #178

Closed stevecharon closed 4 months ago

stevecharon commented 4 months ago

BurmillaOS Version: (ros os version) Running on RancherOS 1.5.8 Where are you running BurmillaOS? (docker-machine, AWS, GCE, baremetal, etc.) KVM VM Which processor architecture you are using? x86 Do you use some extra hardware? (GPU, etc)? no Which console you use (default, ubuntu, centos, etc..) default/debian Do you use some service(s) which are not enabled by default? qemu-guest-agent Have you installed some extra tools to console? docker-compose Do you use some other customizations? no Please share copy of your cloud-init (remember remove all sensitive data first)

I want to upgrade from rancheros 1.5.8 to latest 2.0 I already did this on a copy of the machine and it doesnt upgrade to latest docker versions system-docker is using 17.06-ros6 and user-docker is still on 19.03.15

I noticed that while ros os version reports latest 2.0.1 all containers are running with old 1.5.x versions

olljanat commented 4 months ago

all containers are running with old 1.5.x versions

Which all containers? It is know issue that console does not update automatically but you need force it to default console like mentioned in here.

After that you should be able to update to later versions of docker like described in here

stevecharon commented 4 months ago

Sorry for not being precise enough. Was not aware of "sudo ros engine list" The system containers running after the upgrade: os-console:v1.5.8 os-qemuguestagent:v2.8.1-2 os-base:v1.5.8 container-crontab:v0.4.0 os-docker:19.03.15 os-base:v1.5.8 os-base:v1.5.8 container-crontab:v0.4.0 os-acpid:v1.5.8 os-syslog:v1.5.8

But I will try to switch the console and change the engine and see how it plays out. Thank you

stevecharon commented 4 months ago

After switching consoles (and a reboot): sudo ros console list disabled alpine disabled centos disabled debian current default disabled fedora disabled ubuntu

No progress here with the engine:

sudo ros engine list --update disabled docker-1.12.6 disabled docker-1.13.1 disabled docker-17.03.1-ce disabled docker-17.03.2-ce disabled docker-17.06.1-ce disabled docker-17.06.2-ce disabled docker-17.09.0-ce disabled docker-17.09.1-ce disabled docker-17.12.0-ce disabled docker-17.12.1-ce disabled docker-18.03.0-ce disabled docker-18.03.1-ce disabled docker-18.06.0-ce disabled docker-18.06.1-ce disabled docker-18.06.2-ce disabled docker-18.06.3-ce disabled docker-18.09.0 disabled docker-18.09.1 disabled docker-18.09.2 disabled docker-18.09.3 disabled docker-18.09.4 disabled docker-18.09.5 disabled docker-18.09.6 disabled docker-18.09.7 disabled docker-18.09.8 disabled docker-18.09.9 disabled docker-19.03.0 disabled docker-19.03.1 disabled docker-19.03.11 disabled docker-19.03.14 current docker-19.03.15 disabled docker-19.03.2 disabled docker-19.03.3 disabled docker-19.03.4 disabled docker-19.03.5 disabled docker-19.03.7 disabled docker-19.03.8 disabled docker-19.03.9

since the rancher-os containers with 1.5.8 are still running no change would be expected here. So how do I get rid of good old rancher-os and see the new ones to use docker 20+?

I should also mention that we are in an enterprise environment with proxies and firewalls. Are the new containers supposed to be pulled in after the upgrade? How can I prepare for that? We have a pull-through registry which we could use if that would be any help

olljanat commented 4 months ago

Looks that you are still in RancherOS, not BurmillaOS.

Do you see BurmillaOS already in boot menu which let you choose between current and previous version? If you do, then make sure that you boot to BurmillaOS, if not then try upgrade again.

stevecharon commented 4 months ago

image BurmillaOS is shown in boot, yes, but no 2.0.1 container is picked up I already tried to do a "ros os upgrade" again to no avail sudo uname -a Linux test-vm-62 5.10.211-burmilla #1 SMP Tue Mar 5 11:50:09 UTC 2024 x86_64 GNU/Linux

olljanat commented 4 months ago

Did you do that Root CA update which is mentioned in v2.0.1 release notes? Looks that it fails to download those new images and fallback to old ones.

stevecharon commented 4 months ago

Yes, needed to do that in order to get at least somewhere. I also tried to put in my registry-mirror on docker and system-docker and use docker-remote as prefix mentioned in the docs sudo system-docker pull burmilla/os-base:v2.0.1 worked after that but an sudo ros os upgrade fails with the error that docker-remote/burmilla/os could not be found. So os upgrade does not reflect the registry-mirror do I have to put that in another place as well/instead?

olljanat commented 4 months ago

hmm, so you have some registry mirror stuff in use? Please, share details of those as mentioned in initial questions. Cannot help you without complete picture of your configuration.

stevecharon commented 4 months ago

I was testing registry mirror after the issues but got nowhere further. You can safely ignore all that stuff with registry_mirror and image-prefix. I already pulled the os-images locally with user-docker and with system-docker as well to make sure the needed images are there. Apparently, just a docker pull always tries to contact out there and if that fails it doesn't recognize the local container. Thats an issue with docker though, just wanted to mention it. So although we have all the stuff locally we still need to check external ressources and that obviously fails on upgrade.

Cleaned up config of my setup. all private keys and ssh-keys shortened. IPs and Environment names changed.

[root@machine62 conf]# cat cloud-config.yml

hostname: machine62.example.local
mounts:
- - /dev/sdb
  - /opt/
  - ext4
  - ""
rancher:
  bootstrap:
    bootstrap:
      command: ros-bootstrap
      image: rancher/os-bootstrap:v1.5.8
      labels:
        io.rancher.os.detach: "false"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      volumes:
      - /dev:/host/dev
      - /lib/modules:/lib/modules
      - /lib/firmware:/lib/firmware
      - /usr/bin/ros:/usr/bin/ros:ro
      - /usr/bin/ros:/usr/bin/ros-bootstrap:ro
      - /usr/share/ros:/usr/share/ros:ro
      - /var/lib/rancher:/var/lib/rancher:ro
      - /var/log:/var/log
  bootstrap_docker:
    bridge: none
    graph: /var/lib/system-docker
    group: root
    host:
    - unix:///var/run/system-docker.sock
    registry_mirror: https://machine134.example.local
    restart: false
    storage_driver: overlay2
    userland_proxy: false
  cloud_init:
    datasources:
    - vmware
  cloud_init_services:
    cloud-init:
      command: cloud-init-save
      image: rancher/os-base:v1.5.8
      ipc: host
      labels:
        io.rancher.os.detach: "false"
        io.rancher.os.scope: system
      log_driver: json-file
      net: host
      pid: host
      privileged: true
      uts: host
      volumes:
      - /etc/resolv.conf:/etc/resolv.conf
      - /dev:/host/dev
      - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt.rancher
      - /lib/modules:/lib/modules
      - /lib/firmware:/lib/firmware
      - /usr/bin/ros:/usr/bin/ros:ro
      - /usr/bin/ros:/usr/bin/cloud-init-save
      - /usr/share/ros:/usr/share/ros:ro
      - /var/lib/rancher:/var/lib/rancher
      - /var/lib/rancher/conf:/var/lib/rancher/conf
      - /var/log:/var/log
  console: default
  defaults:
    docker:
      engine: docker-19.03.15
    hostname: rancher
    network:
      dhcp_timeout: 10
      dns:
        nameservers:
        - 8.8.8.8
        - 8.8.4.4
    system_docker_logs: /var/log/system-docker.log
  docker:
    engine: docker-20.10.22
    environment:
    - http_proxy=http://172.26.0.34:8080
    - https_proxy=http://172.26.0.34:8080
    - no_proxy=localhost,127.0.0.1,*.example.local,10.*,192.168.*
    - NO_PROXY=localhost,127.0.0.1,*.example.local,10.*,192.168.*
    group: docker
    host:
    - unix:///var/run/docker.sock
    log_opts:
      max-file: 2
      max-size: 25m
    tls_args:
    - --tlsverify
    - --tlscacert=/etc/docker/tls/ca.pem
    - --tlscert=/etc/docker/tls/server-cert.pem
    - --tlskey=/etc/docker/tls/server-key.pem
    - -H=0.0.0.0:2376
  environment:
    EXTRA_CMDLINE: /init
    HTTPS_PROXY: http://172.26.0.34:8080
    HTTP_PROXY: http://172.26.0.34:8080
    NO_PROXY: localhost,127.0.0.1,*.example.local,10.*,192.168.*
    REGISTRY_DOMAIN: ""
    SUFFIX: null
    VERSION: v1.5.8
    http_proxy: http://172.26.0.34:8080
    https_proxy: http://172.26.0.34:8080
    no_proxy: localhost,127.0.0.1,*.example.local,10.*,192.168.*
  http_load_retries: 6
  hypervisor_service: true
  network:
    dns:
      nameservers:
      - 172.26.16.2
      - 172.26.16.8
      override: true
      search:
      - example.local
    http_proxy: http://172.26.0.34:8080
    https_proxy: http://172.26.0.34:8080
    interfaces:
      eth0:
        address: 172.26.16.162/24
        dhcp: false
        gateway: 172.26.16.254
        mtu: 1500
    no_proxy: localhost,127.0.0.1,*.example.local,10.*,192.168.*
  repositories:
    core:
      url: https://raw.githubusercontent.com/rancher/os-services/v1.5.8
  resize_device: /dev/sda
  services:
    acpid:
      command: /usr/sbin/acpid -f
      image: rancher/os-acpid:v1.5.8
      labels:
        io.rancher.os.scope: system
      net: host
      privileged: true
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    all-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.docker.compose.rebuild: always
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes_from:
      - container-data-volumes
      - command-volumes
      - media-volumes
      - user-volumes
      - system-volumes
    cloud-init-execute:
      command: cloud-init-execute -pre-console
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.after: ntp
        io.rancher.os.detach: "false"
        io.rancher.os.scope: system
      net: host
      privileged: true
      uts: host
      volumes:
      - /usr/bin/ros:/usr/bin/ros:ro
      volumes_from:
      - system-volumes
    command-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes:
      - /usr/bin/ros:/usr/bin/ros:ro
      - /usr/bin/system-docker:/usr/bin/system-docker:ro
      - /usr/bin/system-docker-runc:/usr/bin/system-docker-runc:ro
    console:
      command: ros console-init
      environment:
      - HTTP_PROXY
      - HTTPS_PROXY
      - NO_PROXY
      image: rancher/os-console:v1.5.8
      ipc: host
      labels:
      - io.docker.compose.rebuild=always
      - io.rancher.os.after=network
      - io.rancher.os.console=default
      - io.rancher.os.scope=system
      net: host
      pid: host
      privileged: true
      restart: always
      uts: host
      volumes:
      - /usr/bin/iptables:/sbin/iptables:ro
      volumes_from:
      - all-volumes
    container-data-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes:
      - /var/lib/user-docker:/var/lib/docker
      - /var/lib/m-user-docker:/var/lib/m-user-docker
    docker:
      command: ros user-docker
      environment:
      - HTTP_PROXY
      - HTTPS_PROXY
      - NO_PROXY
      image: rancher/os-docker:19.03.15
      ipc: host
      labels:
        io.rancher.os.after: console
        io.rancher.os.scope: system
      net: host
      pid: host
      privileged: true
      restart: always
      uts: host
      volumes:
      - /sys:/host/sys
      - /var/lib/system-docker:/var/lib/system-docker:shared
      volumes_from:
      - all-volumes
    logrotate:
      command: /usr/sbin/logrotate -v /etc/logrotate.conf
      image: rancher/os-logrotate:v1.5.8
      labels:
        cron.schedule: '@hourly'
        io.rancher.os.before: system-cron
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      net: none
      privileged: true
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    media-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes:
      - /media:/media:shared
      - /mnt:/mnt:shared
    network:
      command: netconf
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.after: udev
        io.rancher.os.reloadconfig: "true"
        io.rancher.os.scope: system
      net: host
      pid: host
      privileged: true
      uts: host
      volumes:
      - /usr/bin/iptables:/sbin/iptables:ro
      volumes_from:
      - system-volumes
      - command-volumes
    ntp:
      command: /bin/start_ntp.sh
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.after: network
        io.rancher.os.scope: system
      net: host
      privileged: true
      restart: always
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    preload-user-images:
      command: ros preload-images
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.after: console
        io.rancher.os.detach: "false"
        io.rancher.os.scope: system
      net: host
      privileged: true
      volumes_from:
      - command-volumes
      - system-volumes
    subscriber:
      command: os-subscriber
      environment:
      - IN_DOCKER=true
      image: rancher/os-base:v1.5.8
      ipc: host
      labels:
        cron.schedule: 0 0 3 * * ?
        io.rancher.os.after: console
        io.rancher.os.scope: system
      net: host
      pid: host
      privileged: true
      uts: host
      volumes:
      - /sys:/host/sys
      volumes_from:
      - command-volumes
      - system-volumes
    syslog:
      command: rsyslogd -n
      image: rancher/os-syslog:v1.5.8
      labels:
        io.rancher.os.scope: system
      log_driver: json-file
      net: host
      privileged: true
      restart: always
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    system-cron:
      environment:
        DOCKER_API_VERSION: "1.22"
      image: rancher/container-crontab:v0.4.0
      labels:
        io.rancher.os.scope: system
      net: none
      privileged: true
      restart: always
      uts: host
      volumes:
      - /var/run/system-docker.sock:/var/run/docker.sock
    system-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes:
      - /dev:/host/dev
      - /etc/docker:/etc/docker
      - /etc/hosts:/etc/hosts
      - /etc/logrotate.d:/etc/logrotate.d
      - /etc/resolv.conf:/etc/resolv.conf
      - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt.rancher
      - /etc/selinux:/etc/selinux
      - /lib/firmware:/lib/firmware
      - /lib/modules:/lib/modules
      - /run:/run
      - /usr/share/ros:/usr/share/ros
      - /var/lib/boot2docker:/var/lib/boot2docker
      - /var/lib/rancher/cache:/var/lib/rancher/cache
      - /var/lib/rancher/conf:/var/lib/rancher/conf
      - /var/lib/rancher:/var/lib/rancher
      - /var/lib/waagent:/var/lib/waagent
      - /var/log:/var/log
      - /var/run:/var/run
    udev:
      command: udevd
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.after: udev-cold
        io.rancher.os.detach: "true"
        io.rancher.os.scope: system
      net: host
      privileged: true
      restart: always
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    udev-cold:
      command: ros udev-settle
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.detach: "false"
        io.rancher.os.scope: system
      net: host
      privileged: true
      uts: host
      volumes_from:
      - command-volumes
      - system-volumes
    user-volumes:
      command: echo
      image: rancher/os-base:v1.5.8
      labels:
        io.rancher.os.createonly: "true"
        io.rancher.os.scope: system
      log_driver: json-file
      net: none
      privileged: true
      read_only: true
      volumes:
      - /home:/home
      - /opt:/opt
      - /var/lib/kubelet:/var/lib/kubelet
  services_include:
    container-cron: true
    open-vm-tools: false
    qemu-guest-agent: true
  shutdown_timeout: 60
  ssh:
    keys:
      dsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      dsa-pub: |+
        ssh-dss AAAA==
      ecdsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      ecdsa-pub: |+
        ecdsa-sha2-nistp256 AAAAE=
      ed25519: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      ed25519-pub: |+
        ssh-ed25519 AAAAC==
      rsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      rsa-pub: |+
        ssh-rsa AAAAB==
  state:
    boot_dev: LABEL=RANCHER_BOOT
    boot_fstype: auto
    dev: LABEL=RANCHER_STATE
    fstype: auto
    oem_dev: LABEL=RANCHER_OEM
    oem_fstype: auto
    rngd: true
    wait: true
  sysctl:
    fs.file-max: 1000000000
  system_docker:
    bip: 172.18.42.1/16
    config_file: /etc/docker/system-docker.json
    environment:
    - http_proxy=http://172.26.0.34:8080
    - https_proxy=http://172.26.0.34:8080
    - no_proxy=localhost,127.0.0.1,*.example.local,10.*,192.168.*
    - NO_PROXY=localhost,127.0.0.1,*.example.local,10.*,192.168.*
    exec: true
    exec_root: /var/run/system-docker
    graph: /var/lib/system-docker
    group: root
    host:
    - unix:///var/run/system-docker.sock
    log_opts:
      max-file: 2
      max-size: 25m
    pid_file: /var/run/system-docker.pid
    registry_mirror: https://machine134.example.local
    restart: false
    storage_driver: overlay2
    userland_proxy: false
  upgrade:
    image: burmilla/os
    policy: download
    url: https://raw.githubusercontent.com/burmilla/releases/v2.0.x/releases.yml
runcmd:
- mkdir -p /opt
- mount -t ext4 /dev/sdb /opt || (mkfs.ext4 /dev/sdb && mount -t ext4 /dev/sdb /opt/)
ssh_authorized_keys:
- ssh-rsa AAA==
- ssh-rsa AAAA==
- ssh-rsa AAAAB==
- ssh-rsa AAAA+328==
write_files:
- content: |+
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  owner: root
  path: /etc/docker/certs.d/machine134.example.local/ca.crt
  permissions: "0644"
olljanat commented 4 months ago

hmm. not sure how you ended up to that situation but now your cloud-config.yml simply overwrites everything coming from BurmillaOS v2.0.1 with RancherOS 1.5.8 settings.

It should be something like this (assuming that I picked your needed customizations correctly):

hostname: machine62.example.local
mounts:
- - /dev/sdb
  - /opt/
  - ext4
  - ""
rancher:
  cloud_init:
    datasources:
    - vmware
  console: default
  environment:
    HTTPS_PROXY: http://172.26.0.34:8080
    HTTP_PROXY: http://172.26.0.34:8080
    NO_PROXY: localhost,127.0.0.1,*.example.local,10.*,192.168.*
  hypervisor_service: true
  network:
    dns:
      nameservers:
      - 172.26.16.2
      - 172.26.16.8
      override: true
      search:
      - example.local
    http_proxy: http://172.26.0.34:8080
    https_proxy: http://172.26.0.34:8080
    interfaces:
      eth0:
        address: 172.26.16.162/24
        dhcp: false
        gateway: 172.26.16.254
        mtu: 1500
    no_proxy: localhost,127.0.0.1,*.example.local,10.*,192.168.*
  resize_device: /dev/sda
  services_include:
    container-cron: true
    open-vm-tools: false
    qemu-guest-agent: true
  shutdown_timeout: 60
  ssh:
    keys:
      dsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      dsa-pub: |+
        ssh-dss AAAA==
      ecdsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      ecdsa-pub: |+
        ecdsa-sha2-nistp256 AAAAE=
      ed25519: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      ed25519-pub: |+
        ssh-ed25519 AAAAC==
      rsa: |+
        -----BEGIN OPENSSH PRIVATE KEY-----
        -----END OPENSSH PRIVATE KEY-----
      rsa-pub: |+
        ssh-rsa AAAAB==
  upgrade:
    image: burmilla/os
    policy: download
    url: https://raw.githubusercontent.com/burmilla/releases/v2.0.x/releases.yml
runcmd:
- mkdir -p /opt
- mount -t ext4 /dev/sdb /opt || (mkfs.ext4 /dev/sdb && mount -t ext4 /dev/sdb /opt/)
ssh_authorized_keys:
- ssh-rsa AAA==
- ssh-rsa AAAA==
- ssh-rsa AAAAB==
- ssh-rsa AAAA+328==
write_files:
- content: |+
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  owner: root
  path: /etc/docker/certs.d/machine134.example.local/ca.crt
  permissions: "0644"
stevecharon commented 4 months ago

Hi, I can't quite place the difference here. You omitted some parts of my config, like state:, cloud_init_services:, defaults: and bootstrap: Did you post the wrong config here? as told, I can revert to the non-mirrored setup and post that again, if that helps. That was the initial working setup until upgrading to 2.x

But it seems easier to just roll a new VM with 2.x and transfer everything. After all, the pet is starting to annoy me.

Our mirror is just a jfrog artifactory-jcr machine which is pulling from hub.docker.com so docker pull burmila/os-base:v2.0.1 becomes docker pull my.artifactory.host/docker-remote/burmilla/os-base:v2.0.1 and it takes care of proxy, docker-login I tried to take the registry-mirror setting "my.artifactory.host" with the image prefix "docker-remote" but that works just for docker and system-docker, but not for ros os update.

olljanat commented 4 months ago

You omitted some parts of my config, like state:, cloud_init_services:, defaults: and bootstrap:

Assuming that I got it right that you pasted content of /var/lib/rancher/conf/cloud-config.yml file here, then yes, should not ever define those parts as they overwrites what is included to OS.

Two useful commands to help understanding who it works. Command sudo ros config export prints what ever is included to your custom cloud-init (which why we ask it on our issue template). Command sudo ros config export -full prints complete cloud-init included OS after it is merged with your custom cloud-init and only this should contains things like bootstrap, cloud_init_services, etc...

stevecharon commented 4 months ago

Well I am sure I did not include something like bootstrap in there on purpose. So maybe by accidentally exporting the full version and merging it there. Seems an easy mistake on my part and maybe leeds to the effects I am seeing? Will clean that up and cycle the system. Thanks for pointing that out

stevecharon commented 4 months ago

Yes! that was it. Did an ros os upgrade -f and afterwards all containers showed with burmilla except console which I expected as in the docs. Only thing is qemugeustagent seems to restart now every minute. But that is a problem for another day. Docker is with current default:

sudo ros engine list|grep current current docker-24.0.9

Thanks for your patience with me.