10% DNS Failures with Alpine 3.19 or 3.20

[richardtearle-nps]

We are having trouble with DNS lookups when using Alpine 3.19 or 3.20 images when using podman on Oracle Linux 8. Our actual application is Ruby based which "sometimes" fails to find a redis container on the same podman network. I can reproduce the issue using the following on our network:

docker-compose.wml

version: '2.1'
services:
        httpd:
                image: <local-container-registry>/httpd:2.4.59-alpine3.19
                networks:
                        - common
                ports:
                         - 1080:80
                container_name: httpd
                hostname: httpd
        test:
                image: <local-container-registry>/alpine:3.19
                networks:
                        - common
                container_name: test
                hostname: test
                entrypoint: /usr/local/bin/entrypoint.sh
                volumes:
                        - ./entrypoint.sh:/usr/local/bin/entrypoint.sh:Z

networks:
        common:
                driver: bridge

entrypoint.sh

#!/bin/ash

while (true) ; do
        ERROR=$(ping -c 1 -q httpd 2>&1)
        if [ "$?" -ne "0" ]; then
                echo $(date -Is) failed with: ${ERROR}
        fi
        sleep 1s
done

We see around 10% DNS lookup failures when running the above for around an hour.

We've tried the following, but still get the same failure rate:

• Changed the host name to match the service/container name (as above)
• Added the proxy settings via env. vars
• Started with podman-compose rather than docker-compose
• Upgraded to Alpine 3.20 (alpine:3.20)
• Used FQDN

What does work is either switching back to docker, or switching to a different flavour of Linux (we've tried OL8-slim and Debian variants)