search

quay / clair

Vulnerability Static Analysis for Containers
https://quay.github.io/clair/
Apache License 2.0
10.36k stars 1.16k forks source link

Error during the internal updaters process for rhel, alpine and ubuntu url #1933

Open flomickl opened 11 months ago

flomickl commented 11 months ago

Description of Problem / Feature Request

In my setup, I get misc error messages during the internal updaters process. I am using clair v4 and version 4.7.2 but also switched to older versions with the same error outcome.

Expected Outcome

The expected outcome is no error messages and the information is stored in the clair database.

Actual Outcome

clair-indexer   | {"level":"error","component":"rhel/internal/common/Updater.Get","error":"Get \"https://access.redhat.com/security/data/metrics/container-name-repos-map.json\": context deadline exceeded","time":"2023-12-17T12:02:53Z","message":"error updating mapping file"}
clair-indexer   | {"level":"error","version":"1.1","component":"rhel/internal/common/Updater.Get","error":"Get \"https://access.redhat.com/security/data/metrics/repository-to-cpe.json\": context deadline exceeded","time":"2023-12-17T12:03:03Z","message":"error updating mapping file"}
clair-matcher   | {"level":"error","component":"libvuln/updates/Manager.Run","error":"alpine: error requesting \"https://secdb.alpinelinux.org/last-update\": Get \"https://secdb.alpinelinux.org/last-update\": dial tcp 172.105.78.12:443: i/o timeout","time":"2023-12-17T12:03:13Z","message":"failed constructing factory, excluding from run"}
clair-matcher   | {"level":"error","component":"libvuln/updates/Manager.Run","error":"debian: examining remote: debian: unable to do request: Get \"https://deb.debian.org/debian/dists/\": dial tcp 146.75.118.132:443: i/o timeout","time":"2023-12-17T12:03:43Z","message":"failed constructing factory, excluding from run"}
clair-matcher   | {"level":"error","component":"libvuln/updates/Manager.Run","error":"Get \"https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST\": dial tcp 23.213.161.217:443: i/o timeout","time":"2023-12-17T12:04:13Z","message":"failed constructing factory, excluding from run"}
clair-matcher   | {"level":"error","component":"libvuln/updates/Manager.Run","error":"ubuntu: error requesting series collection: Get \"https://api.launchpad.net/1.0/ubuntu/series\": dial tcp 185.125.189.224:443: i/o timeout","time":"2023-12-17T12:04:43Z","message":"failed constructing factory, excluding from run"}
clair-matcher   | {"level":"error","component":"libvuln/updates/Manager.Start","error":"updating errors:\naws-AL2-updater: failed to create client: failed to make request for mirrors: Get \"https://cdn.amazonlinux.com/2/core/latest/x86_64/mirror.list\": context deadline exceeded\naws-AL1-updater: failed to create client: failed to make request for mirrors: Get \"http://repo.us-west-2.amazonaws.com/2018.03/updates/x86_64/mirror.list\": context deadline exceeded\nsuse-updater-suse.linux.enterprise.server.12: Get \"https://support.novell.com/security/oval/suse.linux.enterprise.server.12.xml\": dial tcp 130.57.66.5:443: i/o timeout\nphoton-updater-photon2: Get \"https://packages.vmware.com/photon/photon_oval_definitions/com.vmware.phsa-photon2.xml\": dial tcp 2.18.160.25:443: i/o timeout\naws-AL2023-updater: failed to create client: failed to make request for mirrors: Get \"https://cdn.amazonlinux.com/al2023/core/mirrors/latest/x86_64/mirror.list\": context deadline exceeded\nphoton-updater-photon1: Get \"https://packages.vmware.com/photon/photon_oval_definitions/com.vmware.phsa-photon1.xml\": dial tcp 2.18.160.25:443: i/o timeout\nsuse-updater-suse.linux.enterprise.server.15: Get \"https://support.novell.com/security/oval/suse.linux.enterprise.server.15.xml\": dial tcp 130.57.66.5:443: i/o timeout\nsuse-updater-opensuse.leap.15.1: Get \"https://support.novell.com/security/oval/opensuse.leap.15.1.xml\": dial tcp 130.57.66.5:443: i/o timeout\nphoton-updater-photon3: Get \"https://packages.vmware.com/photon/photon_oval_definitions/com.vmware.phsa-photon3.xml\": dial tcp 2.18.160.25:443: i/o timeout\nsuse-updater-opensuse.leap.15.0: Get \"https://support.novell.com/security/oval/opensuse.leap.15.0.xml\": dial tcp 130.57.66.5:443: i/o timeout\nsuse-updater-suse.linux.enterprise.server.11: Get \"https://support.novell.com/security/oval/suse.linux.enterprise.server.11.xml\": dial tcp 130.57.66.5:443: i/o timeout\noracle-2011-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2011.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2012-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2012.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2013-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2013.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2019-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2019.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2020-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2020.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2010-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2010.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2015-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2015.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2018-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2018.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2007-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2007.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2014-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2014.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2016-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2016.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2017-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2017.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2008-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2008.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2009-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2009.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2021-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2021.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\noracle-2022-updater: Get \"https://linux.oracle.com/security/oval/com.oracle.elsa-2022.xml.bz2\": dial tcp 95.101.178.195:443: i/o timeout\n","time":"2023-12-17T12:06:13Z","message":"errors encountered during updater run"}

Environment

My compose setup:

services:
  matcher:
    image: quay.io/projectquay/clair:4.7.2
    depends_on:
      clair-database:
        condition: service_healthy
    environment:
      CLAIR_MODE: matcher
      CLAIR_CONF: /config/config.yaml
    volumes:
      - ./clair-config/:/config
    restart: unless-stopped
    container_name: clair-matcher
    networks:
      - clair-network

  indexer:
    image: quay.io/projectquay/clair:4.7.2
    depends_on:
      clair-database:
        condition: service_healthy
    volumes:
      - ./clair-config/:/config
    restart: unless-stopped
    container_name: clair-indexer
    environment:
      CLAIR_MODE: "indexer"
      CLAIR_CONF: /config/config.yaml
    networks:
      - clair-network

  clair-database:
    container_name: clair-database
    image: docker.io/library/postgres:13
    environment:
      POSTGRES_HOST_AUTH_METHOD: trust
    volumes:
      - ./config/init.sql:/docker-entrypoint-initdb.d/init.sql
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - ./data/postgres:/var/lib/postgresql/data
    healthcheck:
      test:
        - CMD-SHELL
        - "pg_isready -U postgres"
      interval: 5s
      timeout: 4s
      retries: 12
      start_period: 10s
    networks:
      - clair-network

networks:
  clair-network:
    driver: bridge
    internal: true

My InitSQL File

CREATE USER clair WITH PASSWORD 'clair';
CREATE USER quay WITH PASSWORD 'quay';
CREATE DATABASE indexer WITH OWNER clair;
CREATE DATABASE matcher WITH OWNER clair;
CREATE DATABASE notifier WITH OWNER clair;
CREATE DATABASE quay WITH OWNER quay;
\connect matcher
CREATE EXTENSION "uuid-ossp";
\connect notifier
CREATE EXTENSION "uuid-ossp";
\connect quay
CREATE EXTENSION "pg_trgm";

My clair config.yaml

# ===== MATCHER
matcher:                            # Matcher provides Clair matcher node configuration.
  connstring: "host=clair-database port=5432 user=clair dbname=matcher sslmode=disable" # libpq connection string.
  indexer_addr: "clair-indexer:6060" # A Matcher contacts an Indexer to create a VulnerabilityReport. Required!
  # cache_age:                      # Controls how long clients should be hinted to cache responses for.
  migrations: true                  # Whether Matcher nodes handle migrations to their databases.
  period: "1h"                      # Determines how often updates for new security advisories will take place. Default 6h.
  disable_updaters: false           # Whether to run background updates or not.
  update_retention: 2               # Sets the number of update operations to retain between garbage collection cycles. Default 10.
matchers:                           # Matchers provides configuration for the in-tree Matchers and RemoteMatchers.
  names:                            # A list of string values informing the matcher factory about enabled matchers. 
    - alpine
    - aws
    - debian
    - oracle
    - photon
    - python
    - rhel
    - suse
    - ubuntu
    - crda
  config: {}                         # Provides configuration to specific matcher. Example https://quay.github.io/clair/reference/config.html#matchersconfig
updaters:                            # Updaters provides configuration for the Matcher's update manager.
  sets:                              # A list of string values informing the update manager which Updaters to run. If value is nil default set of Updaters will run.
    - alpine
    - aws
    - debian
    - oracle
    - photon
    - pyupio
    - rhel
    - suse
    - ubuntu
  config: {}                         # Provides configuration to specific updater sets. Example https://quay.github.io/clair/reference/config.html#updatersconfig
hdonnay commented 10 months ago

Those are i/o timeouts in the log -- are the indicated URLs reachable from the container?