Error: cannot add image, failed to update archive/DB Could not write to file: No space left on device

[Tim-Schwalbe]

anchore/engine-cli:v0.9.1

What happened:

Error: cannot add image, failed to update archive/DB
HTTP Code: 500
Detail: {'error_codes': [], 'raw_exception_message': '(psycopg2.errors.DiskFull) could not access status of transaction 0\nDETAIL:  Could not write to file "pg_subtrans/0198" at offset 16384: No space left on device.\n\n[SQL: INSERT INTO archive_document (bucket, "archiveId", "userId", "documentName", created_at, last_updated, record_state_key, record_state_val, jsondata, b64_encoded) VALUES (%(bucket)s, %(archiveId)s, %(userId)s, %(documentName)s, %(created_at)s, %(last_updated)s, %(record_state_key)s, %(record_state_val)s, %(jsondata)s, %(b64_encoded)s)]\n[parameters: {\'bucket\': \'manifest_data\', \'archiveId\': \'sha256:b17848d9e9e03bf56a2e956f228d891303a04de31067508465e41c274ee9c5[33](https://gitlab.com/saletelligence/gilead-spl/-/jobs/2055356717#L33)\', \'userId\': \'admin\', \'documentName\': \'sha256:b17848d9e9e03bf56a2e956f228d891303a04de31067508465e41c274ee9c533.json\', \'created_at\': 1643986574, \'last_updated\': 1643986574, \'record_state_key\': \'active\', \'record_state_val\': None, \'jsondata\': \'{"document": "{\\\\"schemaVersion\\\\": 2, \\\\"mediaType\\\\": \\\\"application/vnd.docker.distribution.manifest.v2+json\\\\", \\\\"config\\\\": {\\\\"mediaType\\\\": \\ ... (985 characters truncated) ... mage.rootfs.diff.tar.gzip\\\\", \\\\"size\\\\": 123773399, \\\\"digest\\\\": \\\\"sha256:7123b7194a88bf3c02de443b814eae235d4a594f6d9badd8dd30c1f6302cf20f\\\\"}]}"}\', \'b64_encoded\': False}]\n(Background on this error at: https://sqlalche.me/e/14/e3q8)'}

I already delete all images, but still the same error.

      for i in `anchore-cli image list | grep 'eu.gcr.io' | awk '{print $1}' | sort | uniq`  
      do 
        anchore-cli image del ${i} --force > /dev/null && echo "deleted image ${1}" || echo "could not delete image ${i}"
      done

What other things could be the issue here? The feeds itself got too large?

[zhill]

Hi @Tim-Schwalbe! Howe big is your DB disk/volume? Typically this is due to the vacuum settings on postgres to ensure it actually frees the disk space back. See the postgres VACUUM and autovacuum docs: https://www.postgresql.org/docs/current/runtime-config-autovacuum.html It will depend on how your DB is deployed and it's default conf.

Anchore does delete asynchronously, so the API operation just marks the image for deletion, but it should catch up and delete the images within minutes.

[Tim-Schwalbe]

@zhill the disk is only 10 GB large. I did not edit any setting and I am using the official helm chart.

name: anchore-engine
version: 1.15.0
appVersion: 1.0.0

Maybe there should be a setting in the values.yaml

This is all to configure regarding postgres.

postgresql:
  # To use an external DB or Google CloudSQL in GKE, uncomment & set 'enabled: false'
  # externalEndpoint, postgresUser, postgresPassword & postgresDatabase are required values for external postgres
  # enabled: false
  # If enabled=false specify an external (already existing) postgres deployment for use.
  # Set to the host and port. eg. mypostgres.myserver.io:5432
  externalEndpoint: Null
  postgresUser: anchoreengine
  postgresPassword: anchore-postgres,123
  postgresDatabase: anchore

  # Configure size of the persistent volume used with helm managed chart.
  # This should be commented out if using an external endpoint.
  persistence:
    resourcePolicy: "keep"
    size: 10Gi