High CPU usage at idle

[richstokes]

anchore-db_1 | WARNING: there is no transaction in progress gets spammed as fast as possible (seems to come from postgres), using very high amount of CPU even at idle.

Process using high cpu: /usr/bin/python2 /bin/twistd --logger=anchore_engine.subsys.twistd_logger.logger --pidfile /var/run/anchore-simplequeue.pid -n anchore-simplequeue --config /config

anchore-cli --version
anchore-cli, version 0.2.4

anchore-cli system status
Service simplequeue (dockerhostid-anchore-engine, http://anchore-engine:8083): up
Service catalog (dockerhostid-anchore-engine, http://anchore-engine:8082): up
Service apiext (dockerhostid-anchore-engine, http://anchore-engine:8228): up
Service kubernetes_webhook (dockerhostid-anchore-engine, http://anchore-engine:8338): up
Service analyzer (dockerhostid-anchore-engine, http://anchore-engine:8084): up
Service policy_engine (dockerhostid-anchore-engine, http://anchore-engine:8087): up

Engine DB Version: 0.0.7
Engine Code Version: 0.2.4

docker exec 6c pip show anchore-engine
Name: anchore-engine
Version: 0.2.4
Summary: Anchore Engine
Home-page: http://www.anchore.com
Author: Anchore Inc.
Author-email: dev@anchore.com
License: Apache License 2.0
Location: /usr/lib/python2.7/site-packages
Requires: Twisted, pg8000, boto3, botocore, chardet, click, clickclick, connexion, docker-py, enum34, docutils, Flask, functools32, futures, idna, inflection, itsdangerous, Jinja2, jmespath, jsonschema, MarkupSafe, pathlib, prettytable, python-dateutil, PyYAML, requests, s3transfer, six, SQLAlchemy, swagger-spec-validator, typing, urllib3, watchdog, Werkzeug, passlib, prometheus-client, prometheus-flask-exporter, psutil, python-swiftclient, python-keystoneclient, pytz, certifi
Required-by: 

How to reproduce the issue: Fresh install

Anything else we need to know: Ubuntu 18.04.1 LTS

docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:24:51 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:23:15 2018
  OS/Arch:          linux/amd64
  Experimental:     false

docker-compose version
docker-compose version 1.22.0, build f46880fe
docker-py version: 3.4.1
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.1.0f  25 May 2017

[zhill]

Hi @richstokes , thanks for reaching out. To help better understand, can you give an idea of what "high" CPU usage means? Is it something like 20-30% at "idle" or are you seeing 80-100%? Generally speaking, the system is never really idle, since it is constantly processing tag update checks, policy evaluations and cve feed updates, so you should see some usage, and that will vary based on the number of images and subscribed tags (things to watch in external docker registries) in your system. Can you provide a bit more detail on that state of your system to help triage further? Thanks!

[richstokes]

Sure, the server is an AWS t2.medium (4Gb memory). The system is set up for anchore testing only, so even when nothing else is happening on the box the 'twistd' process is using about 15% CPU. I've not added any new scans since my initial testing a few days ago and the CPU use remains pretty consistent.

If this is expected then fair enough, just feels like its spinning its wheels unnecessarily. :-]

[zhill]

@richstokes understood. Right now, that's about normal. It does feed "busy", but anchore is a very asynchronous system that has a lot of background tasks that have duty cycles. We're aware of the issue and will work to help reduce that, but in the current design that is expected behavior. The db logging thing is on our radar as well. I've been looking into approaches to remove the need for active polling, such as redis-based pub-sub but that involves more dependencies to install/run.