search

cioos-siooc / ckan

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers datahub.io, catalog.data.gov and europeandataportal.eu/data/en/dataset among many other sites.
http://ckan.org/
Other
2 stars 4 forks source link

Limit Container CPU usage #86

Closed fostermh closed 2 years ago

fostermh commented 3 years ago

Note this will require starting the containers using the --compatibility flag otherwise the CPU directives will be ignored. example:

sudo docker-compose --compatibility up -d

This addresses an issue in which SOLR or other containers consume all available CPU resources and crash. Often this happens during high load situations such as harvesting thousands of datasets at a time while also indexing the XML.

This PR also adds several environment variables that can be used to set the limits per container. if not environment variable is set or it is blank the default values will be used as indicated by the number following the ':-'

new enviroment variables:

LIMIT_CPU_CKAN
LIMIT_CPU_CKAN_GATHER_HARVESTER
LIMIT_CPU_CKAN_FETCH_HARVESTER
LIMIT_CPU_CKAN_RUN_HARVESTER
LIMIT_CPU_DATAPUSHER
LIMIT_CPU_DB
LIMIT_CPU_SOLR
LIMIT_CPU_REDIS
github-actions[bot] commented 3 years ago

Image has been pushed to cioos/ckan

Testing Quick Start

Pull image

sudo docker pull cioos/ckan:PR86 or sudo CKAN_TAG=PR86 docker-compose pull ckan

Remove Home Volume and Restart

sudo docker-compose down
sudo docker volume rm docker_ckan_home
sudo docker-compose up -d

for full documentation see TBD

github-actions[bot] commented 2 years ago

Image has been pushed to cioos/ckan

Testing Quick Start

Pull image

sudo docker pull cioos/ckan:PR86 or sudo CKAN_TAG=PR86 docker-compose pull ckan

Remove Home Volume and Restart

sudo docker-compose down
sudo docker volume rm docker_ckan_home
sudo CKAN_TAG=PR86 docker-compose up -d

for full documentation see TBD