Open dj13may91 opened 1 month ago
Fixed above for now,: fixes:
Morded --> has /tmp/ mounted which had permission issue
kibiter --> used kibana and ES for now
Now facing below error:
Traceback (most recent call last): File "/usr/local/bin/sirmordred", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 64, in main debug_mode = config_dict['general']['debug'] KeyError: 'debug'
@zhquan please help with above
Hi @dj13may91
Your setup.cfg
seems incorrect or your volume is not mounted correctly. Could you share your docker-compose.yml
and the path of your setup.cfg
and docker-compose.yml
?
I am using the default that is present in project. Expecting this setup to pull data of grimoire lab project. This is my setup.cfg, no changes : https://github.com/chaoss/grimoirelab/blob/master/default-grimoirelab-settings/setup-opensearch.cfg Also, using a modified docker compose with default elastic versions, but still not able to get the project up and running. Docker compose :
version: "3.8"
volumes: certs: driver: local esdata01: driver: local kibanadata: driver: local sortinghat-static: mariadb_data:
networks: default: name: elastic external: false
services: setup: image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} volumes:
certs:/usr/share/elasticsearch/config/certs user: "0" command: > bash -c ' if [ x${ELASTIC_PASSWORD} == x ]; then echo "Set the ELASTIC_PASSWORD environment variable in the .env file"; exit 1; elif [ x${KIBANA_PASSWORD} == x ]; then echo "Set the KIBANA_PASSWORD environment variable in the .env file"; exit 1; fi; if [ ! -f config/certs/ca.zip ]; then echo "Creating CA"; bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip; unzip config/certs/ca.zip -d config/certs; fi; if [ ! -f config/certs/certs.zip ]; then echo "Creating certs"; echo -ne \ "instances:\n"\ " - name: es01\n"\ " dns:\n"\ " - es01\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ " - name: kibana\n"\ " dns:\n"\ " - kibana\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\
config/certs/instances.yml; bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key; unzip config/certs/certs.zip -d config/certs; fi; echo "Setting file permissions" chown -R root:root config/certs; find . -type d -exec chmod 750 {} \;; find . -type f -exec chmod 640 {} \;; echo "Waiting for Elasticsearch availability"; until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done; echo "Setting kibana_system password"; until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done; echo "All done!"; ' healthcheck: test: [ "CMD-SHELL", "[ -f config/certs/es01/es01.crt ]" ] interval: 1s timeout: 5s retries: 120
es01: depends_on: setup: condition: service_healthy image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} labels: co.elastic.logs/module: elasticsearch volumes:
xpack.license.self_generated.type=${LICENSE} mem_limit: ${ES_MEM_LIMIT} ulimits: memlock: soft: -1 hard: -1 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120
kibana: depends_on: es01: condition: service_healthy image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
volumes:
ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
mem_limit: ${KB_MEM_LIMIT} healthcheck: test: [ "CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'", ] interval: 10s timeout: 10s retries: 120
mariadb: image: mariadb:10.6 expose:
mariadb_data:/var/lib/mysql
redis: image: amd64/redis expose:
"6379" healthcheck: test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ] retries: 5
sortinghat: restart: on-failure:3 image: grimoirelab/sortinghat environment:
sortinghat-static:/opt/venv/lib/python3.9/site-packages/sortinghat/static/ depends_on: mariadb: condition: service_healthy redis: condition: service_healthy
sortinghat_worker: image: grimoirelab/sortinghat-worker environment:
SORTINGHAT_REDIS_PASSWORD= depends_on: mariadb: condition: service_healthy redis: condition: service_healthy
mordred: restart: on-failure:5 image: grimoirelab/grimoirelab:latest volumes:
../docker-compose/logs:/home/grimoire/logs
mem_limit: 4g
nginx: restart: on-failure:3 image: nginx:latest volumes:
I don't have any issues running on Linux using the default configurations.
$ docker-compose -f docker-compose-opensearch.yml up -d
Starting docker-compose_redis_1 ... done
Starting docker-compose_opensearch-node1_1 ... done
Starting docker-compose_mariadb_1 ... done
Starting docker-compose_opensearch-dashboards_1 ... done
Starting docker-compose_sortinghat_1 ... done
Starting docker-compose_sortinghat_worker_1 ... done
Starting docker-compose_nginx_1 ... done
Starting docker-compose_mordred_1 ... done
GET _cat/indices?s=i
yellow open git-aoc_demo_enriched O2Ku2SnXSe6bikKWdi6Lsg 1 1 3921 0 5.5mb 5.5mb
yellow open git-onion_demo_enriched 4u9miSmfQUeGgnRcYUXIEw 1 1 536 0 123.4kb 123.4kb
yellow open git_demo_enriched DqgGv0uoQfCP-zWjGLnThg 1 1 754 0 9.6mb 9.6mb
yellow open git_demo_raw N3Y7Q6QrRrOUhttRCO8Raw 1 1 754 3 956.7kb 956.7kb
But I can reproduce the error if I change the permission of the file to `600` (`docker` user cannot read the `setup.cfg` file)
$ docker logs --tail=10 docker-compose_mordred_1
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 64, in main
debug_mode = config_dict['general']['debug']
KeyError: 'debug'
Traceback (most recent call last):
File "/usr/local/bin/sirmordred", line 8, in
Could you try again but change the permission to `777` for `../default-grimoirelab-settings/setup-opensearch.cfg` and `../default-grimoirelab-settings/projects.json`?
Trying to run out of the box service but not able to get it running on M1 pro. Using branch : master