Application not running on M1 pro

[dj13may91]

Trying to run out of the box service but not able to get it running on M1 pro. Using branch : master

Mordred Traceback (most recent call last): File "/usr/local/bin/sirmordred", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 66, in main logger = setup_logs(logs_dir, debug_mode, short_name) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 98, in setup_logs fh = logging.FileHandler(fh_filepath) File "/usr/local/lib/python3.8/logging/__init__.py", line 1147, in __init__ StreamHandler.__init__(self, self._open()) File "/usr/local/lib/python3.8/logging/__init__.py", line 1176, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) PermissionError: [Errno 13] Permission denied: '/home/grimoire/logs/all.log'

kibiter runtime: failed to create new OS thread (have 2 already; errno=22) fatal error: runtime.newosproc runtime stack: runtime.throw(0x56be06) /usr/src/go/src/runtime/panic.go:491 +0xad runtime.newosproc(0xc208020000, 0xc208030000) /usr/src/go/src/runtime/os_linux.c:170 +0x10a newm(0x430700, 0x0) /usr/src/go/src/runtime/proc.c:1157 +0xc5 runtime.newsysmon() /usr/src/go/src/runtime/proc.c:169 +0x33 runtime.onM(0x57cb30) /usr/src/go/src/runtime/asm_amd64.s:257 +0x68 runtime.mstart() /usr/src/go/src/runtime/proc.c:818

[dj13may91]

Fixed above for now,: fixes:

Morded --> has /tmp/ mounted which had permission issue

kibiter --> used kibana and ES for now

Now facing below error: Traceback (most recent call last): File "/usr/local/bin/sirmordred", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 64, in main debug_mode = config_dict['general']['debug'] KeyError: 'debug'

[dj13may91]

@zhquan please help with above

[zhquan]

Hi @dj13may91

Your setup.cfg seems incorrect or your volume is not mounted correctly. Could you share your docker-compose.yml and the path of your setup.cfg and docker-compose.yml?

[dj13may91]

I am using the default that is present in project. Expecting this setup to pull data of grimoire lab project. This is my setup.cfg, no changes : https://github.com/chaoss/grimoirelab/blob/master/default-grimoirelab-settings/setup-opensearch.cfg Also, using a modified docker compose with default elastic versions, but still not able to get the project up and running. Docker compose :

version: "3.8"

volumes:
certs:
driver: local
esdata01:
driver: local
kibanadata:
driver: local
sortinghat-static:
mariadb_data:

networks: default: name: elastic external: false

services: setup: image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} volumes:

• certs:/usr/share/elasticsearch/config/certs user: "0" command: > bash -c ' if [ x${ELASTIC_PASSWORD} == x ]; then echo "Set the ELASTIC_PASSWORD environment variable in the .env file"; exit 1; elif [ x${KIBANA_PASSWORD} == x ]; then echo "Set the KIBANA_PASSWORD environment variable in the .env file"; exit 1; fi; if [ ! -f config/certs/ca.zip ]; then echo "Creating CA"; bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip; unzip config/certs/ca.zip -d config/certs; fi; if [ ! -f config/certs/certs.zip ]; then echo "Creating certs"; echo -ne \ "instances:\n"\ " - name: es01\n"\ " dns:\n"\ " - es01\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ " - name: kibana\n"\ " dns:\n"\ " - kibana\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\

  config/certs/instances.yml; bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key; unzip config/certs/certs.zip -d config/certs; fi; echo "Setting file permissions" chown -R root:root config/certs; find . -type d -exec chmod 750 {} \;; find . -type f -exec chmod 640 {} \;; echo "Waiting for Elasticsearch availability"; until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done; echo "Setting kibana_system password"; until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done; echo "All done!"; ' healthcheck: test: [ "CMD-SHELL", "[ -f config/certs/es01/es01.crt ]" ] interval: 1s timeout: 5s retries: 120

  es01: depends_on: setup: condition: service_healthy image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} labels: co.elastic.logs/module: elasticsearch volumes:

• certs:/usr/share/elasticsearch/config/certs
• esdata01:/usr/share/elasticsearch/data ports:
• ${ES_PORT}:9200 environment:
• node.name=es01
• cluster.name=${CLUSTER_NAME}

  - discovery.type=single-node

• cluster.initial_master_nodes=es01
• ELASTIC_PASSWORD=${ELASTIC_PASSWORD}

  - bootstrap.memory_lock=true

• xpack.security.enabled=true
• xpack.security.http.ssl.enabled=true
• xpack.security.http.ssl.key=certs/es01/es01.key
• xpack.security.http.ssl.certificate=certs/es01/es01.crt
• xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
• xpack.security.transport.ssl.enabled=true
• xpack.security.transport.ssl.key=certs/es01/es01.key
• xpack.security.transport.ssl.certificate=certs/es01/es01.crt
• xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
• xpack.security.transport.ssl.verification_mode=certificate

• xpack.license.self_generated.type=${LICENSE} mem_limit: ${ES_MEM_LIMIT} ulimits: memlock: soft: -1 hard: -1 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120

  kibana: depends_on: es01: condition: service_healthy image: docker.elastic.co/kibana/kibana:${STACK_VERSION}

  labels:

  co.elastic.logs/module: kibana

  volumes:

• certs:/usr/share/kibana/config/certs
• kibanadata:/usr/share/kibana/data ports:
• ${KIBANA_PORT}:5601 environment:
• SERVERNAME=kibana
• ELASTICSEARCH_HOSTS="https://es01:9200"
• ELASTICSEARCH_USERNAME=kibana_system
• ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}

• ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt

  - XPACK_SECURITY_ENCRYPTIONKEY=${ENCRYPTION_KEY}

  - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${ENCRYPTION_KEY}

  - XPACK_REPORTING_ENCRYPTIONKEY=${ENCRYPTION_KEY}

  mem_limit: ${KB_MEM_LIMIT} healthcheck: test: [ "CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'", ] interval: 10s timeout: 10s retries: 120

  mariadb: image: mariadb:10.6 expose:

• "3306" environment:
• MYSQL_ROOT_PASSWORD=
• MYSQL_ALLOW_EMPTY_PASSWORD=yes healthcheck: test: [ "CMD", "/usr/local/bin/healthcheck.sh", "--su=root", "--connect", "--innodb_initialized" ] retries: 5 volumes:

• mariadb_data:/var/lib/mysql

  redis: image: amd64/redis expose:

• "6379" healthcheck: test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ] retries: 5

  sortinghat: restart: on-failure:3 image: grimoirelab/sortinghat environment:

• SORTINGHAT_SECRET_KEY=secret
• SORTINGHAT_DB_HOST=mariadb
• SORTINGHAT_DB_PORT=3306
• SORTINGHAT_DB_DATABASE=sortinghat_db
• SORTINGHAT_DB_USER=root
• SORTINGHAT_DB_PASSWORD=
• SORTINGHAT_REDIS_HOST=redis
• SORTINGHAT_REDIS_PASSWORD=
• SORTINGHAT_SUPERUSER_USERNAME=root
• SORTINGHAT_SUPERUSER_PASSWORD=root
• SORTINGHAT_ALLOWED_HOST=sortinghat,nginx,localhost,127.0.0.1,[::1]
• SORTINGHAT_CORS_ALLOWED_ORIGINS=http://localhost:8000,http://127.0.0.1:8000 expose:
• "9314" volumes:

• sortinghat-static:/opt/venv/lib/python3.9/site-packages/sortinghat/static/ depends_on: mariadb: condition: service_healthy redis: condition: service_healthy

  sortinghat_worker: image: grimoirelab/sortinghat-worker environment:

• SORTINGHAT_SECRET_KEY=secret
• SORTINGHAT_DB_HOST=mariadb
• SORTINGHAT_DB_PORT=3306
• SORTINGHAT_DB_DATABASE=sortinghat_db
• SORTINGHAT_DB_USER=root
• SORTINGHAT_DB_PASSWORD=
• SORTINGHAT_REDIS_HOST=redis

• SORTINGHAT_REDIS_PASSWORD= depends_on: mariadb: condition: service_healthy redis: condition: service_healthy

  mordred: restart: on-failure:5 image: grimoirelab/grimoirelab:latest volumes:

• ../default-grimoirelab-settings/setup-opensearch.cfg:/home/grimoire/conf/setup.cfg
• ../default-grimoirelab-settings/projects.json:/home/grimoire/conf/projects.json

  - /tmp/:/home/grimoire/logs

• ../docker-compose/logs:/home/grimoire/logs

  depends_on:

  nginx:

  condition: service_healthy

  mem_limit: 4g

  nginx: restart: on-failure:3 image: nginx:latest volumes:

• ../default-grimoirelab-settings/nginx.conf.template:/etc/nginx/templates/default.conf.template
• ../default-grimoirelab-settings/uwsgi_params:/etc/nginx/uwsgi_params
• sortinghat-static:/sortinghat:ro ports:
• 8000:8000 environment:
• KIBANA_HOST=http://localhost:5601/ depends_on:
• sortinghat healthcheck: test: "curl -s --head http://localhost:8000/identities/api/ | grep Set-Cookie || exit 1" retries: 5

[zhquan]

I don't have any issues running on Linux using the default configurations.

$ docker-compose -f docker-compose-opensearch.yml up -d
Starting docker-compose_redis_1                 ... done
Starting docker-compose_opensearch-node1_1      ... done
Starting docker-compose_mariadb_1               ... done
Starting docker-compose_opensearch-dashboards_1 ... done
Starting docker-compose_sortinghat_1            ... done
Starting docker-compose_sortinghat_worker_1     ... done
Starting docker-compose_nginx_1                 ... done
Starting docker-compose_mordred_1               ... done

• Here are my indexes

  
  GET _cat/indices?s=i
  yellow open git-aoc_demo_enriched        O2Ku2SnXSe6bikKWdi6Lsg 1 1 3921  0   5.5mb   5.5mb
  yellow open git-onion_demo_enriched      4u9miSmfQUeGgnRcYUXIEw 1 1  536  0 123.4kb 123.4kb
  yellow open git_demo_enriched            DqgGv0uoQfCP-zWjGLnThg 1 1  754  0   9.6mb   9.6mb
  yellow open git_demo_raw                 N3Y7Q6QrRrOUhttRCO8Raw 1 1  754  3 956.7kb 956.7kb

But I can reproduce the error if I change the permission of the file to `600` (`docker` user cannot read the `setup.cfg` file)

$ docker logs --tail=10 docker-compose_mordred_1 sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 64, in main debug_mode = config_dict['general']['debug'] KeyError: 'debug' Traceback (most recent call last): File "/usr/local/bin/sirmordred", line 8, in sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/sirmordred/bin/sirmordred.py", line 64, in main debug_mode = config_dict['general']['debug'] KeyError: 'debug'

Could you try again but change the permission to `777` for `../default-grimoirelab-settings/setup-opensearch.cfg` and `../default-grimoirelab-settings/projects.json`?