Could not deploy local albs while following the build-system wiki

[linlinger]

I tried to deploy ALBS based on the albs-deploy guide and the guide located at build-system wiki . At first I couldn't access the frontend .I submitted a issue before and the detail is here . When I solved this . I found some of the containers keeps restarting all the time.

Here is some logs I get from containers . The logs are located at pastebin for now. If you need to let me post it here. Please let me know. Logs from albs_alts-celery_1 logs from albs_alts-scheduler_1 logs from albs_build_node_1 logs from albs_gitea_listener_1 logs from albs_sign_node_1

I forgot to add my personal vars.yml `---

frontend_baseurl: http://192.168.1.15:8080 github_client: <It's a secret> github_client_secret: <It's a secret>

immudb_username: immudb immudb_password: immudb immudb_database: defaultdb immudb_address: 192.168.1.15:3322 immudb_public_key_file: /home/albs/.ssh/id_rsa.pub`

[zklevsha]

@linlinger hi I`ve managed to fix containers restart issue in this branch

Could you please perform clean depoy using this branch and test if everething is OK now? I`ll merge this branch to master after your testing

Thanks)

[linlinger]

@linlinger hi I`ve managed to fix containers restart issue in this branch

Could you please perform clean depoy using this branch and test if everething is OK now? I`ll merge this branch to master after your testing

Thanks)

Follow the instruction at the branch you refer ? I will try it now

[zklevsha]

@linlinger you can use same instructions but checkout albs-deploy not from master but from this branch

[linlinger]

@linlinger you can use same instructions but checkout albs-deploy not from master but from this branch

Got it. I just finished system installation, setting up dependencies now

[linlinger]

@linlinger you can use same instructions but checkout albs-deploy not from master but from this branch

[linlinger@localhost albs-deploy]$ git checkout origin/#132 Note: switching to 'origin/#132'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at be64592 - Fix permissions for .gnupg directory - Changed port for MQTT Is it okay ?

[zklevsha]

Yep, you can now start depoy

[linlinger]

is immudb_public_key_file required in vars.yml? if yes. May I generate one using ssh-keygen command ?

[linlinger]

Sometime it requires me to enter password when I run ansible playbook

[linlinger@localhost albs-deploy]$ ansible-playbook -i inventories/one_vm -vv -u albs -e "@vars.yml" playbooks/albs_on_one_vm.yml ansible-playbook [core 2.14.2] config file = /home/linlinger/albs-deploy/ansible.cfg configured module search path = ['/home/linlinger/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.11/site-packages/ansible ansible collection location = /home/linlinger/.ansible/collections:/usr/share/ansible/collections executable location = /usr/bin/ansible-playbook python version = 3.11.2 (main, Oct 5 2023, 18:41:49) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3.11) jinja version = 3.1.2 libyaml = True Using /home/linlinger/albs-deploy/ansible.cfg as config file statically imported: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml redirecting (type: action) ansible.builtin.synchronize to ansible.posix.synchronize redirecting (type: action) ansible.builtin.synchronize to ansible.posix.synchronize statically imported: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/gpg.yml statically imported: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/configs.yml statically imported: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/services.yml statically imported: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/misc.yml redirecting (type: callback) ansible.builtin.debug to ansible.posix.debug redirecting (type: callback) ansible.builtin.debug to ansible.posix.debug redirecting (type: callback) ansible.builtin.profile_tasks to ansible.posix.profile_tasks Skipping callback 'default', as we already have a stdout callback. Skipping callback 'minimal', as we already have a stdout callback. Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: albs_on_one_vm.yml ** 1 plays in playbooks/albs_on_one_vm.yml

PLAY [Deploy the ALBS] ****

TASK [Gathering Facts] **** task path: /home/linlinger/albs-deploy/playbooks/albs_on_one_vm.yml:2 Wednesday 15 November 2023 07:27:57 -0500 (0:00:00.032) 0:00:00.032 **** ok: [albs_on_one_vm]

TASK [dev_deploy : include_tasks] ***** task path: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/main.yml:7 Wednesday 15 November 2023 07:27:58 -0500 (0:00:00.723) 0:00:00.755 **** included: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/ansible_preparation.yml for albs_on_one_vm

TASK [dev_deploy : Install DNF packages for right works of Ansible] *** task path: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/ansible_preparation.yml:2 Wednesday 15 November 2023 07:27:58 -0500 (0:00:00.019) 0:00:00.775 **** fatal: [albs_on_one_vm]: FAILED! => { "changed": false, "rc": 1 }

MSG:

MODULE FAILURE See stdout/stderr for the exact error

MODULE_STDERR:

sudo: a password is required

NO MORE HOSTS LEFT ****

PLAY RECAP **** albs_on_one_vm : ok=2 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

Wednesday 15 November 2023 07:27:58 -0500 (0:00:00.159) 0:00:00.934 ****

Gathering Facts ---------------------------------------------------------------------------------------- 0.72s /home/linlinger/albs-deploy/playbooks/albs_on_one_vm.yml:2 --------------------------------------------------- dev_deploy : Install DNF packages for right works of Ansible ------------------------------------------- 0.16s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/ansible_preparation.yml:2 --------------------------------- dev_deploy : include_tasks ----------------------------------------------------------------------------- 0.02s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/main.yml:7 ------------------------------------------------ [linlinger@localhost albs-deploy]$ sudo tail /etc/sudoers [sudo] password for linlinger:

cdrom as root

%users ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom

Allows members of the users group to shutdown this system

%users localhost=/sbin/shutdown -h now

Read drop-in files from /etc/sudoers.d (the # here does not mean a comment)

includedir /etc/sudoers.d

albs ALL=(ALL) NOPASSWD: ALL albs ALL=(ALL) NOPASSWD: ALL [linlinger@localhost albs-deploy]$

[zklevsha]

If you deploying with immudb (you`re planning to notarize builds) then yes

Yes, you can use ssh-keygen for key generation

[zklevsha]

Regarding password Do you able to run passwordless sudo as albs manually?

[linlinger]

yes , when I switched to albs I can always run sudo xxx without password like this

[linlinger@localhost albs-deploy]$ su - albs Password: Last login: Wed Nov 15 07:37:02 EST 2023 on pts/0 [albs@localhost ~]$ sudo ps PID TTY TIME CMD 30173 pts/0 00:00:00 su 30206 pts/0 00:00:00 sudo 30208 pts/0 00:00:00 ps [albs@localhost ~]$

[linlinger]

It looks like the user run playbook should also has access to docker

TASK [dev_deploy : Create and start services] ***** task path: /home/linlinger/albs-deploy/roles/dev_deploy/tasks/services.yml:43 Wednesday 15 November 2023 07:38:55 -0500 (0:00:00.021) 0:00:38.147 **** fatal: [albs_on_one_vm]: FAILED! => { "changed": true, "cmd": "/usr/bin/docker-compose -p albs --compatibility up -d --build --force-recreate", "delta": "0:00:00.030041", "end": "2023-11-15 07:38:55.872795", "rc": 1, "start": "2023-11-15 07:38:55.842754" }

STDERR:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.project%3Dalbs%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied

MSG:

non-zero return code

NO MORE HOSTS LEFT ****

PLAY RECAP **** albs_on_one_vm : ok=25 changed=10 unreachable=0 failed=1 skipped=8 rescued=0 ignored=0

Wednesday 15 November 2023 07:38:55 -0500 (0:00:00.168) 0:00:38.316 ****

dev_deploy : Clone ALBS sources ----------------------------------------------------------------------- 22.03s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:86 --------------------------------------------- dev_deploy : Build NodeJS modules for albs-frontend ---------------------------------------------------- 6.29s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:112 -------------------------------------------- dev_deploy : Generate a service's config --------------------------------------------------------------- 2.07s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/configs.yml:3 --------------------------------------------- dev_deploy : Create services' folders ------------------------------------------------------------------ 1.51s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:121 -------------------------------------------- dev_deploy : Check and install necessary pip packages -------------------------------------------------- 0.98s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:8 ---------------------------------------------- dev_deploy : Install DNF packages for right works of Ansible ------------------------------------------- 0.89s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/ansible_preparation.yml:2 --------------------------------- dev_deploy : Install packages -------------------------------------------------------------------------- 0.78s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:2 ---------------------------------------------- Gathering Facts ---------------------------------------------------------------------------------------- 0.72s /home/linlinger/albs-deploy/playbooks/albs_on_one_vm.yml:2 --------------------------------------------------- dev_deploy : Switch to Node.js version ----------------------------------------------------------------- 0.54s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:43 --------------------------------------------- dev_deploy : Generate a GPG scenario ------------------------------------------------------------------- 0.34s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/gpg.yml:7 ------------------------------------------------- dev_deploy : Check current Node.js version ------------------------------------------------------------- 0.26s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:23 --------------------------------------------- dev_deploy : Check Node.js version --------------------------------------------------------------------- 0.26s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:53 --------------------------------------------- dev_deploy : Install NVM ------------------------------------------------------------------------------- 0.22s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:17 --------------------------------------------- dev_deploy : Generate albs_jwt_token ------------------------------------------------------------------- 0.21s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:64 --------------------------------------------- dev_deploy : Generate alts_jwt_token ------------------------------------------------------------------- 0.20s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/common.yml:75 --------------------------------------------- dev_deploy : Create and start services ----------------------------------------------------------------- 0.17s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/services.yml:43 ------------------------------------------- dev_deploy : Get already existing GPG key's fingerprint ------------------------------------------------ 0.14s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/gpg.yml:3 ------------------------------------------------- dev_deploy : Allow access to .gnupg directory to sign_node container ----------------------------------- 0.14s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/gpg.yml:24 ------------------------------------------------ dev_deploy : Get already existing GPG key's fingerprint ------------------------------------------------ 0.14s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/gpg.yml:20 ------------------------------------------------ dev_deploy : Check if docker-compose command exists ---------------------------------------------------- 0.14s /home/linlinger/albs-deploy/roles/dev_deploy/tasks/services.yml:3 -------------------------------------------- [linlinger@localhost albs-deploy]$ su - albs Password: Last login: Wed Nov 15 07:37:10 EST 2023 on pts/0 [albs@localhost ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [albs@localhost ~]$ exit logout [linlinger@localhost albs-deploy]$ whoami linlinger [linlinger@localhost albs-deploy]$ docker ps permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json": dial unix /var/run/docker.sock: connect: permission denied [linlinger@localhost albs-deploy]$

[linlinger]

It seems that I got something wrong, I should create albs user first ,set sudo rules and log in as albs . And do the rest of the deployment

[zklevsha]

User albs must be in wheel and docker groups I`ve created playbook prepare_alma9_one_vm.yml for AlmaLinux9.2

It installs required packages, creates albs user and generates RSA keys for immudb If you`re deloying albs to RHEL based host, you can try to use it

[linlinger]

However I installed AlmaLinux 8.8. I'm afraid that I use prepare_alma9_one_vm.yml might mess up something . Okay, I will install another AlmaLinux 9.3 vm and try the yml file.

BTW , It stucks at "dev_deploy : Create and start services" for now . I may not be able to tell you whether the fixes works or not very soon

[zklevsha]

Ive noticed that currently one_vm_deploy does not support immudb notarization 23.11.15 14:51:05 DEBUG [MainThread]: cleaning up the /srv/alternatives/sign_node working directory Traceback (most recent call last): File "/sign-node/almalinux_sign_node.py", line 67, in <module> sys.exit(main()) File "/sign-node/almalinux_sign_node.py", line 60, in main signer = Signer(config, password_db, gpg) File "/sign-node/sign_node/signer.py", line 81, in __init__ self.__notary = Codenotary( File "/sign-node/sign_node/utils/codenotary.py", line 15, in __init__ self.wrapper = ImmudbWrapper( File "/sign-node/env/lib64/python3.9/site-packages/immudb_wrapper.py", line 62, in __init__ super().__init__( File "/sign-node/env/lib64/python3.9/site-packages/immudb/client.py", line 83, in __init__ self.loadKey(publicKeyFile) File "/sign-node/env/lib64/python3.9/site-packages/immudb/client.py", line 91, in loadKey with open(kfile) as f: FileNotFoundError: [Errno 2] No such file or directory: '/var/immudb_keys/immudb.pub'

For now, you can set all immudb variables as '' I`ll will look into this problem

[zklevsha]

I`ve been told that you dont need immunify when deploying all_in_one_vm So you can leave all immudb variables as ''

[zklevsha]

@linlinger Hi, did you able to run deploy-tool?

[linlinger]

And the issue seems to be happening again, I tried deploy ALBS on Alma Linux 9 again. It keeps telling me that unable to prepare context: path "/home/albs/albs/alma-tests-cacher" not found . I will provide the logs and my config below My config

---

frontend_baseurl: http://192.168.1.11:8080
github_client: it's a secret
github_client_secret: top secret

immudb_username: immudb
immudb_password: immudb
immudb_database: defaultdb
immudb_address: 192.168.1.11:3322
immudb_public_key_file: /home/albs/.ssh/id_rsa.pub`
```[albs@qlbs albs-deploy]$ cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.3 (Shamrock Pampas Cat)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.3"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.3 (Shamrock Pampas Cat)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.3"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

Finally the whole log file 
[ansible-deploy-error.txt](https://github.com/AlmaLinux/build-system/files/13976334/ansible-deploy-error.txt)

My system info 

[albs@qlbs albs-deploy]$ cat /etc/os-release
NAME="AlmaLinux"
VERSION="9.3 (Shamrock Pampas Cat)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.3"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.3 (Shamrock Pampas Cat)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.3"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

[metalefty]

I have submitted this but there is still some other issue. https://github.com/AlmaLinux/albs-deploy/pull/10

[zklevsha]

@metalefty @linlinger issue with alma-tests-cacher was fixed today by merging https://github.com/AlmaLinux/albs-deploy/pull/8

[linlinger]

I will try it with the same config I used before