tarball-based installation should not depend on UID inside of tarball, prevents installation if UID with which tarball's contents were created don't match installing user's

[gustavoberman]

Describe the bug install.py fails with

KeyError: 'getpwuid(): uid not found: 1001'

I only have one sudo user "sim" and its UID is 1000

To Reproduce Steps to reproduce the behavior:

1. download all *py and tar.gz from release 24.06.0
2. chmod +x install.py
3. sudo ./install.py
4. Follow options and when asked from Store PCAP, log, indexes fails with:

Store PCAP, log and index files in /opt/malcom? (Y / n): 
Creating /opt/malcom/opensearch failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/opensearch-backup failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/pcap/arkime-live failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/pcap/processed failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/pcap/upload/tmp/spool failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/pcap/upload/variants failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/suricata-logs/live failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/zeek-logs/current failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/zeek-logs/live failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/zeek-logs/upload failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/zeek-logs/extract_files/preserved failed: 'getpwuid(): uid not found: 1001'
Creating /opt/malcom/zeek-logs/extract_files/quarantine failed: 'getpwuid(): uid not found: 1001'

5. The installation script keep asking more questions and at the end it gives:

KeyError: 'getpwuid(): uid not found: 1001'

Expected behavior No error

Screenshots and/or Logs

$ id
uid=1000(sim) gid=1000(sim) groups=1000(sim),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),122(lpadmin),135(lxd),136(sambashare),999(docker)

$ uname -a
Linux sim 6.2.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

Malcolm Version:

• Version 24.06.0

How are you running Malcolm?

• [X ] other (please describe)

Trying to install it in a ubuntu 22.04.

[mmguero]

Hmmm, I'm not sure where the 1001 came from as (like you said) you only have the one user ID and I can't see a reference to 1001 anywhere in the Malcolm source code.

When you saw this (or the equivalent non-dialog version of it) did you answer "Y"?

I'll spin up an Ubuntu 22.04 vm and see if i get the same thing as you.

[gustavoberman]

Hmmm, I'm not sure where the 1001 came from as (like you said) you only have the one user ID and I can't see a reference to 1001 anywhere in the Malcolm source code.

When you saw this (or the equivalent non-dialog version of it) did you answer "Y"?

Yes, exactly.

I think the problem is the extracted tar.gz:

At this point in the install, it already extracts with those permisions:

$ sudo ./install.py 
Installing required packages: ['apache2-utils', 'make', 'openssl', 'python3-dialog', 'python3-dotenv', 'python3-requests', 'python3-yaml', 'xz-utils']

Add a non-root user to the "docker" group? (Y / N): n

Extract Malcolm runtime files from /home/sim/Downloads/Malcomv24.06.0/malcolm_20240626_134945_75fe54ba.tar.gz? (Y / n): 

Enter installation path for Malcolm [/home/sim/Downloads/Malcomv24.06.0/malcolm] (/home/sim/Downloads/Malcomv24.06.0/malcolm): /opt/malcom
Malcolm runtime files extracted to /opt/malcom

$ ll /opt/malcom/
total 108
drwxr-xr-x 19 root root  4096 jul 22 14:46 ./
drwxr-xr-x  5 root root  4096 jul 22 14:46 ../
drwxr-xr-x  3 1001 1001  4096 jun 26 17:47 arkime/
drwxr-xr-x  2 1001 1001  4096 jun 26 17:47 config/
-rw-r--r--  1 1001 1001 22493 jun 26 17:47 docker-compose.yml
drwxr-xr-x  3 1001 1001  4096 jun 26 17:47 filebeat/
drwxr-xr-x  2 1001 1001  4096 jun 26 17:47 htadmin/
drwxr-xr-x  2 1001 1001  4096 jun 26 17:47 kubernetes/
drwxr-xr-x  4 1001 1001  4096 jun 26 17:47 logstash/
drwxr-xr-x  7 1001 1001  4096 jun 26 17:47 netbox/
-rw-r--r--  1 1001 1001     2 jun 26 17:47 net-map.json
drwxr-xr-x  4 1001 1001  4096 jun 26 17:47 nginx/
drwxr-xr-x  3 1001 1001  4096 jun 26 17:47 opensearch/
drwxr-xr-x  2 1001 1001  4096 jun 26 17:47 opensearch-backup/
-rw-------  1 1001 1001     0 jun 26 17:47 .opensearch.primary.curlrc
-rw-------  1 1001 1001     0 jun 26 17:47 .opensearch.secondary.curlrc
drwxr-xr-x  5 1001 1001  4096 jun 26 17:47 pcap/
-rw-r--r--  1 1001 1001  3657 jun 26 17:47 README.md
drwxr-xr-x  2 1001 1001  4096 jun 26 17:47 scripts/
drwxr-xr-x  4 1001 1001  4096 jun 26 17:47 suricata/
drwxr-xr-x  3 1001 1001  4096 jun 26 17:47 suricata-logs/
drwxr-xr-x  3 1001 1001  4096 jun 26 17:47 yara/
drwxr-xr-x  4 1001 1001  4096 jun 26 17:47 zeek/
drwxr-xr-x  7 1001 1001  4096 jun 26 17:47 zeek-logs/

[mmguero]

I see what you're saying. the sudo install.py should be chowning the directory and its contents and extraction but apparently is not. Thanks for bringing this to my attention, I'll get it fixed. In the meantime if you manually extract that tarball, then chown it and its contents to your UID it should work.