search

jokob-sk / NetAlertX

🖧🔍 WIFI / LAN intruder detector. Scans for devices connected to your network and alerts you if new and unknown devices are found.
GNU General Public License v3.0
2.89k stars 170 forks source link

[env configuration] Can't install on Docker due to permissions issues #520

Closed s4en closed 9 months ago

s4en commented 9 months ago

Hi, I've tried more than one Compose File, and also new Volumes I can't install pi.alert No idea why, i use the latest image of pi.alert and external volumes

My Docker Compose:

version: "3"
services:

  pialert:
    container_name: PiAlert
    hostname: PiAlert
    privileged: true
    # use the below line if you want to test the latest dev image
    # image: "jokobsk/pi.alert_dev:latest" 
    image: jokobsk/pi.alert:latest
    environment:
      - TZ=Europe/Berlin
    restart: always
    volumes:
      - pialertdb:/home/pi/pialert/db
      - pialertconfig:/home/pi/pialert/config
    network_mode: host
volumes:
  pialertconfig:
    external: true
  pialertdb:
    external: true

Error:

---------------------------------------------------------
[INSTALL]                                    Run start.sh
---------------------------------------------------------
sed: can't read /home/pi/pialert/back/pialert.conf_bak: No such file or directory
[INSTALL] Run setup scripts
---------------------------------------------------------
[INSTALL]                             Run user-mapping.sh
---------------------------------------------------------
Nothing to do here.
---------------------------------------------------------
[INSTALL]                     Run install_dependencies.sh
---------------------------------------------------------
Reading package lists...
Building dependency tree...
Reading state information...
tini is already the newest version (0.19.0-1).
snmp is already the newest version (5.9.3+dfsg-2).
ca-certificates is already the newest version (20230311).
curl is already the newest version (7.88.1-10+deb12u4).
libwww-perl is already the newest version (6.68-1).
arp-scan is already the newest version (1.10.0-2).
perl is already the newest version (5.36.0-7+deb12u1).
apt-utils is already the newest version (2.6.1).
cron is already the newest version (3.0pl1-162).
sudo is already the newest version (1.9.13p3-1+deb12u1).
nginx-light is already the newest version (1.22.1-9).
php is already the newest version (2:8.2+93).
php-cgi is already the newest version (2:8.2+93).
php-fpm is already the newest version (2:8.2+93).
php-sqlite3 is already the newest version (2:8.2+93).
php-curl is already the newest version (2:8.2+93).
sqlite3 is already the newest version (3.40.1-2).
dnsutils is already the newest version (1:9.18.19-1~deb12u1).
net-tools is already the newest version (2.10-0.1).
python3 is already the newest version (3.11.2-1+b1).
iproute2 is already the newest version (6.1.0-3).
nmap is already the newest version (7.93+dfsg1-1).
python3-pip is already the newest version (23.0.1+dfsg-1).
zip is already the newest version (3.0-13).
systemctl is already the newest version (1.4.4181-1.1).
usbutils is already the newest version (1:014-1).
traceroute is already the newest version (1:2.1.2-1).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
sudo: unable to resolve host PiAlert: Temporary failure in name resolution
Reading package lists...
Building dependency tree...
Reading state information...
nginx is already the newest version (1.22.1-9).
nginx-core is already the newest version (1.22.1-9).
mtr is already the newest version (0.95-1).
php-fpm is already the newest version (2:8.2+93).
php8.2-fpm is already the newest version (8.2.7-1~deb12u1).
php-cli is already the newest version (2:8.2+93).
php8.2 is already the newest version (8.2.7-1~deb12u1).
php8.2-sqlite3 is already the newest version (8.2.7-1~deb12u1).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
sudo: unable to resolve host PiAlert: Temporary failure in name resolution
Reading package lists...
Building dependency tree...
Reading state information...
python3.11-venv is already the newest version (3.11.2-6).
python3.11-venv set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Requirement already satisfied: requests in /myenv/lib/python3.11/site-packages (2.31.0)
Requirement already satisfied: paho-mqtt in /myenv/lib/python3.11/site-packages (1.6.1)
Requirement already satisfied: scapy in /myenv/lib/python3.11/site-packages (2.5.0)
Requirement already satisfied: cron-converter in /myenv/lib/python3.11/site-packages (1.0.2)
Requirement already satisfied: pytz in /myenv/lib/python3.11/site-packages (2023.3.post1)
Requirement already satisfied: json2table in /myenv/lib/python3.11/site-packages (1.1.5)
Requirement already satisfied: dhcp-leases in /myenv/lib/python3.11/site-packages (0.1.6)
Requirement already satisfied: pyunifi in /myenv/lib/python3.11/site-packages (2.21)
Requirement already satisfied: speedtest-cli in /myenv/lib/python3.11/site-packages (2.1.3)
Requirement already satisfied: chardet in /myenv/lib/python3.11/site-packages (5.2.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /myenv/lib/python3.11/site-packages (from requests) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /myenv/lib/python3.11/site-packages (from requests) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /myenv/lib/python3.11/site-packages (from requests) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in /myenv/lib/python3.11/site-packages (from requests) (2023.11.17)
Requirement already satisfied: python-dateutil in /myenv/lib/python3.11/site-packages (from cron-converter) (2.8.2)
Requirement already satisfied: six>=1.5 in /myenv/lib/python3.11/site-packages (from python-dateutil->cron-converter) (1.16.0)
[INSTALL] Setup NGINX
Disabling default NGINX site, removing sym-link in /etc/nginx/sites-enabled
sudo: unable to resolve host PiAlert: Temporary failure in name resolution
sudo: unable to resolve host PiAlert: Temporary failure in name resolution
Setting webserver to user-supplied port (20211)
sudo: unable to resolve host PiAlert: Temporary failure in name resolution
[INSTALL] Run the hardware vendors update
The file ieee-oui.txt exists. Skipping update_vendors.sh...
[INSTALL] Fixing file permissions
jokob-sk commented 9 months ago

Hi there,

Can you try to remove:

hostname: PiAlert

From your docker-compose?

Is there anything else in the log file? If not it also could be permission related. Try specifying UID/GUID as per instructions:

https://github.com/jokob-sk/Pi.Alert/tree/main/dockerfiles#docker-environment-variables

J

s4en commented 9 months ago

hostname: PiAlert Removed the above. Same Issue I use a NFS Share for the external volume. Maybe this is a problem?

jokob-sk commented 9 months ago

hummm, I don't use an NFS share, but I still think this is permissions related. Could you please try to map the DB and config file directly? If this works then the issue is permissions related. You can also try to start the container without mapping the the DB/config for testing purposes, if it starts up that way.

coltondick commented 9 months ago

I'm also facing the same issue. I've attempted to start the container without mapping the volumes, but it doesn't create anything in the config or db directories. I am not using NFS. I'm trying to run this on x86_64.

version: "3.9"
services:  
  pialert:
    container_name: pialert
    image: "jokobsk/pi.alert:latest"      
    network_mode: "host"
    environment:
      TZ: $TZ
      HOST_USER_ID: $PUID
      HOST_USER_GID: $PGID
      PORT: 20211
    # volumes:
    #   - $DOCKERDIR/appdata/pialert/config:/home/pi/pialert/config
    #   - $DOCKERDIR/appdata/pialert/db:/home/pi/pialert/db    
    #   # (optional) useful for debugging if you have issues setting up the container
    #   - $DOCKERDIR/appdata/pialert/logs:/home/pi/pialert/front/log
jokob-sk commented 9 months ago

Hi @coltondick ,

I just deployed an instance with the simplified below config without issues:

version: "3.9"
services:
  pialert_test:
    container_name: pialert_test
    image: "jokobsk/pi.alert:latest"
    network_mode: "host"
    environment:
      PORT: 20213

can you provide me with the last few log lines?

coltondick commented 9 months ago

@jokob-sk

I just used the simplified config as a test and the issue still persists.

Installation Log - 12/31/2023

Time Action
02:15:53 PM [INSTALL] Run start.sh
02:15:53 PM [INSTALL] Run setup scripts
02:15:53 PM [INSTALL] Run user-mapping.sh
02:15:53 PM Nothing to do here.
02:15:53 PM [INSTALL] Run install_dependencies.sh
02:16:01 PM Reading package lists...
Building dependency tree...
Reading state information...
02:16:03 PM Packages tini, snmp, ca-certificates, curl,
libwww-perl, arp-scan, perl, apt-utils, cron,
sudo, nginx-light, php, php-cgi, php-fpm,
php-sqlite3, php-curl, sqlite3, dnsutils,
net-tools, python3, iproute2, nmap, python3-pip,
zip, systemctl, usbutils, traceroute are all
already at the newest version.
02:16:03 PM 0 upgraded, 0 newly installed, 0 to remove, and 2 not upgraded.
02:16:04 PM Packages nginx, nginx-core, mtr, php-fpm,
php8.2-fpm, php-cli, php8.2, php8.2-sqlite3 are
already at the newest version.
02:16:04 PM 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
02:16:05 PM Package python3.11-venv is already the newest version.
02:16:05 PM python3.11-venv set to manually installed.
02:16:05 PM 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
02:16:25 PM Requirements already satisfied in /myenv/lib/python3.11/site-packages: requests, paho-mqtt, scapy, cron-converter, pytz, json2table, dhcp-leases, pyunifi, speedtest-cli, chardet, charset-normalizer, idna, urllib3, certifi, python-dateutil.
02:16:26 PM [INSTALL] Setup NGINX
Disabling default NGINX site, removing sym-link in /etc/nginx/sites-enabled
Setting webserver to user-supplied port (20213)
02:16:26 PM [INSTALL] Run the hardware vendors update
The file ieee-oui.txt exists. Skipping update_vendors.sh...
02:16:26 PM [INSTALL] Fixing file permissions
jokob-sk commented 9 months ago

Once this build finishes (https://github.com/jokob-sk/Pi.Alert/actions/runs/7373252577), can you try to spin up the dev container?

version: "3.9"
services:
  pialert_test:
    container_name: pialert_test
    image: "jokobsk/pi.alert_dev:latest"
    network_mode: "host"
    environment:
      PORT: 20213

I've added a couple of logging lines to see where the script fails + re-ordered a sequence of calls that might cause the script to fail.

s4en commented 9 months ago

It must bei a permission Issue. I've change the permission of the pialert folder to everyone. After that pi.alert runs. Even on the nfs share.

coltondick commented 9 months ago

@jokob-sk I've tried the latest dev container and the issue persists. However, after leaving it for an hour it does come online. I think this could be related to my environment. I've got a Truenas Scale server running docker inside of a jail, which is essentially docker within docker.

I have deployed this successfully on a Raspberry Pi 4.

jokob-sk commented 9 months ago

@s4en - great - glad it's working :) @coltondick - ah, I haven't tested such a setup but glad to hear you found a workaround. With network scanning software, especially if you want to use NMAP or ARP-SCAN, I think you need to run it on the host network where the scans are performed.

lzsix commented 9 months ago

unfortunately i seem to be running in the same issue. Please note that i a fairly new to this topic so it´s not of the table that i misconfigured something.

No need to rush, thanks for the help in advance :)

docker-compose.yml:

version: "3" services: pialert: container_name: pialert image: "jokobsk/pi.alert:latest" network_mode: "host" restart: unless-stopped volumes:

Portainer Log: 2024-01-02T23:24:20.974980130Z ;pi@pialert: ~pi@pialert:~$ 2024-01-02T23:24:21.551171400Z --------------------------------------------------------- 2024-01-02T23:24:21.551244825Z [INSTALL] Run start.sh 2024-01-02T23:24:21.551263695Z --------------------------------------------------------- 2024-01-02T23:24:21.558781400Z [INSTALL] Run setup scripts 2024-01-02T23:24:21.564247473Z --------------------------------------------------------- 2024-01-02T23:24:21.564329675Z [INSTALL] Run user-mapping.sh 2024-01-02T23:24:21.564345656Z --------------------------------------------------------- 2024-01-02T23:24:21.568420253Z USER_ID : 1000 2024-01-02T23:24:21.568494345Z USER_GID : 1000 2024-01-02T23:24:21.568510085Z USER_HOME: 2024-01-02T23:24:21.568649231Z TZ : Europe/Berlin 2024-01-02T23:24:21.580386326Z chown: missing operand after '1000:1000 ' 2024-01-02T23:24:21.580647211Z Try 'chown --help' for more information.

PiAlert Log:

@raspberrypi:/docker/pialert/logs $ cat pialert.log 00:11:11 [MAIN] Setting up ... 00:11:11 [conf.tz] Setting up ... 00:11:11

00:11:11 The container restarted (started). If this is unexpected check https://bit.ly/PiAlertDebug for troubleshooting tips. 00:11:11

00:11:11 Permissions check (All should be True) 00:11:11 ------------------------------------------------ 00:11:11 /config/pialert.conf | READ | True 00:11:11 /config/pialert.conf | WRITE | True 00:11:11 /db/pialert.db | READ | True 00:11:11 /db/pialert.db | WRITE | True 00:11:11 ------------------------------------------------ 00:11:11 [Setup] Attempting to fix permissions. 00:11:11 [Setup] Attempting to fix permissions. 00:11:11 [Database] Opening DB 00:11:11 [upgradeDB] Adding dev_Network_Node_MAC_ADDR to the Devices table 00:11:11 [upgradeDB] Adding dev_Network_Node_port to the Devices table 00:11:11 [upgradeDB] Adding dev_Icon to the Devices table 00:11:11 [upgradeDB] Re-creating Settings table 00:11:11 [upgradeDB] Re-creating Pholus_Scan table 00:11:11 [upgradeDB] Re-creating Parameters table 00:11:11 [Config] reading config file 00:11:11 [Config] Plugins: Number of dynamically loaded plugins: 25 00:11:11 [Plugin utils] --------------------------------------------- 00:11:11 [Plugin utils] display_name: Internet-Check 00:11:11 [Plugin utils] description: A plugin to check your internet connectivity and IP. 00:11:11 [Plugin utils] --------------------------------------------- 00:11:11 [Plugin utils] display_name: Rogue DHCP 00:11:11 [Plugin utils] description: This plugin is to use NMAP to monitor for rogue DHCP servers. 00:11:11 [Plugin utils] --------------------------------------------- 00:11:11 [Plugin utils] display_name: NTFY publisher 00:11:11 [Plugin utils] description: A plugin to publish a notification via the NTFY gateway. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: SNMP discovery 00:11:12 [Plugin utils] description: This plugin is used to discover devices via the arp table(s) of a RFC1213 compliant router or switch. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: CSV backup 00:11:12 [Plugin utils] description: A plugin to auto-generate devices.csv backups. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: DDNS update 00:11:12 [Plugin utils] description: A plugin update the DDNS record. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Pushsafer publisher 00:11:12 [Plugin utils] description: A plugin to publish a notification via the Pushsafer gateway. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: New Devices 00:11:12 [Plugin utils] description: The template used for new devices. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: MQTT publisher 00:11:12 [Plugin utils] description: A plugin to publish a notification via the Apprise gateway. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Email publisher (SMTP) 00:11:12 [Plugin utils] description: A plugin to publish a notification via Email (SMTP) gateway. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Set password 00:11:12 [Plugin utils] description: A simple plugin to set the web ui password on app start. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Arp-Scan (Network scan) 00:11:12 [Plugin utils] description: This plugin is to execute an arp-scan on the local network 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Services & Ports (NMAP) 00:11:12 [Plugin utils] description: This plugin shows all services discovered by NMAP scans. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: DHCP Leases (Device import) 00:11:12 [Plugin utils] description: This plugin is to import devices from dhcp.leases files. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Maintenance 00:11:12 [Plugin utils] description: A plugin for maintenance tasks. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Internet speedtest 00:11:12 [Plugin utils] description: A plugin to perform a scheduled internet speedtest. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Un-Discoverable Devices 00:11:12 [Plugin utils] description: This plugin is to import undiscoverable devices from a file. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Website monitor 00:11:12 [Plugin utils] description: This plugin is to monitor status changes of services or websites. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: PiHole (Device sync) 00:11:12 [Plugin utils] description: This plugin syncs devices from the PiHole database 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Vendor update 00:11:12 [Plugin utils] description: A plugin to schedule vendor database updates for mac based vendor resolution. 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Pholus-Scan (Name discovery) 00:11:12 [Plugin utils] description: This plugin is to execute a Pholus-scan (name discovery) on the local network 00:11:12 [Plugin utils] --------------------------------------------- 00:11:12 [Plugin utils] display_name: Apprise publisher 00:11:12 [Plugin utils] description: A plugin to publish a notification via the Apprise gateway. 00:11:13 [Plugin utils] --------------------------------------------- 00:11:13 [Plugin utils] display_name: DB cleanup 00:11:13 [Plugin utils] description: A plugin to schedule database cleanup & upkeep tasks. 00:11:13 [Plugin utils] --------------------------------------------- 00:11:13 [Plugin utils] display_name: UniFi import 00:11:13 [Plugin utils] description: This plugin is used to import devices from an UNIFI controller. 00:11:13 [Plugin utils] --------------------------------------------- 00:11:13 [Plugin utils] display_name: Webhook publisher 00:11:13 [Plugin utils] description: A plugin to publish a notification via Webhooks. 00:11:13 [API] Updating table_settings.json file in /front/api 00:11:13 [Config] Imported new config 00:11:13 [API] Updating table_devices.json file in /front/api 00:11:13 [API] Updating table_events_pending_alert.json file in /front/api 00:11:13 [API] Updating table_plugins_events.json file in /front/api 00:11:13 [API] Updating table_plugins_history.json file in /front/api 00:11:13 [API] Updating table_plugins_objects.json file in /front/api 00:11:13 [API] Updating table_plugins_language_strings.json file in /front/api 00:11:13 [Database] - SQL ERROR: no such table: Notifications 00:11:13 [API] Updating table_notifications.json file in /front/api 00:11:13 [API] Updating table_custom_endpoint.json file in /front/api 00:11:13 [Notification] Check if something to report 00:11:13 [Notification] Open text Template 00:11:13 [Notification] Open html Template 00:11:13 [Notification] Using template/back/report_template.html 00:11:13 [Notification] included sections: ['internet', 'new_devices', 'downdevices', 'events'] 00:11:13 [Notification] New Devices sections done. 00:11:13 [Notification] Down Devices sections done. 00:11:13 [Notification] Events sections done. 00:11:13 [Notification] Udating API files 00:11:13 [Send API] Updating notification files in /home/pi/pialert/front/api/ 00:11:13 [Notification] No changes to report 00:11:13 [MAIN] Process: Wait 00:11:18 [API] Updating table_notifications.json file in /front/api 00:11:18 [MAIN] waiting to start next loop 00:11:23 [MAIN] waiting to start next loop 00:11:28 [MAIN] waiting to start next loop 00:11:34 [MAIN] waiting to start next loop 00:11:39 [MAIN] waiting to start next loop 00:11:44 [MAIN] waiting to start next loop 00:11:49 [MAIN] waiting to start next loop 00:11:54 [MAIN] waiting to start next loop 00:11:59 [MAIN] waiting to start next loop 00:12:04 [MAIN] waiting to start next loop 00:12:09 [MAIN] waiting to start next loop 00:12:14 [Notification] Check if something to report 00:12:14 [Notification] Open text Template 00:12:14 [Notification] Open html Template 00:12:14 [Notification] Using template/back/report_template.html 00:12:15 [Notification] included sections: ['internet', 'new_devices', 'downdevices', 'events'] 00:12:15 [Notification] New Devices sections done. 00:12:15 [Notification] Down Devices sections done. 00:12:15 [Notification] Events sections done. 00:12:15 [Notification] Udating API files 00:12:15 [Send API] Updating notification files in /home/pi/pialert/front/api/ 00:12:15 [Notification] No changes to report 00:12:15 [MAIN] Process: Wait 00:12:20 [MAIN] waiting to start next loop 00:12:25 [MAIN] waiting to start next loop 00:12:30 [MAIN] waiting to start next loop 00:12:35 [MAIN] waiting to start next loop 00:12:40 [MAIN] waiting to start next loop 00:12:45 [MAIN] waiting to start next loop 00:12:50 [MAIN] waiting to start next loop 00:12:55 [MAIN] waiting to start next loop 00:13:00 [MAIN] waiting to start next loop 00:13:05 [MAIN] waiting to start next loop 00:13:11 [MAIN] waiting to start next loop

jokob-sk commented 9 months ago

Hi @lzsix ,

It seems like your issue is different so I opened a new thread. Let's continue the discussion over there: https://github.com/jokob-sk/Pi.Alert/issues/525

Thanks, j