jokob-sk / NetAlertX

🖧🔍 WIFI / LAN intruder detector. Scans for devices connected to your network and alerts you if new and unknown devices are found.
GNU General Public License v3.0
2.84k stars 163 forks source link

"The original database could not be saved" when running DB Backup #197

Closed delipastrami closed 1 year ago

delipastrami commented 1 year ago

Describe the issue Hello, whenever I navigate to Maintenance > Backup / Restore > DB Backup, it does not complete successfully. Export from CSV works alright. Might there be something I am doing incorrectly?

Paste last few lines from pialert.log


[2023-03-14 01:13:11] Scan Devices:
    arp-scan start
  Processing scan results
    Devices Detected.......: 19
        arp-scan detected..: 17
        Pi-hole detected...: +0
        New Devices........: 0
    Devices in this cycle..: 18
        Down Alerts........: 0
        New Down Alerts....: 0
        New Connections....: 0
        Disconnections.....: 0
        IP Changes.........: 0
  Updating DB Info
    Sessions Events (connect / discconnect)
    Creating new devices
    Updating Devices Info
        Trying to resolve devices without name
          Pholus entries from prev scans: 52
        Names Found (DiG/Pholus): 0 (0/0)
        Names Not Found         : 2
    Voiding false (ghost) disconnections
    Pairing session events (connection / disconnection) 
    Creating sessions snapshot
    Inserting scan results into Online_History
    Skipping repeated notifications
  Check if something to report
    No changes to report
[2023-03-14 01:13:23] Notifications: 0
[2023-03-14 01:13:23] Last action: network_scan
[2023-03-14 01:13:23] Process: Wait
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
     [API] Updating files in /front/api
[2023-03-14 01:14:13] Process: Start
[2023-03-14 01:14:13] Process: Wait
     [API] Updating files in /front/api
     [API] Updating files in /front/api

Paste your pialert.conf (remove personal info)


#-----------------AUTOGENERATED FILE-----------------#
#                                                    #
#         Generated:  2023-03-14_00-52-47            #
#                                                    #
#   Config file for the LAN intruder detection app:  #
#      https://github.com/jokob-sk/Pi.Alert          #
#                                                    #
#-----------------AUTOGENERATED FILE-----------------#

# General
#---------------------------
ENABLE_ARPSCAN=True
SCAN_SUBNETS=['192.168.86.0/24 --interface=ovs_eth0']
LOG_LEVEL='verbose'
TIMEZONE='America/New_York'
PIALERT_WEB_PROTECTION=False
PIALERT_WEB_PASSWORD=''
INCLUDED_SECTIONS=['internet','new_devices','down_devices','events']
SCAN_CYCLE_MINUTES=5
DAYS_TO_KEEP_EVENTS=90
REPORT_DASHBOARD_URL='http://192.168.86.91:20211'
DIG_GET_IP_ARG='-4 myip.opendns.com @resolver1.opendns.com'
UI_LANG='English'

# Email
#---------------------------
REPORT_MAIL=False
SMTP_SERVER='smtp.gmail.com'
SMTP_PORT=587
REPORT_TO='user@gmail.com'
REPORT_FROM='Pi.Alert <user@gmail.com>'
SMTP_SKIP_LOGIN=False
SMTP_USER='user@gmail.com'
SMTP_PASS='password'
SMTP_SKIP_TLS=False
SMTP_FORCE_SSL=False

# Webhooks
#---------------------------
REPORT_WEBHOOK=True
WEBHOOK_URL=''
WEBHOOK_PAYLOAD='html'
WEBHOOK_REQUEST_METHOD='GET'

# Apprise
#---------------------------
REPORT_APPRISE=False
APPRISE_HOST='http://localhost:8000/notify'
APPRISE_URL='mailto://smtp-relay.sendinblue.com:587?from=user@gmail.com&name=apprise&user=user@gmail.com&pass=password&to=user@gmail.com'
APPRISE_PAYLOAD='html'

# NTFY
#---------------------------
REPORT_NTFY=False
NTFY_HOST='https://ntfy.sh'
NTFY_TOPIC='replace_my_secure_topicname_91h889f28'
NTFY_USER='user'
NTFY_PASSWORD='passw0rd'

# PUSHSAFER
#---------------------------
REPORT_PUSHSAFER=False
PUSHSAFER_TOKEN='ApiKey'

# MQTT
#---------------------------
REPORT_MQTT=False
MQTT_BROKER='192.168.1.2'
MQTT_PORT=1883
MQTT_USER='mqtt'
MQTT_PASSWORD='passw0rd'
MQTT_QOS=0
MQTT_DELAY_SEC=2

# DynDNS
#---------------------------
DDNS_ACTIVE=False
DDNS_DOMAIN='your_domain.freeddns.org'
DDNS_USER='dynu_user'
DDNS_PASSWORD='A0000000B0000000C0000000D0000000'
DDNS_UPDATE_URL='https://api.dynu.com/nic/update?'

# PiHole
#---------------------------
PIHOLE_ACTIVE=False
DHCP_ACTIVE=False

# Pholus
#---------------------------
PHOLUS_ACTIVE=False
PHOLUS_TIMEOUT=120
PHOLUS_FORCE=False
PHOLUS_RUN='once'
PHOLUS_RUN_TIMEOUT=600
PHOLUS_RUN_SCHD='0 4 * * *'
PHOLUS_DAYS_DATA=7

# Nmap
#---------------------------
NMAP_ACTIVE=True
NMAP_TIMEOUT=150
NMAP_RUN='none'
NMAP_RUN_SCHD='0 2 * * *'
NMAP_ARGS='-p -10000'

# API
#---------------------------
API_CUSTOM_SQL='SELECT * FROM Devices WHERE dev_PresentLastScan = 0'

# NMAPSRV
#---------------------------
NMAPSRV_RUN='disabled'
NMAPSRV_CMD='SELECT  dv.dev_Name as Object_PrimaryID, cast({s-quote}http://{s-quote} || dv.dev_LastIP as VARCHAR(100)) || {s-quote}:{s-quote} || cast( SUBSTR(ns.Port ,0, INSTR(ns.Port , {s-quote}/{s-quote})) as VARCHAR(100)) as Object_SecondaryID,  datetime() as DateTime,  ns.Service as Watched_Value1,        ns.State as Watched_Value2,        {s-quote}null{s-quote} as Watched_Value3,        {s-quote}null{s-quote} as Watched_Value4,        ns.Extra as Extra, dv.dev_MAC as ForeignKey        FROM (SELECT * FROM Nmap_Scan) ns LEFT JOIN (SELECT dev_Name, dev_MAC, dev_LastIP FROM Devices) dv   ON ns.MAC = dv.dev_MAC'
NMAPSRV_RUN_SCHD='0 2 * * *'
NMAPSRV_WATCH=['Watched_Value1']
NMAPSRV_REPORT_ON=['new','watched-changed']

# WEBMON
#---------------------------
WEBMON_RUN='disabled'
WEBMON_CMD='python3 /home/pi/pialert/front/plugins/website_monitor/script.py urls={urls}'
WEBMON_RUN_SCHD='0 2 * * *'
WEBMON_API_SQL='SELECT * FROM plugin_website_monitor'
WEBMON_RUN_TIMEOUT=5
WEBMON_WATCH=['Watched_Value1']
WEBMON_REPORT_ON=['new','watched-changed']
WEBMON_urls_to_check=['https://google.com','https://duck.com']
WEBMON_SQL_internet_ip='SELECT dev_LastIP FROM Devices WHERE dev_MAC = {s-quote}Internet{s-quote}'

#-------------------IMPORTANT INFO-------------------#
#   This file is ingested by a python script, so if  #
#        modified it needs to use python syntax      #
#-------------------IMPORTANT INFO-------------------#

Paste your docker-compose.yml and .env (remove personal info)

docker-compose.yml


version: "3"
services:
    pialert:
        container_name: pialert
        image: "jokobsk/pi.alert:latest"
        network_mode: "host"
        restart: unless-stopped
        volumes:
            - /volume2/docker/pialert/config:/home/pi/pialert/config
            - /volume2/docker/pialert/db:/home/pi/pialert/db
            - /volume2/docker/pialert/logs:/home/pi/pialert/front/log
        environment:
            - TZ=America/New_York
            - HOST_USER_ID=1026
            - HOST_USER_GID=100
            - PORT=20211

.env


paste here

Screenshots https://i.imgur.com/ON8OmoT.png

delipastrami commented 1 year ago

Apologies, I thought there was some kind of error. Additionally, I was looking in completely the wrong location for the backup files:

root@ragnarok:/volume2/docker# ls -l pialert/db/
total 4612
-rwxrwxrwx 1 33 33  479232 Mar 14 02:35 pialert.db
-rwxrwxrwx 1 33 33   32768 Mar 14 02:57 pialert.db-shm
-rwxrwxrwx 1 33 33 4210672 Mar 14 02:57 pialert.db-wal
root@ragnarok:/volume2/docker#
root@ragnarok:/volume2/docker#

versus

root@ragnarok:/volume2/docker# ls -l pialert/config/
total 36
-rwxrwxrwx 1 estevan users 4420 Mar 14 01:19 pialert.conf
-rw-rw-rw- 1 estevan users 3445 Mar 12 23:51 pialert.conf_2023-03-12_23-51-29.backup
-rw-rw-rw- 1 estevan users 4372 Mar 13 00:07 pialert.conf_2023-03-13_00-07-58.backup
-rw-rw-rw- 1 estevan users 4372 Mar 14 00:52 pialert.conf_2023-03-14_00-52-47.backup
-rwxrwxrwx 1 estevan users 4420 Mar 14 00:52 pialert.confe
tarkilhk commented 1 year ago

Hi guys,

I have the exact same behaviour as described by @delipastrami above. And I confirm that my local/path/pialert/db folder doesn't contain any backup generated ("The database backups are located in the database directory as a zip-archive, named with the creation date. There is no maximum number of backups."), and that the UI shows 0 backup ("0 backups were found / total disk usage: 0,00 MB").

My log is empty of any error (same as what @delipastrami posted above).

Not sure what I'm doing wrong if anything please ?

Thanks in advance for the help.

jokob-sk commented 1 year ago

Hi,

Backups are not created automatically. You have to manually click the DB backup button:

image

I recommend the CSV Export function for backup purposes.

j

tarkilhk commented 1 year ago

Hello,

Thank you very much for taking the time to answer. Sorry if I wasn't clear enough, I did mean that this happens after I click on the "DB Backup" button.

Meaning I click, then I get the message "The original database could not be saved" (same as in the screenshot from OP), and no backup gets generated in the /db folder.

Thanks vm in advance for the help if you hav any idea what else I could check (nothing appears in the logs)

delipastrami commented 1 year ago

For what it’s worth, I could never figure out how to get that db backup button to work. Only the csv export.

leiweibau commented 1 year ago

In most cases, this error is a permission problem. However, since integrity problems (in the backup itself) could also occur in this context, this function has been changed. However, even with the changed version, the correct permission in DB folder is important.