Memory Usage Above 4GB

[fire1ce]

Describe the issue

When submitting an issue ❗enable debug❗ and have a look at the docs

Massive Memory Usage - above 4GB of ram.

Paste your pialert.conf (remove personal info)

#-----------------AUTOGENERATED FILE-----------------#
#                                                    #
#         Generated:  2023-07-20_10-38-26            #
#                                                    #
#   Config file for the LAN intruder detection app:  #
#      https://github.com/jokob-sk/Pi.Alert          #
#                                                    #
#-----------------AUTOGENERATED FILE-----------------#

# General
#---------------------------
ENABLE_ARPSCAN=True
SCAN_SUBNETS=['10.10.10.0/24 --interface=eth0']
LOG_LEVEL='debug'
TIMEZONE='something/something'
ENABLE_PLUGINS=False
PIALERT_WEB_PROTECTION=True
PIALERT_WEB_PASSWORD='something'
INCLUDED_SECTIONS=['new_devices','events']
SCAN_CYCLE_MINUTES=5
DAYS_TO_KEEP_EVENTS=7
REPORT_DASHBOARD_URL='https://something.something.something'
DIG_GET_IP_ARG='-4 myip.opendns.com @resolver1.opendns.com'
UI_LANG='English'
UI_PRESENCE=['online','offline','archived']

# Email
#---------------------------
REPORT_MAIL=True
SMTP_SERVER='something.something'
SMTP_PORT=25
REPORT_TO='something@something.something'
REPORT_FROM='Pi-Alert@something.something'
SMTP_SKIP_LOGIN=True
SMTP_USER='user@gmail.com'
SMTP_PASS='password'
SMTP_SKIP_TLS=True
SMTP_FORCE_SSL=False

# Webhooks
#---------------------------
REPORT_WEBHOOK=False
WEBHOOK_URL='http://n8n.local:5555/webhook-test/aaaaaaaa-aaaa-aaaa-aaaaa-aaaaaaaaaaaa'
WEBHOOK_PAYLOAD='json'
WEBHOOK_REQUEST_METHOD='GET'

# Apprise
#---------------------------
REPORT_APPRISE=False
APPRISE_HOST='http://localhost:8000/notify'
APPRISE_URL='mailto://smtp-relay.sendinblue.com:587?from=user@gmail.com&name=apprise&user=user@gmail.com&pass=password&to=user@gmail.com'
APPRISE_PAYLOAD='html'

# NTFY
#---------------------------
REPORT_NTFY=False
NTFY_HOST='https://ntfy.sh'
NTFY_TOPIC='replace_my_secure_topicname_91h889f28'
NTFY_USER='user'
NTFY_PASSWORD='passw0rd'

# PUSHSAFER
#---------------------------
REPORT_PUSHSAFER=False
PUSHSAFER_TOKEN='ApiKey'

# MQTT
#---------------------------
REPORT_MQTT=False
MQTT_BROKER='192.168.1.2'
MQTT_PORT=1883
MQTT_USER='mqtt'
MQTT_PASSWORD='passw0rd'
MQTT_QOS=0
MQTT_DELAY_SEC=2

# DynDNS
#---------------------------
DDNS_ACTIVE=False
DDNS_DOMAIN='your_domain.freeddns.org'
DDNS_USER='dynu_user'
DDNS_PASSWORD='A0000000B0000000C0000000D0000000'
DDNS_UPDATE_URL='https://api.dynu.com/nic/update?'

# PiHole
#---------------------------
PIHOLE_ACTIVE=False
DHCP_ACTIVE=False

# Pholus
#---------------------------
PHOLUS_ACTIVE=True
PHOLUS_TIMEOUT=120
PHOLUS_FORCE=False
PHOLUS_RUN='once'
PHOLUS_RUN_TIMEOUT=600
PHOLUS_RUN_SCHD='0 4 * * *'
PHOLUS_DAYS_DATA=7

# Nmap
#---------------------------
NMAP_ACTIVE=True
NMAP_TIMEOUT=150
NMAP_RUN='none'
NMAP_RUN_SCHD='0 2 * * *'
NMAP_ARGS='-p -10000'

# API
#---------------------------
API_CUSTOM_SQL='SELECT * FROM Devices WHERE dev_PresentLastScan = 0'

#-------------------IMPORTANT INFO-------------------#
#   This file is ingested by a python script, so if  #
#        modified it needs to use python syntax      #
#-------------------IMPORTANT INFO-------------------#

Paste your docker-compose.yml and .env (remove personal info)

docker-compose.yml

  pialert:
    image: jokobsk/pi.alert:latest
    container_name: pialert
    hostname: pialert
    restart: always
    network_mode: host
    volumes:
      - pialert_db:/home/pi/pialert/db
      - pialert_config:/home/pi/pialert/config
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    environment:
      - TZ=something/something
      - PUID=1000
      - PGID=1000
    labels:
      - traefik.enable=true
      - traefik.http.routers.pialert.rule=Host(`pialert.something`)
      - traefik.http.routers.pialert.entrypoints=web,websecure
      - traefik.http.routers.pialert.tls.certresolver=letsencrypt
      - traefik.http.services.pialert.loadbalancer.server.port=20211 #Optional

Paste last few lines from pialert.log

You can use tail -100 /home/pi/pialert/front/log/pialert.log

10:39:35 [API] Updating table_settings.json file in /front/api
10:39:35 [Config] Imported new config
10:39:36 [API] Update API starting
10:39:36 [Database] - get_table_as_json - Query: select dev_MAC, dev_Name, dev_DeviceType, dev_Vendor, dev_Group,
                     dev_FirstConnection, dev_LastConnection, dev_LastIP, dev_StaticIP,
                     dev_PresentLastScan, dev_LastNotification, dev_NewDevice,
                     dev_Network_Node_MAC_ADDR, dev_Network_Node_port,
                     dev_Icon from Devices
10:39:36 [Database] - get_table_as_json - returning 55 rows with columns: ['dev_MAC', 'dev_Name', 'dev_DeviceType', 'dev_Vendor', 'dev_Group', 'dev_FirstConnection', 'dev_LastConnection', 'dev_LastIP', 'dev_StaticIP', 'dev_PresentLastScan', 'dev_LastNotification', 'dev_NewDevice', 'dev_Network_Node_MAC_ADDR', 'dev_Network_Node_port', 'dev_Icon']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Nmap_Scan
10:39:36 [Database] - get_table_as_json - returning 279 rows with columns: ['Index', 'MAC', 'Port', 'Time', 'State', 'Service', 'Extra']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Pholus_Scan
10:39:36 [Database] - get_table_as_json - returning 1197 rows with columns: ['Index', 'Info', 'Time', 'MAC', 'IP_v4_or_v6', 'Record_Type', 'Value', 'Extra']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Events where eve_PendingAlertEmail is not 0
10:39:36 [Database] - get_table_as_json - returning 0 rows with columns: ['eve_MAC', 'eve_IP', 'eve_DateTime', 'eve_EventType', 'eve_AdditionalInfo', 'eve_PendingAlertEmail', 'eve_PairEventRowid']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Settings
10:39:36 [Database] - get_table_as_json - returning 66 rows with columns: ['Code_Name', 'Display_Name', 'Description', 'Type', 'Options', 'RegEx', 'Value', 'Group', 'Events']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Plugins_Events
10:39:36 [Database] - get_table_as_json - returning 0 rows with columns: ['Index', 'Plugin', 'Object_PrimaryID', 'Object_SecondaryID', 'DateTimeCreated', 'DateTimeChanged', 'Watched_Value1', 'Watched_Value2', 'Watched_Value3', 'Watched_Value4', 'Status', 'Extra', 'UserData', 'ForeignKey']
10:39:36 [Database] - get_table_as_json - Query: SELECT  * FROM Plugins_History ORDER BY 'Index' DESC
10:39:38 [Database] - get_table_as_json - returning 37900 rows with columns: ['Index', 'Plugin', 'Object_PrimaryID', 'Object_SecondaryID', 'DateTimeCreated', 'DateTimeChanged', 'Watched_Value1', 'Watched_Value2', 'Watched_Value3', 'Watched_Value4', 'Status', 'Extra', 'UserData', 'ForeignKey']
10:39:38 [Database] - get_table_as_json - Query: SELECT  * FROM Plugins_Objects
10:40:14 [Database] - get_table_as_json - returning 1786698 rows with columns: ['Index', 'Plugin', 'Object_PrimaryID', 'Object_SecondaryID', 'DateTimeCreated', 'DateTimeChanged', 'Watched_Value1', 'Watched_Value2', 'Watched_Value3', 'Watched_Value4', 'Status', 'Extra', 'UserData', 'ForeignKey']

10:40:14 [Database] - get_table_as_json - returning 1786698 rows with columns: ['Index', 'Plugin', 'Object_PrimaryID', looks like a very big Query.

Screenshot

[jokob-sk]

Hey,

Probably related to: #253

You can try the troubleshooting mentioned in that thread.

Thanks, j

[jokob-sk]

closing as duplicate, I hope the linked issue addresses this too