search

AdguardTeam / AdGuardHome

Network-wide ads & trackers blocking DNS server
https://adguard.com/adguard-home/overview.html
GNU General Public License v3.0
25.57k stars 1.84k forks source link

Unnecessary memory allocations when loading filters metadata #604

Closed zeus10000 closed 5 years ago

zeus10000 commented 5 years ago

Env: Ubuntu 16.04 AWS, 1vCPU, 1GB RAM.

Shortly after installing v0.93, enabling DOT, and adding some custom filters started seeing no memory left for system use.

=========================================== systemctl status adguard.home -bash: fork: Cannot allocate memory top -bash: fork: Cannot allocate memory

ameshkov commented 5 years ago

Shortly after installing v0.93, enabling DOT, and adding some custom filters started seeing no memory left for system use.

Just in case, what exact custom filters have you added? Could you share your AG config?

ameshkov commented 5 years ago

Also, please show what you have in the output: ps aux --sort -rss

tomiams commented 5 years ago

Could be relevant. Here's the top 2 processes after running top on Raspberry Pi 3B+

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                              
 5190 root      20   0 1203284 879852   5412 S  34.0 92.7  43:14.92 AdGuardHome                                                                                                                                          
 5705 pi        20   0    8108   1048    580 R   0.7  0.1   0:00.55 top

The pi is only used for AGH. On 0.92 Hotfix 2 the pi temperature was always around 49C. On 0.93 it is always around 55C.

More importantly, the settings tab on AGH web view loaded instantly on 0.92 while it takes more than 30 seconds to load on 0.93. The filters tab always shows empty filters and starts redownloading all the filters and it is a couple of minutes before it actually shows the filters including the custom ones. This happens every time I open the web view.

I'm thinking this is all related to the same issue so I did not create a separate one.

I have a single upstream DNSCrypt (sdns://...) My filters combine to around 1.2million rules

Here's ps aux --sort -rss

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      5717 74.1 81.3 1021332 772560 ?      Ssl  12:31   2:55 /home/pi/AdGuardHome/AdGuardHome --host 192.168.2.100
root         1  0.0  0.3   9588  3728 ?        Ss   Feb26   0:10 /sbin/init
pi        5662  0.1  0.3   6232  3280 pts/0    Ss   12:28   0:00 -bash
pi        5744  0.0  0.3   7888  2908 pts/0    R+   12:35   0:00 ps aux --sort -rss
root        95  0.0  0.2  20980  2684 ?        Ss   Feb26   0:20 /lib/systemd/systemd-journald
root       306  0.0  0.1   2872  1320 ?        Ss   Feb26   0:04 /sbin/dhcpcd -q -b
message+   289  0.0  0.1   6492   952 ?        Ss   Feb26   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
pi        5650  0.0  0.0  11252   916 ?        S    12:28   0:00 (sd-pam)
pi        5659  0.0  0.0  11656   752 ?        S    12:28   0:00 sshd: pi@pts/0
root      5641  0.1  0.0  11520   724 ?        Ss   12:28   0:00 sshd: pi [priv]
pi        5647  0.0  0.0   9656   716 ?        Ss   12:28   0:00 /lib/systemd/systemd --user
root       320  0.0  0.0   7380   648 ?        Ss   Feb26   0:00 /lib/systemd/systemd-logind
avahi      287  0.0  0.0   6640   280 ?        Ss   Feb26   0:44 avahi-daemon: running [raspberrypi.local]
root       281  0.0  0.0  23756   256 ?        Ssl  Feb26   0:04 /usr/sbin/rsyslogd -n
root       361  0.0  0.0  10200    64 ?        Ss   Feb26   0:00 /usr/sbin/sshd -D
root       121  0.0  0.0  14348    56 ?        Ss   Feb26   0:01 /lib/systemd/systemd-udevd
systemd+   237  0.0  0.0  17276    56 ?        Ssl  Feb26   0:00 /lib/systemd/systemd-timesyncd
root       321  0.0  0.0   5296    56 ?        Ss   Feb26   0:02 /usr/sbin/cron -f
root         2  0.0  0.0      0     0 ?        S    Feb26   0:00 [kthreadd]
root         4  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/0:0H]
root         6  0.0  0.0      0     0 ?        I<   Feb26   0:00 [mm_percpu_wq]
root         7  1.9  0.0      0     0 ?        S    Feb26  23:47 [ksoftirqd/0]
root         8  0.0  0.0      0     0 ?        I    Feb26   0:15 [rcu_sched]
root         9  0.0  0.0      0     0 ?        I    Feb26   0:00 [rcu_bh]
root        10  0.0  0.0      0     0 ?        S    Feb26   0:00 [migration/0]
root        11  0.0  0.0      0     0 ?        S    Feb26   0:00 [cpuhp/0]
root        12  0.0  0.0      0     0 ?        S    Feb26   0:00 [cpuhp/1]
root        13  0.0  0.0      0     0 ?        S    Feb26   0:00 [migration/1]
root        14  0.0  0.0      0     0 ?        S    Feb26   0:20 [ksoftirqd/1]
root        16  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/1:0H]
root        17  0.0  0.0      0     0 ?        S    Feb26   0:00 [cpuhp/2]
root        18  0.0  0.0      0     0 ?        S    Feb26   0:00 [migration/2]
root        19  0.0  0.0      0     0 ?        S    Feb26   0:41 [ksoftirqd/2]
root        21  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/2:0H]
root        22  0.0  0.0      0     0 ?        S    Feb26   0:00 [cpuhp/3]
root        23  0.0  0.0      0     0 ?        S    Feb26   0:00 [migration/3]
root        24  0.0  0.0      0     0 ?        S    Feb26   0:23 [ksoftirqd/3]
root        26  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/3:0H]
root        27  0.0  0.0      0     0 ?        S    Feb26   0:00 [kdevtmpfs]
root        28  0.0  0.0      0     0 ?        I<   Feb26   0:00 [netns]
root        33  0.0  0.0      0     0 ?        S    Feb26   0:00 [khungtaskd]
root        34  0.0  0.0      0     0 ?        S    Feb26   0:03 [oom_reaper]
root        35  0.0  0.0      0     0 ?        I<   Feb26   0:00 [writeback]
root        36  0.0  0.0      0     0 ?        S    Feb26   0:00 [kcompactd0]
root        37  0.0  0.0      0     0 ?        I<   Feb26   0:00 [crypto]
root        38  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kblockd]
root        39  0.0  0.0      0     0 ?        I<   Feb26   0:00 [watchdogd]
root        40  0.0  0.0      0     0 ?        I<   Feb26   0:00 [rpciod]
root        41  0.0  0.0      0     0 ?        I<   Feb26   0:00 [xprtiod]
root        44  0.4  0.0      0     0 ?        S    Feb26   5:03 [kswapd0]
root        45  0.0  0.0      0     0 ?        I<   Feb26   0:00 [nfsiod]
root        55  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kthrotld]
root        56  0.0  0.0      0     0 ?        I<   Feb26   0:00 [iscsi_eh]
root        57  0.0  0.0      0     0 ?        I<   Feb26   0:00 [dwc_otg]
root        58  0.0  0.0      0     0 ?        I<   Feb26   0:00 [DWC Notificatio]
root        59  0.0  0.0      0     0 ?        S<   Feb26   0:00 [vchiq-slot/0]
root        60  0.0  0.0      0     0 ?        S<   Feb26   0:00 [vchiq-recy/0]
root        61  0.0  0.0      0     0 ?        S<   Feb26   0:00 [vchiq-sync/0]
root        62  0.0  0.0      0     0 ?        S    Feb26   0:00 [vchiq-keep/0]
root        63  0.0  0.0      0     0 ?        S<   Feb26   0:00 [SMIO]
root        66  0.4  0.0      0     0 ?        S    Feb26   5:58 [mmcqd/0]
root        67  0.0  0.0      0     0 ?        S    Feb26   0:00 [jbd2/mmcblk0p7-]
root        68  0.0  0.0      0     0 ?        I<   Feb26   0:00 [ext4-rsv-conver]
root        69  0.0  0.0      0     0 ?        I<   Feb26   0:00 [ipv6_addrconf]
root        86  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/3:1H]
root       145  0.0  0.0      0     0 ?        S    Feb26   0:00 [irq/169-usb-001]
avahi      301  0.0  0.0   6392     0 ?        S    Feb26   0:00 avahi-daemon: chroot helper
root       305  0.0  0.0      0     0 ?        I<   Feb26   0:00 [cfg80211]
root       356  0.0  0.0   3964     0 ?        Ss+  Feb26   0:00 /sbin/agetty --keep-baud 115200,38400,9600 ttyAMA0 vt220
root       357  0.0  0.0   4184     0 tty1     Ss+  Feb26   0:00 /sbin/agetty --noclear tty1 linux
root       372  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/1:1H]
root       426  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/0:1H]
root       590  0.0  0.0      0     0 ?        I<   Feb26   0:00 [kworker/2:1H]
root      4728  0.0  0.0      0     0 ?        I    08:39   0:00 [kworker/2:0]
root      5336  0.0  0.0      0     0 ?        I    11:06   0:00 [kworker/3:2]
root      5372  0.0  0.0      0     0 ?        I    11:15   0:01 [kworker/3:0]
root      5473  0.0  0.0      0     0 ?        I    11:43   0:00 [kworker/u8:2]
root      5569  0.0  0.0      0     0 ?        I    12:08   0:00 [kworker/1:1]
root      5623  0.0  0.0      0     0 ?        I    12:23   0:00 [kworker/2:1]
root      5632  0.0  0.0      0     0 ?        I    12:23   0:00 [kworker/0:2]
root      5688  0.0  0.0      0     0 ?        I    12:28   0:00 [kworker/1:0]
root      5690  0.0  0.0      0     0 ?        I    12:28   0:00 [kworker/0:0]
root      5703  0.0  0.0      0     0 ?        I    12:29   0:00 [kworker/u8:1]
root      5706  0.0  0.0      0     0 ?        I    12:30   0:00 [kworker/2:2]
root      5712  0.0  0.0      0     0 ?        I    12:31   0:00 [kworker/3:1]
root      5734  0.0  0.0      0     0 ?        I    12:33   0:00 [kworker/1:2]
root      5743  0.0  0.0      0     0 ?        I    12:34   0:00 [kworker/0:1]
ameshkov commented 5 years ago

@tomiams frankly, I think your issue is different.

My filters combine to around 1.2million rules

I must admit that's A LOT. How many filters do you have installed?

Are these filters public? Can you share your config file so that I could reproduce it on my side?

tomiams commented 5 years ago

I must admit that's A LOT. How many filters do you have installed?

Likewise, I still need to clean them up. Didn't feel the need to rush with an average processing time in the twenties.

Here's my config:

bind_host: 192.168.2.100
bind_port: 3000
auth_name: redacted
auth_pass: "redacted"
language: ""
dns:
  bind_host: 0.0.0.0
  port: 53
  protection_enabled: true
  filtering_enabled: true
  blocked_response_ttl: 300
  querylog_enabled: true
  ratelimit: 150
  ratelimit_whitelist: []
  refuse_any: true
  bootstrap_dns: tls://1.1.1.1
  parental_sensitivity: 13
  parental_enabled: false
  safesearch_enabled: false
  safebrowsing_enabled: false
  upstream_dns:
  - sdns://AQMAAAAAAAAAFDE3Ni4xMDMuMTMwLjEzMDo1NDQzINErR_JS3PLCu_iZEIbq95zkSV2LFsigxDIuUso_OQhzIjIuZG5zY3J5cHQuZGVmYXVsdC5uczEuYWRndWFyZC5jb20
tls:
  enabled: false
  server_name: ""
  force_https: false
  port_https: 443
  port_dns_over_tls: 853
  certificate_chain: ""
  private_key: ""
filters:
- enabled: true
  url: https://raw.githubusercontent.com/marktron/fakenews/master/fakenews
  name: Fake News
  id: 1544278924
- enabled: true
  url: https://tgc.cloud/downloads/iOSAds.txt
  name: iOS ads
  id: 1544390137
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/ABP-Clickbait-Blocklist.txt
  name: ABP-Clickbait-Blocklist.txt
  id: 1544390138
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/ABP-X-Files.txt
  name: ABP X Files
  id: 1544390139
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/ABPindo.txt
  name: ABPindo
  id: 1544390140
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/AakList.txt
  name: AAkList
  id: 1544390141
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/Adblock-Filters-by-Gurudee.txt
  name: Gurudee
  id: 1544390142
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/Ayucat-Powerful.txt
  name: Ayucat-Powerful
  id: 1544390144
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/CAMELEON-Hosts.txt
  name: CAMELEON Hosts
  id: 1544390145
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/CryptoWall-Ransomware-C2-Domain-blocklist.txt
  name: CryptoWall-Ransomware-C2-Domain-blocklist
  id: 1544390146
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/DNS-BH-Free-Web-Hosts.txt
  name: DNS-BH-Free-Web-Hosts
  id: 1544390147
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/DNS-BH-Malware-Domains.txt
  name: DNS-BH-Malware-Domains
  id: 1544390148
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/Steven-Blacks-Trackers.txt
  name: Steven Blacks Trackers
  id: 1544390149
- enabled: false
  url: https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/Steven-Blacks-Unified-Hosts.txt
  name: Steven Blacks Unified
  id: 1544390150
- enabled: true
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/ParsedBlacklists/Schacks-Adblock-Plus-Liste.txt
  name: Schacks-Adblock-Plus-Liste
  id: 1544390151
- enabled: true
  url: https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt
  name: AdGuard Simplified Domain Names filter
  id: 1542890249
- enabled: true
  url: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn/hosts
  name: Hosts
  id: 1542890340
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/4.txt
  name: AdGuard Social Media filter
  id: 1542890341
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/14.txt
  name: AdGuard Annoyances filter
  id: 1542890342
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/5.txt
  name: AdGuard Experimental filter
  id: 1542890343
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/12.txt
  name: AdGuard Safari filter
  id: 1542890344
- enabled: true
  url: https://raw.githubusercontent.com/Spam404/lists/master/adblock-list.txt
  name: Spam404
  id: 1542890345
- enabled: true
  url: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
  name: No Track
  id: 1542890346
- enabled: true
  url: https://raw.githubusercontent.com/hl2guide/All-in-One-Customized-Adblock-List/master/deanoman-adblocklist.txt
  name: "\U0001F60D All-in-One Customized Adblock List 2.5"
  id: 1542890350
- enabled: true
  url: http://hosts-file.net/hphosts-partial.txt
  name: HpHosts Update
  id: 1542890352
- enabled: true
  url: http://hphosts.gt500.org/hosts.txt
  name: HpHosts
  id: 1542890353
- enabled: true
  url: https://raw.githubusercontent.com/BlackJack8/iOSAdblockList/master/Hosts.txt
  name: 'iOS Ads #2'
  id: 1548514713
- enabled: true
  url: https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/nocoin.txt
  name: NoCoin Filter List
  id: 1549376462
- enabled: true
  url: https://www.i-dont-care-about-cookies.eu/abp/
  name: I don't care about cookies
  id: 1549376463
- enabled: true
  url: https://easylist.to/easylist/easylist.txt
  name: EasyList
  id: 1549376464
- enabled: true
  url: https://easylist.to/easylist/easyprivacy.txt
  name: EasyPrivacy
  id: 1549376465
- enabled: true
  url: https://easylist.to/easylist/fanboy-annoyance.txt
  name: Fanboy's Annoyance List
  id: 1549376466
- enabled: true
  url: https://easylist.to/easylist/fanboy-social.txt
  name: Fanboy's Social Blocking List
  id: 1549376467
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/3.txt
  name: AdGuard Tracking Protection filter
  id: 1549376468
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/2.txt
  name: AdGuard Base filter
  id: 1549376469
- enabled: true
  url: https://filters.adtidy.org/extension/chromium/filters/11.txt
  name: AdGuard Mobile Ads filter
  id: 1549376470
- enabled: true
  url: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=adblockplus&mimetype=plaintext
  name: Peter Lowe's List
  id: 1549376471
- enabled: false
  url: https://github.com/deathbybandaid/piholeparser/raw/master/Subscribable-Lists/CombinedBlacklists/CombinedBlackLists.txt
  name: DeathByBandAid
  id: 1549373333
- enabled: true
  url: https://easylist-downloads.adblockplus.org/malwaredomains_full.txt
  name: Malware Domains
  id: 1550617054
user_rules:
- 127.0.0.1 local
- 192.168.2.100 raspberrypi.local
- '||doubleclick.net^$important'
- '||googleadservices.com^$important'
- '||googlesyndication.com^$important'
- '||crashlytics.com^$important'
- '||ads.doubleclick.net^$important'
- '||ad.youtube.com^$important'
- '||ads.youtube.com^$important'
- '@@||clients1.google.com'
- '@@||pool.ntp.org^'
dhcp:
  enabled: false
  interface_name: ""
  gateway_ip: 192.168.2.1
  subnet_mask: 255.255.255.0
  range_start: 192.168.2.100
  range_end: 192.168.2.202
  lease_duration: 864000
log_file: ""
verbose: false
schema_version: 2
zeus10000 commented 5 years ago

@ameshkov - I believe my issue and @tomiams issue is same. Even I had more than a million filters (all publicly available). Since I was restricted to 1Gb ram, my VM was starved of resources.

I added a 4GB Swap which seems to resolve the issue with Adguard using upto 2GB of Swap on top of 1GB RAM with all filters activated in below config.

Most of the Memory was used by two of the fiters, upon disabling them I saw memory usage return to within 1GB.

free -m total used free shared buff/cache available Mem: 990 892 72 0 25 8 Swap: 4095 120 3975

ps aux --sort -rss USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1669 1.9 87.3 1146224 885660 ? Ssl 08:23 17:09 /root/AdGuardHome/AdGuardHome --host 0.0.0.0 root 1091 0.0 0.3 5720 3520 ? S<Ls 06:51 0:09 /sbin/iscsid root 2599 0.0 0.3 36228 3300 pts/0 R+ 23:15 0:00 ps aux --sort -rss root 2567 0.0 0.2 21316 2272 pts/0 S 22:36 0:00 /bin/bash root 2485 0.0 0.1 92804 1796 ? Ss 22:34 0:00 sshd: ubuntu [priv] root 2566 0.0 0.1 55740 1636 pts/0 S 22:36 0:00 sudo -s root 392 0.0 0.1 28352 1560 ? Ss 06:51 0:01 /lib/systemd/systemd-journald daemon 1037 0.0 0.1 26044 1536 ? Ss 06:51 0:00 /usr/sbin/atd -f root 1165 0.0 0.1 14472 1436 ttyS0 Ss+ 06:51 0:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyS0 vt220 root 888 0.0 0.1 16124 1404 ? Ss 06:51 0:00 /sbin/dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -I -df /var/lib/dhcp/dhclient6.eth0.leases eth0 ubuntu 2487 0.0 0.1 45148 1284 ? Ss 22:34 0:00 /lib/systemd/systemd --user root 1164 0.0 0.1 14656 1244 tty1 Ss+ 06:51 0:00 /sbin/agetty --noclear tty1 linux ubuntu 2550 0.0 0.1 92804 1140 ? S 22:34 0:00 sshd: ubuntu@pts/0 ubuntu 2551 0.0 0.1 21412 1104 pts/0 Ss 22:34 0:00 -bash root 455 0.0 0.1 42376 1044 ? Ss 06:51 0:00 /lib/systemd/systemd-udevd root 1100 0.0 0.1 13372 1044 ? Ss 06:51 0:00 /sbin/mdadm --monitor --pid-file /run/mdadm/monitor.pid --daemonise --scan --syslog root 1044 0.0 0.0 27728 900 ? Ss 06:51 0:00 /usr/sbin/cron -f root 1105 0.0 0.0 279340 892 ? Ssl 06:51 0:00 /usr/lib/policykit-1/polkitd --no-debug root 1084 0.0 0.0 65512 776 ? Ss 06:51 0:00 /usr/sbin/sshd -D root 1061 0.0 0.0 28616 692 ? Ss 06:51 0:00 /lib/systemd/systemd-logind message+ 1054 0.0 0.0 42896 676 ? Ss 06:51 0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation syslog 1059 0.0 0.0 260628 620 ? Ssl 06:51 0:00 /usr/sbin/rsyslogd -n root 1057 0.0 0.0 4396 548 ? Ss 06:51 0:00 /usr/sbin/acpid root 425 0.0 0.0 94772 412 ? Ss 06:51 0:00 /sbin/lvmetad -f root 1 0.0 0.0 119592 280 ? Ss 06:51 0:03 /sbin/init root 1036 0.0 0.0 274488 176 ? Ssl 06:51 0:01 /usr/lib/accountsservice/accounts-daemon ubuntu 2493 0.0 0.0 143044 56 ? S 22:34 0:00 (sd-pam) systemd+ 557 0.0 0.0 100324 28 ? Ssl 06:51 0:00 /lib/systemd/systemd-timesyncd root 1090 0.0 0.0 5220 28 ? Ss 06:51 0:01 /sbin/iscsid root 2 0.0 0.0 0 0 ? S 06:51 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S 06:51 0:02 [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S< 06:51 0:00 [kworker/0:0H] root 7 0.0 0.0 0 0 ? S 06:51 0:00 [rcu_sched] root 8 0.0 0.0 0 0 ? S 06:51 0:00 [rcu_bh] root 9 0.0 0.0 0 0 ? S 06:51 0:00 [migration/0] root 10 0.0 0.0 0 0 ? S 06:51 0:00 [watchdog/0] root 11 0.0 0.0 0 0 ? S 06:51 0:00 [kdevtmpfs] root 12 0.0 0.0 0 0 ? S< 06:51 0:00 [netns] root 13 0.0 0.0 0 0 ? S< 06:51 0:00 [perf] root 14 0.0 0.0 0 0 ? S 06:51 0:00 [xenwatch] root 15 0.0 0.0 0 0 ? S 06:51 0:00 [xenbus] root 17 0.0 0.0 0 0 ? S 06:51 0:00 [khungtaskd] root 18 0.0 0.0 0 0 ? S< 06:51 0:00 [writeback] root 19 0.0 0.0 0 0 ? SN 06:51 0:00 [ksmd] root 20 0.0 0.0 0 0 ? SN 06:51 0:01 [khugepaged] root 21 0.0 0.0 0 0 ? S< 06:51 0:00 [crypto] root 22 0.0 0.0 0 0 ? S< 06:51 0:00 [kintegrityd] root 23 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 24 0.0 0.0 0 0 ? S< 06:51 0:00 [kblockd] root 25 0.0 0.0 0 0 ? S< 06:51 0:00 [ata_sff] root 26 0.0 0.0 0 0 ? S< 06:51 0:00 [md] root 27 0.0 0.0 0 0 ? S< 06:51 0:00 [devfreq_wq] root 30 0.0 0.0 0 0 ? S 06:51 0:07 [kswapd0] root 31 0.0 0.0 0 0 ? S< 06:51 0:00 [vmstat] root 32 0.0 0.0 0 0 ? S 06:51 0:00 [fsnotify_mark] root 33 0.0 0.0 0 0 ? S 06:51 0:00 [ecryptfs-kthrea] root 49 0.0 0.0 0 0 ? S< 06:51 0:00 [kthrotld] root 50 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 51 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 52 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 53 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 54 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 55 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 56 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 57 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 58 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 59 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 60 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 61 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 62 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 63 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 64 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 65 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 66 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 67 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 68 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 69 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 70 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 71 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 72 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 73 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 74 0.0 0.0 0 0 ? S 06:51 0:00 [scsi_eh_0] root 75 0.0 0.0 0 0 ? S< 06:51 0:00 [scsi_tmf_0] root 76 0.0 0.0 0 0 ? S 06:51 0:00 [scsi_eh_1] root 77 0.0 0.0 0 0 ? S< 06:51 0:00 [scsi_tmf_1] root 79 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 83 0.0 0.0 0 0 ? S< 06:51 0:00 [ipv6_addrconf] root 96 0.0 0.0 0 0 ? S< 06:51 0:00 [deferwq] root 248 0.0 0.0 0 0 ? S< 06:51 0:00 [raid5wq] root 278 0.0 0.0 0 0 ? S< 06:51 0:00 [bioset] root 301 0.0 0.0 0 0 ? S 06:51 0:00 [jbd2/xvda1-8] root 302 0.0 0.0 0 0 ? S< 06:51 0:00 [ext4-rsv-conver] root 363 0.0 0.0 0 0 ? S< 06:51 0:00 [kworker/0:1H] root 374 0.0 0.0 0 0 ? S< 06:51 0:00 [iscsi_eh] root 379 0.0 0.0 0 0 ? S< 06:51 0:00 [ib_addr] root 383 0.0 0.0 0 0 ? S< 06:51 0:00 [ib_mcast] root 384 0.0 0.0 0 0 ? S< 06:51 0:00 [ib_nl_sa_wq] root 386 0.0 0.0 0 0 ? S< 06:51 0:00 [ib_cm] root 389 0.0 0.0 0 0 ? S< 06:51 0:00 [iw_cm_wq] root 391 0.0 0.0 0 0 ? S< 06:51 0:00 [rdma_cm] root 402 0.0 0.0 0 0 ? S 06:51 0:00 [kauditd] root 2412 0.0 0.0 0 0 ? S 20:13 0:00 [kworker/0:2] root 2482 0.0 0.0 0 0 ? S 22:30 0:00 [kworker/u30:1] root 2488 0.0 0.0 0 0 ? S 22:34 0:00 [kworker/0:1] root 2585 0.0 0.0 0 0 ? S 22:45 0:00 [kworker/u30:2]

Config:

bind_host: 0.0.0.0 bind_port: 3000 auth_name: ######## auth_pass: ######## language: "" dns: bind_host: 0.0.0.0 port: 53 protection_enabled: true filtering_enabled: true blocked_response_ttl: 10 querylog_enabled: true ratelimit: 20 ratelimit_whitelist: [] refuse_any: true bootstrap_dns: 8.8.8.8:53 parental_sensitivity: 13 parental_enabled: true safesearch_enabled: false safebrowsing_enabled: true upstream_dns:

ameshkov commented 5 years ago

Ah, got it, so it's huge filters in both cases after all.

I think I know the reason, v0.93 loads filter from disk more often, and this might be the culprit.

ameshkov commented 5 years ago

@tomiams on a side note:)

bootstrap_dns: tls://1.1.1.1 -- currently, only plain DNS are supported here