MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.85k stars 495 forks source link

DietPi-Software | PHP-FPM: fails to fork multiple processes on single core systems #1298

Closed pablopoo closed 6 years ago

pablopoo commented 6 years ago

Creating a bug report/issue:

Pi-hole timeout when tries to show the block page for blocked domains.

The block page (/var/www/html/pihole/index.php) internally calls a php script on the same server using an http request, but php5-fpm is configured with only 1 server and 1 child process ( /etc/php5/fpm/pool.d/www.conf ) , so the call timeout because can't fork any more process to reply that self request.

Required Information:

Additional Information (if applicable):

Pi-hole version is v3.2 (Latest: v3.2) AdminLTE version is v3.2 (Latest: v3.2) FTL version is vDev-437af07 (Latest: v2.12)

Expected behaviour:

Pi-hole must show the block page instead of a timeout message.

Actual behaviour:

The request from http://pi.hole/pihole/index.php to http://127.0.0.1/admin/scripts/pi-hole/php/queryads.php?domain=$serverName&bp fails.

Steps to reproduce:

In my case, just open a blocked domain.

Fourdee commented 6 years ago

@pablopoo

VM but php5-fm is configured with only 1 server and 1 child process ( /etc/php5/fpm/pool.d/www.conf

Many thanks for the report 👍 i'll need to investigate. Are you running your VM as single core/CPU?

pablopoo commented 6 years ago

@fourdee Yes, it’s a single core hyper-v vm. Mapping 1 core to max 1 one php-fpm process could do the job in some scenarios, but I think limiting it to 1 thread per fpm process is not enough.

k-plan commented 6 years ago

The block page (/var/www/html/pihole/index.php) internally calls a php script on ...

No, path on DietPi is /var/www/html/admin/index.php ... or the symbolic links

root@NanoPi-Neo:~# ls -lah /var/www/ | grep admin
lrwxrwxrwx  1 www-data www-data   19 Dez 11 20:11 admin -> /var/www/html/admin
lrwxrwxrwx  1 www-data www-data   19 Dez 11 20:11 pihole -> /var/www/html/admin

Do you use a Custom Block Page ? Like https://github.com/WaLLy3K/Pi-hole-Block-Page ?

This project is discontinued as of Pi-hole v3.2 If you would like to implement a custom block page, it is since Pi-hole v3.2 recommended to create a file called /var/www/html/pihole/custom.php and use that instead.

This script will not presume where the default document-root is, as installations such as DietPi are known to change this

Pi-hole must show the block page instead of a timeout message. Steps to reproduce: In my case, just open a blocked domain.

171215-0001

The request from http://pi.hole/pihole/index.php to http://127.0.0.1/admin/scripts/pi-hole/php/queryads.php?domain=$serverName&bp fails.

171215-0005

171215-0003

I think, you just want to simple click in Dashboard on a Blocked Domain and get this? http://pi.hole/admin/queries.php?domain=ping.dozuki.com

171215-0004

pablopoo commented 6 years ago

@k-plan I believe that project was integrated in the latest pihole: https://github.com/pi-hole/pi-hole/pull/1688

I didn't have time yet to replicate the issue on a clean install, but I only did normal dietpi and pihole update proceses.

Fourdee commented 6 years ago

@pablopoo

Yes, it’s a single core hyper-v vm. Mapping 1 core to max 1 one php-fpm process could do the job in some scenarios, but I think limiting it to 1 thread per fpm process is not enough.

Yep, please can you try increasing the child count to say 10?

If that resolves it, i'll set pm.max_children = $(( $CPU_CORES_TOTAL * 10 )) by default


edit: x 10 is possibly a little high:

https://myshell.co.uk/blog/2012/07/adjusting-child-processes-for-php-fpm-nginx/ pm.max_children = Total RAM dedicated to the web server / Max child process size - in my case it was 85MB On an average each PHP-FPM process took ~75MB of RAM on my machine. You can check an average memory usage by single PHP-FPM process with this handy command: ps --no-headers -o "rss,cmd" -C php-fpm | awk '{ sum+=$1 } END { printf ("%d%s\n", sum/NR/1024,"M") }'

Fourdee commented 6 years ago

@pablopoo

I've increased the pm.max_children = $(( $CPU_CORES_TOTAL * 3 )) for v160. This should resolve issues, however, if you have a chance to test and verify (https://github.com/Fourdee/DietPi/issues/1298#issuecomment-352170289), i'd appreciate it.

pablopoo commented 6 years ago

@Fourdee

Ok, did this test:

1.- Installed the VirtualBox DietPi image. imagen

2.- Dietpi auto-updated to latest v159

3.- Installed only Pi-hole imagen

-Noticed this time installed php7-fmp instead of php5-fpm

4.- cat /etc/php/7.0/fpm/pool.d/www.conf

imagen

-Got the same config I have in php5-fpm.

5.-Tested Pi-hole:

imagen

-Got the usual blocking page.

6.-Updated Pi-hole: pihole -up pihole already up to date.

7.- Now started to wondering why I got another blocking page, not the text one above I got when first installed pihole months ago, because the only thing I did was install the pihole and dietpi updates. I started comparing my current pihole install files with the test install until I found what I believe is the reason: When pihole is installed the first time using dietpi it uses a different /etc/lighttpd/lighttpd.conf than the one installed after a pihole -up upgrade.

8.- Replaced my test lighttpd.conf:


`server.modules = ( "mod_access", "mod_alias", "mod_compress", "mod_redirect", )

server.document-root = "/var/www" server.upload-dirs = ( "/var/cache/lighttpd/uploads" ) server.errorlog = "/var/log/lighttpd/error.log" server.pid-file = "/var/run/lighttpd.pid" server.username = "www-data" server.groupname = "www-data" server.port = 80

index-file.names = ( "index.php", "index.html", "index.lighttpd.html" ) url.access-deny = ( "~", ".inc" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir = "/var/cache/lighttpd/compress/" compress.filetype = ( "application/javascript", "text/css", "text/html", "text/plain" )

default listening port for IPv6 falls back to the IPv4 port

include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port include_shell "/usr/share/lighttpd/create-mime.assign.pl" include_shell "/usr/share/lighttpd/include-conf-enabled.pl"`


with the one I have trouble:

`# Pi-hole: A black hole for Internet advertisements

(c) 2017 Pi-hole, LLC (https://pi-hole.net)

Network-wide ad blocking via your own hardware.

#

Lighttpd config for Pi-hole

#

This file is copyright under the latest version of the EUPL.

Please see LICENSE file for your rights under this license.

###############################################################################

FILE AUTOMATICALLY OVERWRITTEN BY PI-HOLE INSTALL/UPDATE PROCEDURE.

ANY CHANGES MADE TO THIS FILE AFTER INSTALL WILL BE LOST ON THE NEXT UPDATE

CHANGES SHOULD BE MADE IN A SEPARATE CONFIG FILE:

/etc/lighttpd/external.conf

###############################################################################

server.modules = ( "mod_access", "mod_accesslog", "mod_auth", "mod_expire", "mod_compress", "mod_redirect", "mod_setenv", "mod_rewrite" )

server.document-root = "/var/www/html" server.error-handler-404 = "pihole/index.php" server.upload-dirs = ( "/var/cache/lighttpd/uploads" ) server.errorlog = "/var/log/lighttpd/error.log" server.pid-file = "/var/run/lighttpd.pid" server.username = "www-data" server.groupname = "www-data" server.port = 80 accesslog.filename = "/var/log/lighttpd/access.log" accesslog.format = "%{%s}t|%V|%r|%s|%b"

index-file.names = ( "index.php", "index.html", "index.lighttpd.html" ) url.access-deny = ( "~", ".inc", ".md", ".yml", ".ini" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir = "/var/cache/lighttpd/compress/" compress.filetype = ( "application/javascript", "text/css", "text/html", "text/plain" )

default listening port for IPv6 falls back to the IPv4 port

include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port include_shell "/usr/share/lighttpd/create-mime.assign.pl"

Prevent Lighttpd from enabling Let's Encrypt SSL for every blocked domain

include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

include_shell "find /etc/lighttpd/conf-enabled -name '*.conf' -a ! -name 'letsencrypt.conf' -printf 'include \"%p\"\n' 2>/dev/null"

If the URL starts with /admin, it is the Web interface

$HTTP["url"] =~ "^/admin/" {

Create a response header for debugging using curl -I

setenv.add-response-header = (
    "X-Pi-hole" => "The Pi-hole Web interface is working!",
    "X-Frame-Options" => "DENY"
)

$HTTP["url"] =~ ".ttf$" {
    # Allow Block Page access to local fonts
    setenv.add-response-header = ( "Access-Control-Allow-Origin" => "*" )
}

}

Block . files from being served, such as .git, .github, .gitignore

$HTTP["url"] =~ "^/admin/.(.*)" { url.access-deny = ("") }

Add user chosen options held in external file

include_shell "cat external.conf 2>/dev/null"`

9- Then tested again: imagen

Now got some progress, that was the error message I was looking for.

10.- Edited /etc/php/7.0/fpm/pool.d/www.conf changed pm.max_children from 1 to 2. imagen

11.- restarted php-fpm, then I got this blocking page: imagen

12.- @Fourdee , Answering your question, for me, works with at least 2 max_children. Thanks.

k-plan commented 6 years ago

@pablopoo

Thanks! 👍

@Fourdee

hmm.. will we get the new block page feature?

Fourdee commented 6 years ago

@pablopoo

Many thanks 👍

@k-plan

hmm.. will we get the new block page feature?

I'll take a look, if its just a standard html or php page, we can use that one instead of our current:

# - Generate index page that replaces adverts and prevents popups
cat << _EOF_ > /var/www/index.html
<html>
Blocked by Pi-hole.
<script>window.close();</script>
</html>
_EOF_

When we added the above, Pi-Hole didnt have a block page at that time (if i remember correctly).


Edit looks like the main Pi-Hole index page handles the block page aswell? https://github.com/pi-hole/pi-hole/blob/master/advanced/lighttpd.conf.debian#L30

Needs more research to see if this can be implemented compatible with our global webserver confs, and, if it has no effect on other web applications (eg: nextcloud).

Fourdee commented 6 years ago

I'll mark this as closed, original issue is resolved in v160.

Ticket for block page: https://github.com/Fourdee/DietPi/issues/1313