search

zabbix / zabbix-docker

Official Zabbix Dockerfiles
https://www.zabbix.com
GNU Affero General Public License v3.0
2.39k stars 1.38k forks source link

Parameter ZBX_POSTMAXSIZE not affecting Apache config resulting in an HTTP 413 error at 10,000 bytes of payload size #1418

Closed Archerymaister closed 5 months ago

Archerymaister commented 5 months ago
SUMMARY

Setting the parameter ZBX_POSTMAXSIZE for zabbix-web-apache-pgsql above 10k seems to be irrelevant. While it sets the php variable post_max_size, it does not set the limit LimitRequestBody in the /etc/apache2/httpd.conf file which is set to 10000. I came across this error when trying to update a dashboard. Once the JSON payload exceeds exactly 10000 bytes, the http error 413 is thrown. Somehow, setting the parameter ZBX_POSTMAXSIZE to a lower value did not impact this behavior either.

OS / ENVIRONMENT / Used docker-compose files

Oracle Linux Server 8.9 Docker version 24.0.6, build ed223bc

zabbix-web-apache-pgsql:6.4.15-alpine zabbix-server-pgsql:6.4.15-alpine postgres:15.5-alpine3.19

CONFIGURATION
ZBX_POSTMAXSIZE: "5k" # I changed this parameter ranging from 5K to 64M
ZBX_UPLOADMAXFILESIZE: "64M"
ZBX_MEMORYLIMIT: "512M"
ZBX_MAXEXECUTIONTIME: 300
ZBX_MAXINPUTTIME: 300
STEPS TO REPRODUCE

I tested it by increasing the size of the dashboard until the payload in the update call (zabbix.php?action=dashboard.update) reached exactly 10,000 bytes at which point it still succeeded. Then I added only one letter to the dashboard name so the payload had a size of 10,001 bytes and I got the error.

EXPECTED RESULTS

Changing the value of ZBX_POSTMAXSIZE should change the limit when the HTTP error 413 is thrown.

ACTUAL RESULTS

An HTTP 413 error is thrown when the POST payload exceeds 10,001 bytes. The error is displayed as "Failed to update dashboard".

dotneft commented 5 months ago

Hi! Do you have logs from Apache container?

dotneft commented 5 months ago

Default is LimitRequestBody 1073741824. So I do not think it is related.

Archerymaister commented 5 months ago

Hey, thanks for the quick response!

These are from when the size is exactly 10,000 bytes:

[Wed May 29 11:26:28.239242 2024] [php:warn] [pid 308] [client 10.yy.yy.yy:53958] PHP Warning:  PHP Request Startup: POST Content-Length of 10000 bytes exceeds the limit of 5120 bytes in Unknown on line 0, referer: http://10.yy.yy.yy/zabbix.php?action=dashboard.view&dashboardid=323&page=3
WARNING:  database "zabbix" has no actual collation version, but a version was recorded
10.yy.yy.yy - - [29/May/2024:11:26:28 +0000] "POST /zabbix.php?action=dashboard.update&_csrf_token=e940e95c907ab838f1fa35a6dd0b0d0b1a965ad026d25b3fc7deb65faa7d2b49 HTTP/1.1" 200 61 "http://10.yy.yy.yy/zabbix.php?action=dashboard.view&dashboardid=323&page=3" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"

These are when the size is 10,001 bytes:

[Wed May 29 11:22:40.530454 2024] [php:warn] [pid 264] [client 10.yy.yy.yy:51658] PHP Warning:  PHP Request Startup: POST Content-Length of 10001 bytes exceeds the limit of 5120 bytes in Unknown on line 0, referer: http://10.xx.xx.xx/zabbix.php?action=dashboard.view&dashboardid=323
WARNING:  database "zabbix" has no actual collation version, but a version was recorded
10.yy.yy.yy - - [29/May/2024:11:22:40 +0000] "POST /zabbix.php?action=dashboard.update&_csrf_token=e940e95c907ab838f1fa35a6dd0b0d0b1a965ad026d25b3fc7deb65faa7d2b49 HTTP/1.1" 200 222 "http://10.xx.xx.xx/zabbix.php?action=dashboard.view&dashboardid=323" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
[Wed May 29 11:22:41.651257 2024] [php:warn] [pid 213] [client 10.yy.yy.yy:51663] PHP Warning:  PHP Request Startup: POST Content-Length of 10001 bytes exceeds the limit of 5120 bytes in Unknown on line 0, referer: http://10.xx.xx.xx/zabbix.php?action=dashboard.view&dashboardid=323
WARNING:  database "zabbix" has no actual collation version, but a version was recorded
10.yy.yy.yy - - [29/May/2024:11:22:41 +0000] "POST /zabbix.php?action=dashboard.update&_csrf_token=e940e95c907ab838f1fa35a6dd0b0d0b1a965ad026d25b3fc7deb65faa7d2b49 HTTP/1.1" 200 222 "http://10.xx.xx.xx/zabbix.php?action=dashboard.view&dashboardid=323" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
WARNING:  database "zabbix" has no actual collation version, but a version was recorded

Notice, how in both cases php warns that the size exceeds the specified 5k but does not do anything.

At first I disregarded this too because of the default value. But the config /etc/apache2/httpd.conf has these directives set:

#For Req 16 the webserver must be protected against overload
RequestReadTimeout header=10-20,MinRate=500 body=10-20,MinRate=500
Timeout 30
KeepAliveTimeout 5
LimitRequestBody 10000
LimitRequestFields 50
LimitRequestFieldSize 8190
LimitRequestLine 8190
LimitXMLRequestBody 10000
MaxRequestWorkers 256
dotneft commented 5 months ago

Hmm... are you sure you use exactly Alpine image without any modifications? 

cat /etc/apache2/httpd.conf | grep LimitRequestBody

no output on Alpine.

Archerymaister commented 5 months ago

At first I was using my company's mirror but to rule out any changes there I switched to the official docker hub image. Could it be that docker is caching something because both images are named the same?

dotneft commented 5 months ago

please share sha256 for the image.

Archerymaister commented 5 months ago

They seem to be the same:

[root@xxx docker]# docker inspect --format='{{index .RepoDigests 0}}' zabbix/zabbix-web-apache-pgsql:6.4.15-alpine
zabbix/zabbix-web-apache-pgsql@sha256:5cc98d3ffcff1148628c51fa945369f3caf010c83876b8e93816e452ec6eec3a
[root@xxx docker]# docker inspect --format='{{index .RepoDigests 0}}' devops.company.de/yyy/zabbix-web-apache-pgsql:6.4.15-alpine
zabbix/zabbix-web-apache-pgsql@sha256:5cc98d3ffcff1148628c51fa945369f3caf010c83876b8e93816e452ec6eec3a
dotneft commented 5 months ago

try to inspect running container.

Archerymaister commented 5 months ago

This is what it gave me:

[root@xxx docker]# docker inspect 6169a90aa4f1 | grep sha
        "Image": "sha256:81d4cce37f63e234ada13ebd3acf16ed482e19697069fb0b2210211705f63851",
            "WorkingDir": "/usr/share/zabbix",
                "com.docker.compose.image": "sha256:81d4cce37f63e234ada13ebd3acf16ed482e19697069fb0b2210211705f63851",
dotneft commented 5 months ago

Looks like we do not have such image...

Archerymaister commented 5 months ago

I got the same hash from the official image. There might be some shenanigans happening on our end that I am not aware of. If you say you do not have anything to do with it I'll try to find out what is happening in my company! Thank you for your help nevertheless!

dotneft commented 5 months ago

Yes, most biggest surprise for me, that Alpine image for Apache web-server does not have such LimitRequestBody parameters at all.

Archerymaister commented 5 months ago

The default value for it is kinda big. And the ZBX_POSTMAXSIZE seems not to enforce anything. Setting it to 5K does not hinder me from sending bigger payloads, it only prints a warning in the log which the user never sees.