search

TrafeX / docker-php-nginx

Docker image with PHP-FPM 8.3 & Nginx 1.24 on Alpine Linux
https://hub.docker.com/r/trafex/php-nginx
MIT License
1.33k stars 721 forks source link

The Nginx log indicates a 404 error even though the route exists #133

Closed foremtehan closed 10 months ago

foremtehan commented 1 year ago

I am using the exact same nginx config file in a Laravel application. However, today I noticed that some bots are encountering a 404 error on /robots.txt, whereas when I visit the route myself, everything seems to be working fine. What could be causing this issue?

this is a 404 log on AhrefsBot:

172.71.131.89 - - [09/Jun/2023:04:25:08 +0000] "GET /robots.txt HTTP/1.1" 👉 404 12175 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)" "54.36.148.162" 0.025 0.025 . -

Visit by myself in a browser and i can see the content of file:

172.18.0.1 - - [09/Jun/2023:06:32:38 +0000] "GET /robots.txt HTTP/1.0" 👉 200 24 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/113.0" "my.ip.here" 0.000 - . -

what's the issue?

TrafeX commented 10 months ago

Hi @foremtehan,

I don't see how this could be caused by the Nginx configuration. What I notice in your log lines is that the request from Ahrefs has a upstream response time (0.025) and the request from you doesn't have that. That could mean that the request from Ahrefs is forwarded to PHP-FPM and your request isn't. Is robots.txt an actual file or is it handled by the Laravel application?

foremtehan commented 10 months ago

I have that file under /public/robots.txt, but nevermind; I did not find any 404 errors from that moment until now. Thanks for the response <3