Closed wwklnd closed 1 year ago
cc @MichaIng
@wwklnd you should split your reports in 3 please:
This will lead to a better follow up. Thanks 👍
Firefox reports two X-Robots-Tag headers, one of which reads "none" and one of which reads "noindex, nofollow", so the warning is correct about one header but incorrect overall.
This was and is the same with all headers Nextcloud checks for: If there are two identical headers, it intentionally prints the warning, even if both contain the same "correct" value. We do not know how (and which) search engines deal with two headers, whether they respect the stronger, the weaker, the first, the second or none. It makes sense IMO to expect a single unambiguous header value, since its about the security and privacy of your data. Find the source for the second header, eliminate it, and the warning will be resolved. Or ignore it and hope for the best.
I guess you use the Apache2 webserver? The .htaccess
shipped with Nextcloud assures that no 2 headers are set: It unsets headers from the "onsuccess" table and sets them in the "always" table. And since this is the last (highest priority) config applied by Apache2 for the Nextcloud directory, it should be correct what it sends out. So in this case the culprit would be e.g. on a proxy or CDN in front of your Apache2 webserver.
EDIT: Ah, I just see that you use Nginx. In this case .htaccess
is not used at all and you need to check your Nginx configs instead. I didn't know that Nginx is even possible to send our two identical header keys (like Apache2). Proxy or CDN is still a possible culprit.
EDIT2:
Community Docker image
Is is actually supported/intended to update a Nextcloud Docker container via admin panel? I thought, and it makes sense, to update it only with the container itself. Would be quite a logical explanation that the NC25 Docker container's Nginx config still ships the none
header, while you need to wait for a NC26 version of the container. The same may or may not be responsible for your avatar issues, and the missing PHP module.
I can confirm the broken integrity check shortcut link. Actually the whole NC26 docs version is missing: https://docs.nextcloud.com/server/26/ This should fix it: https://github.com/nextcloud/documentation/pull/9947
I can confirm that i had the Warning with Nginx, after upgrade from 25.0.4 to 25.0.5, because I had a configuration file setting the header to none
.
After changing that file to noindex, nofollow
warning is gone.
@wwklnd you should split your reports in 3 please:
* This one about the Header tag * a 2nd about docs link * a 3rd about avatar
This will lead to a better follow up. Thanks +1
Ah, sorry! I'll do that!
Firefox reports two X-Robots-Tag headers, one of which reads "none" and one of which reads "noindex, nofollow", so the warning is correct about one header but incorrect overall.
This was and is the same with all headers Nextcloud checks for: If there are two identical headers, it intentionally prints the warning, even if both contain the same "correct" value. We do not know how (and which) search engines deal with two headers, whether they respect the stronger, the weaker, the first, the second or none. It makes sense IMO to expect a single unambiguous header value, since its about the security and privacy of your data. Find the source for the second header, eliminate it, and the warning will be resolved. Or ignore it and hope for the best.
Ah! That makes sense.
I guess you use the Apache2 webserver? The
.htaccess
shipped with Nextcloud assures that no 2 headers are set: It unsets headers from the "onsuccess" table and sets them in the "always" table. And since this is the last (highest priority) config applied by Apache2 for the Nextcloud directory, it should be correct what it sends out. So in this case the culprit would be e.g. on a proxy or CDN in front of your Apache2 webserver. EDIT: Ah, I just see that you use Nginx. In this case.htaccess
is not used at all and you need to check your Nginx configs instead. I didn't know that Nginx is even possible to send our two identical header keys (like Apache2). Proxy or CDN is still a possible culprit.
This also makes sense, I wasn't sure if .htaccess
was relevant since I use Nginx. I'll do some digging in the config files to see what's up.
EDIT2:
Community Docker image
Is is actually supported/intended to update a Nextcloud Docker container via admin panel? I thought, and it makes sense, to update it only with the container itself. Would be quite a logical explanation that the NC25 Docker container's Nginx config still ships the
none
header, while you need to wait for a NC26 version of the container. The same may or may not be responsible for your avatar issues, and the missing PHP module.
I've always run upgrades from the admin panel before without issue, so I thought that was the proper way to do it. I'm using the linuxserver/nextcloud
docker container which is already at 26.0.0 and is as I understand it just a repackaging of the mainline Nextcloud build with Nginx. I might ask them if it might be something that needs fixing on their end. Pulling the latest docker image did fix the PHP module error.
I can confirm the broken integrity check shortcut link. Actually the whole NC26 docs version is missing: https://docs.nextcloud.com/server/26/ This should fix it: nextcloud/documentation#9947
Thank you for the extensive reply! I appreciate it. :)
I can confirm that i had the Warning with Nginx, after upgrade from 25.0.4 to 25.0.5, because I had a configuration file setting the header to
none
.After changing that file to
noindex, nofollow
warning is gone.
Thank you for the reply, I went and changed this in the default.conf
file and the error went away. Closing this issue since the main issue is resolved, I believe. I'll contact the linuxserver/nextcloud
maintainers about the conf file.
If this is a Docker container, I guess using the Nextcloud updater is not intended. If I'm not mistaken, it is possible to disable it via config.php
, which would then make sense for this Docker image. So any Nextcloud update would come via container update, which allows the maintainers as well to ship updated Nginx configs and such when required.
@MichaIng The Nextcloud updater is listed as the first option in the container readme, with the caveat that the latest image should be pulled first. I spoke to one of the linuxserver/nextcloud
maintainers on Discord, and they said that it is preferable to use the updater.phar command they provide because it is interactive and shows the full update log.
It is however quite an uncommon way of using Docker containers and breaks major benefits/intentions of using them. One of the major points of using Docker containers is that you have a fixed setup which is precisely composed so that all components are assured to work with each other, in this case database, PHP, webserver and web application, possible Redis server and others. Containers are usually not designed/intended to be manipulated by accessing the internal console and change the system. The same way it is usually not intended to update software within the container, but only to update the whole container image and use it like that until a new updated image is available. Also, not only are all your customisations lost with a container upgrade, also the software (Nextcloud) update could be reverted if a new container does not yet ship the latest software version.
Sticking with only container updates would have prevented all your tree issues + the missing sysvsem
module, since all this will be addressed with a new container image when Nextcloud 26 is included. Although not sure where the inverted avatar is coming from, which may be a NC26 bug, if the false core/js/mimetypelist.js
is not the reason.
I can confirm that i had the Warning with Nginx, after upgrade from 25.0.4 to 25.0.5, because I had a configuration file setting the header to
none
.After changing that file to
noindex, nofollow
warning is gone.
I updated from 25.0.4 to 25.0.5 and also got that warning, which is a bug in itself, as "none" (which is what I had) is equivalent to "noindex, nofollow" (see https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag), to Nextcloud should not warn if the header value is "none".
~Can you please read through this thread and answer yourself that "none" is NOT equivalent to "noindex, nofollow"?~ EDIT: Sorry, I mixed up the issues.
I had to find the relevant commit (https://github.com/nextcloud/server/commit/5f90b8eb118324627d5845e2a7a6fa8613bf4579) to know the rationale for this change, which I could not find in this thread here (which only mentions the issue of having duplicate headers).
So OK.
Ah sorry, I got confused with two other issues about this: #37355 and #37386
As an admin, if you want to stay informed about changes like this, I recommend to subscribe to #34692 and its follow-ups #37039, which are also pinned to the top of the issues page.
for me it is solved next cloud plugins the truenas core I changed from "none" to "noindex, nofollow" in the file /mnt/Truenas/iocage/jails/nextcloud/root/usr/local/etc/nginx/conf.d/nextcloud.inc
Right, compare with latest docs as well: https://docs.nextcloud.com/server/latest/admin_manual/installation/nginx.html
changing that file to
noindex, nofollow
warning is gone.
where do i have to set this?
Depends on your webserver and config structure. The same place where you set the X-Robots-Tag: none
before, replacing it.
NC runs at a Debian Container Apache Webserver behind an other container running Nginx Reverse Proxy manager.
With Apache webserver, you just need to take care that NPM forwards the X-
headers to clients. The .htaccess
shipped with Nextcloud sets the header correctly (of course).
I assume that this Apache webserver generally reads .htaccess
files, as this is the default when installing it on Debian from their APT repo.
changing that file to
noindex, nofollow
warning is gone.
For nginx the new setting from the documentation didn't help. I changed my /etc/nginx/sites-enabled/....conf (where ... is my website):
- add_header X-Robots-Tag "none" always;
+ # add_header X-Robots-Tag "none" always;
+ add_header X-Robots-Tag "noindex, nofollow" always;
Then:
systemctl reload nginx
systemctl restart php8.1-fpm
(I ensured that is the version used in /etc/nginx/conf.d/10-fpm.conf
)systemctl restart nginx
Version: Nextcloud Hub 6 (27.1.0)
Verify that the header really is set, e.g. like
curl -I 127.0.0.1
Verify that the header really is set, e.g. like
curl -I 127.0.0.1
yes:
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Thu, 21 Sep 2023 00:03:29 GMT
Content-Type: text/html
Content-Length: 5779
Last-Modified: Sun, 25 Jul 2021 22:11:39 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "60fde19b-1693"
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: noindex, nofollow
X-XSS-Protection: 1; mode=block
Accept-Ranges: bytes
This is not the Nextcloud instance, is it? Is it within a sub directory, like
curl -IL 127.0.0.1/nextcloud
This is not the Nextcloud instance, is it? Is it within a sub directory, like
curl -IL 127.0.0.1/nextcloud
Ugh good point. The setting isn't showing up if I put /nextcloud in the URL after curl -I
. I put the setting in the location / {
section by mistake. Nginx is so painful...
If I put it in the
location ^~ /nextcloud {
section it is fixed now (Someone make an Nginx configuration GUI, and not just a text editor). Thank you.
This is one of the things I do not like about Nginx: Once a location/if/... block contains any add_header
directory, none of the parent block add_header
directives are added anymore. So naturally it can lead to a lot of repetitiv add_header
directives, e.g. when you adjust browser caching per location/file/mime types etc: https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header
There could be several
add_header
directives. These directives are inherited from the previous configuration level if and only if there are noadd_header
directives defined on the current level.
⚠️ This issue respects the following points: ⚠️
Bug description
I updated Nextcloud Server to 26.0.0 today, and faced some odd issues. My setup went from no warnings to this:
I checked both the
.htaccess
file and the actual HTTP headers. Firefox reports two X-Robots-Tag headers, one of which reads "none" and one of which reads "noindex, nofollow", so the warning is correct about one header but incorrect overall. I'm not sure where the "none" comes from, though.When I click the "documentation ↗" link, it takes me to this page which simply shows a "File not found." message in black on white. The same thing happens with the "installation guide ↗" link, here.
Apart from this, after upgrading to 26.0.0 I also noticed that my user avatar had its colours inverted, but only in the top right corner, which seems incredibly odd. Image for reference, showing what the image looks like on my user profile (which renders it properly):
Steps to reproduce
Expected behavior
Installation method
Community Docker image
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.0
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Updated to a major version (ex. 22.2.3 to 23.0.1)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
No response
Additional info
I installed Nextcloud using the
linuxserver/nextcloud
docker image, but I don't believe this should be relevant to the problem.