Closed WNYmathGuy closed 5 years ago
GitMate.io thinks possibly related issues are https://github.com/nextcloud/server/issues/3833 (files have failed the integrity check), https://github.com/nextcloud/server/issues/12051 (There were problems with the code integrity check. More information...), https://github.com/nextcloud/server/issues/115 (.user.ini not passed the integrity check), https://github.com/nextcloud/server/issues/7233 (Contacts List Issue Not Resolved Sensibly), and https://github.com/nextcloud/server/issues/1366 (Integrity Check: add files/change files).
I'm not sure why you created a new issue but at least you closed the old one :+1:
Could you post the content of .htaccess
here? It's possible to disable code integrity check but not recommend https://help.nextcloud.com/t/disable-code-integrity-check/31289
Having the same issue... this is idiotic. Let me reign over my own .htaccess and .user.ini! They are not (really) part of Nextcloud, but of my damn webserver, which I configure as I wish.
Please exclude those files from the integrity check. This is overkill.
@Chrisly3ear @WNYmathGuy is this not working for you (everything you put below #### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####
is ignored - only for .htaccess
)?
@danielkesselberg No, unfortunately this doesn't work for me, because I'm using Nginx with php-fpm. Therefore I have to use a ".user.ini" instead of a ".htaccess" file. This is straight from the documentation (https://docs.nextcloud.com/server/13/admin_manual/configuration_files/big_file_upload_configuration.html#configuring-your-web-server)
So, if I change the php configuration using the ".user.ini" file I get the "signature check failed" error. This is rather annoying, so I had to turn the whole code sigining thing off via config.php.
I'm all for keeping the base system tamper proof, but not at the price of no custom webserver configs.
Chris
Index: lib/private/IntegrityCheck/Checker.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- lib/private/IntegrityCheck/Checker.php (revision 35251928d57092a50529da8d439c0de3cf428298)
+++ lib/private/IntegrityCheck/Checker.php (date 1542662534000)
@@ -226,6 +226,19 @@
}
}
+ // The .user.ini file in the root folder of nextcloud can contain
+ // custom content. Thus we ignore everything below the first
+ // occurrence of "; DO NOT CHANGE ANYTHING ABOVE THIS LINE"
+ // and have the hash generated based on this.
+ if($filename === $this->environmentHelper->getServerRoot() . '/.user.ini') {
+ $fileContent = file_get_contents($tmpFolder . '/.user.ini');
+ $explodedArray = explode('; DO NOT CHANGE ANYTHING ABOVE THIS LINE', $fileContent);
+ if(\count($explodedArray) === 2) {
+ $hashes[$relativeFileName] = hash('sha512', $explodedArray[0]);
+ continue;
+ }
+ }
+
$hashes[$relativeFileName] = hash_file('sha512', $filename);
}
You could try the patch above (but please backup lib/private/IntegrityCheck/Checker.php before). Then use the default .user.ini
from nextcloud and add
; DO NOT CHANGE ANYTHING ABOVE THIS LINE
your custom php settings here
Edit: Made a mistake. You need to remove the statement below after applying the patch.
@danielkesselberg the one in the config
folder has:
# line below if for Apache 2.4
<ifModule mod_authz_core.c>
Require all denied
</ifModule>
# line below if for Apache 2.2
<ifModule !mod_authz_core.c>
deny from all
</ifModule>
# section for Apache 2.2 and 2.4
<ifModule mod_autoindex.c>
IndexIgnore *
</ifModule>
The one in the web root folder has:
<IfModule mod_headers.c>
<IfModule mod_setenvif.c>
<IfModule mod_fcgid.c>
SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
</IfModule>
<IfModule mod_proxy_fcgi.c>
SetEnvIfNoCase Authorization "(.+)" HTTP_AUTHORIZATION=$1
</IfModule>
</IfModule>
<IfModule mod_env.c>
# Add security and privacy related headers
Header set X-Content-Type-Options "nosniff"
Header set X-XSS-Protection "1; mode=block"
Header set X-Robots-Tag "none"
Header set X-Download-Options "noopen"
Header set X-Permitted-Cross-Domain-Policies "none"
Header set Referrer-Policy "strict-origin-when-cross-origin"
SetEnv modHeadersAvailable true
</IfModule>
# Add cache control for static resources
<FilesMatch "\.(css|js|svg|gif)$">
Header set Cache-Control "max-age=15778463"
</FilesMatch>
# Let browsers cache WOFF files for a week
<FilesMatch "\.woff$">
Header set Cache-Control "max-age=604800"
</FilesMatch>
</IfModule>
<IfModule mod_php5.c>
php_value upload_max_filesize 511M
php_value post_max_size 511M
php_value memory_limit 512M
php_value mbstring.func_overload 0
php_value always_populate_raw_post_data -1
php_value default_charset 'UTF-8'
php_value output_buffering 0
<IfModule mod_env.c>
SetEnv htaccessWorking true
</IfModule>
</IfModule>
<IfModule mod_php7.c>
php_value upload_max_filesize 511M
php_value post_max_size 511M
php_value memory_limit 512M
php_value mbstring.func_overload 0
php_value default_charset 'UTF-8'
php_value output_buffering 0
<IfModule mod_env.c>
SetEnv htaccessWorking true
</IfModule>
</IfModule>
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} DavClnt
RewriteRule ^$ /remote.php/webdav/ [L,R=302]
RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]
RewriteRule ^\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
RewriteRule ^remote/(.*) remote.php [QSA,L]
RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
RewriteRule ^(?:\.|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>
<IfModule mod_mime.c>
AddType image/svg+xml svg svgz
AddEncoding gzip svgz
</IfModule>
<IfModule mod_dir.c>
DirectoryIndex index.php index.html
</IfModule>
AddDefaultCharset utf-8
Options -Indexes
<IfModule pagespeed_module>
ModPagespeed Off
</IfModule>
#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####
ErrorDocument 403 /
ErrorDocument 404 /
and the one in .well-known/acme-challenge
has:
AuthType None
Require all granted
Satisfy any
The nextcloud-bot caught me on the old post that I forgot about and the old post seemed less "on the nose" than it could have been so I closed the old one.
I agree that the code integrity check should not be disabled. The lies we tell ourselves are eventually catastrophic.
Index: lib/private/IntegrityCheck/Checker.php IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- lib/private/IntegrityCheck/Checker.php (revision 35251928d57092a50529da8d439c0de3cf428298) +++ lib/private/IntegrityCheck/Checker.php (date 1542662534000) @@ -226,6 +226,19 @@ } } + // The .user.ini file in the root folder of nextcloud can contain + // custom content. Thus we ignore everything below the first + // occurrence of "; DO NOT CHANGE ANYTHING ABOVE THIS LINE" + // and have the hash generated based on this. + if($filename === $this->environmentHelper->getServerRoot() . '/.user.ini') { + $fileContent = file_get_contents($tmpFolder . '/.user.ini'); + $explodedArray = explode('; DO NOT CHANGE ANYTHING ABOVE THIS LINE', $fileContent); + if(\count($explodedArray) === 2) { + $hashes[$relativeFileName] = hash('sha512', $explodedArray[0]); + continue; + } + } + $hashes[$relativeFileName] = hash_file('sha512', $filename); }
You could try the patch above (but please backup lib/private/IntegrityCheck/Checker.php before). Then use the default
.user.ini
from nextcloud and add; DO NOT CHANGE ANYTHING ABOVE THIS LINE your custom php settings here
Edit: Made a mistake. You need to remove the statement below after applying the patch.
server/lib/private/IntegrityCheck/Checker.php
Lines 205 to 211 in 72e745b
// The .user.ini file can contain custom modifications to the file size // as well. if($filename === $this->environmentHelper->getServerRoot() . '/.user.ini') { $fileContent = file_get_contents($tmpFolder . '/.user.ini'); $hashes[$relativeFileName] = hash('sha512', $fileContent); continue; }
That looks amazing, but I don't understand it. :disappointed:
php occ maintenance:update:htaccess
php occ integrity:check-core
When you first run update:htaccess
than should integrity:check-core
pass?
@danielkesselberg at the shell prompt:
$ cd domains/cloud.mydomain.com/public_html/nextcloud/
$ php occ maintenance:update:htaccess
.htaccess has been updated
$ php occ integrity:check-core
- INVALID_HASH:
- .htaccess:
- expected: 70183f641f39fc5f43065aeb15b86ac99e86b82310d5467e1c4d935df30acc6155c0905f84f59616885275c19ffacea59e354a87a5c97c94f50d50d4eff0370e
- current: e05ca07248d85fbaa8d0b033257dec1e62a1308462eba2e21a57eb574e2c4e41af3872d372f5d835f659f943010c4bceb7f3c8113b11eb65ff730e8a30f76a93
- EXTRA_FILE:
- .well-known/acme-challenge/.htaccess:
- expected:
- current: d1c54aa5adc100187bee69c06d79c6a9a54dc6338c398c21c8dd48c8fbdfd72a2f2ca73433ae5bf4255a61c6d2806ba8ba3fff12d3e677273345eea85ed47bc5
$
Yesterday I moved the .well-known/
folder back in.
Just as reference https://github.com/nextcloud/server/issues/663 I never noticed a .htaccess inside .well-known/acme-challange/. Does letsencrypt renew work when you remove it?
The hash of .htaccess does not match because this line is not expected.
Header set Referrer-Policy "strict-origin-when-cross-origin"
I know there is warning when Referrer-Policy is not set. For nextcloud 14.0.3 something like below should be added after #### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####
<IfModule mod_headers.c>
<IfModule mod_env.c>
# Add security and privacy related headers
Header set Referrer-Policy "strict-origin-when-cross-origin"
</IfModule>
</IfModule>
Starting with nextcloud 14.0.4 (not released, testing right now) Referrer-Policy is set by default (https://github.com/nextcloud/server/pull/11798).
Obviously I was wrong about php occ maintenance:update:htaccess
. https://github.com/nextcloud/server/blob/14f7b2c46633b7a21656872ebf6d809398a342e4/lib/private/Setup.php#L492 only the part below #### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####
is replaced when you call php occ maintenance:update:htaccess
:see_no_evil:
Just as reference #663 I never noticed a .htaccess inside .well-known/acme-challange/. Does letsencrypt renew work when you remove it?
It might be something that Virtualmin does when it manages SSL on virtual servers. It's something I see in all my SSL enabled domains.
Okay, @danielkesselberg just got back to it.
I moved the .well-known
folder back out of nextcloud's space. Here is the current state of my web root .htaccess
file:
I then went into the admin web interface .com/index.php/settings/admin/overview
and clicked the red "Rescan" link. Here are the results:
I wish I understood the in's and out's of these hash passings. How can I upgrade Nextcloud with Nextcloud from Nextcloud and have bad files? Do you know a link to some knowledge of what is going on with the hash passing and checking? I don't like to be ignorant but in this case I don't see the door out of the cave.
I really wish I would have thought of using the details, summary tags earlier. Cleaner dialogue.
# Header set Referrer-Policy "strict-origin-when-cross-origin"
delete this line. Hash is calculated from file content. If any character is not the same the hash does not match.
:1st_place_medal: hashtag-FTW! It makes sense now. I had no idea that the check was making a hash based on the current file contents. It feels so obvious it's almost like when I found out corn seed was the part of the corn plant that we eat.
The reason I'm really here...
PLEASE FOR THE LOVE OF RANDOM VARIABLES JUST TELL ME WHERE THE FUCK I CAN PUT THE RIGHT CODE THAT WILL MAKE THIS GO AWAY!?!?!?!?!?!
Where does the new current hash go? Can't I just paste the expected into a file somewhere? Or go backwards and paste the [current] into the file I see the [expected] hash?
Steps to reproduce
Expected behaviour
The installer installs without mucking up things. Maybe it moves offensive files and folders out, and then back in after it's done?
Actual behaviour
Described in "Steps to reproduce" section above.
Server configuration detail
Operating system: Linux 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64
Webserver: Apache/2.4.18 (cgi-fcgi)
Database: mysql 10.0.36
PHP version:
7.0.32-0ubuntu0.16.04.1 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, pcntl, Reflection, SPL, session, standard, cgi-fcgi, mysqlnd, PDO, xml, apcu, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, imap, intl, json, ldap, exif, mcrypt, mysqli, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, readline, redis, shmop, SimpleXML, soap, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache
Nextcloud version: 14.0.3 - 14.0.3.0
Updated from an older Nextcloud/ownCloud or fresh install: Updated from 13 to 14
Where did you install Nextcloud from: Nextcloud
Signing status
Array ( [core] => Array ( [INVALID_HASH] => Array ( [.htaccess] => Array ( [expected] => 70183f641f39fc5f43065aeb15b86ac99e86b82310d5467e1c4d935df30acc6155c0905f84f59616885275c19ffacea59e354a87a5c97c94f50d50d4eff0370e [current] => e05ca07248d85fbaa8d0b033257dec1e62a1308462eba2e21a57eb574e2c4e41af3872d372f5d835f659f943010c4bceb7f3c8113b11eb65ff730e8a30f76a93 ) ) ) )List of activated apps
``` Enabled: - accessibility: 1.0.1 - activity: 2.7.0 - admin_audit: 1.4.0 - admin_notifications: 1.0.2 - announcementcenter: 3.3.1 - apporder: 0.5.0 - audioplayer: 2.4.1 - bruteforcesettings: 1.2.0 - calendar: 1.6.3 - carnet: 0.9.0 - checksum: 0.4.1 - cloud_federation_api: 0.0.1 - cms_pico: 0.9.7 - comments: 1.4.0 - contacts: 2.1.7 - data_request: 1.1.0 - dav: 1.6.0 - deck: 0.5.0 - dicomviewer: 1.0.2 - drawio: 0.9.1 - drop_account: 0.0.13 - event_update_notification: 0.3.0 - federatedfilesharing: 1.4.0 - federation: 1.4.0 - files: 1.9.0 - files_accesscontrol: 1.4.0 - files_automatedtagging: 1.4.0 - files_markdown: 2.0.5 - files_pdfviewer: 1.3.2 - files_retention: 1.3.0 - files_sharing: 1.6.2 - files_texteditor: 2.6.0 - files_trashbin: 1.4.1 - files_versions: 1.7.1 - files_videoplayer: 1.3.0 - firstrunwizard: 2.3.0 - flowupload: 0.0.8 - gallery: 18.1.0 - gpxedit: 0.0.9 - gpxmotion: 0.0.7 - gpxpod: 2.3.2 - groupfolders: 1.3.3 - issuetemplate: 0.4.0 - logreader: 2.0.0 - lookup_server_connector: 1.2.0 - metadata: 0.8.0 - mindmaps: 0.1.0 - music: 0.9.2 - nextcloud_announcements: 1.3.0 - notes: 2.5.0 - notifications: 2.2.1 - oauth2: 1.2.1 - ocsms: 2.0.2 - password_policy: 1.4.0 - phonetrack: 0.3.8 - polls: 0.8.3 - previewgenerator: 2.0.0 - provisioning_api: 1.4.0 - quota_warning: 1.3.0 - radio: 0.6.3 - rainloop: 6.0.1 - ransomware_detection: 0.4.1 - ransomware_protection: 1.2.0 - registration: 0.4.5 - richdocuments: 3.0.5 - serverinfo: 1.4.0 - sharebymail: 1.4.0 - socialsharing_diaspora: 1.0.3 - socialsharing_email: 1.0.4 - socialsharing_facebook: 1.0.3 - socialsharing_googleplus: 1.0.3 - socialsharing_twitter: 1.0.3 - spreed: 4.0.1 - support: 1.0.0 - survey_client: 1.2.0 - systemtags: 1.4.0 - tasks: 0.9.7 - telephoneprovider: 1.0.1 - terms_of_service: 1.0.3 - theming: 1.5.0 - theming_customcss: 1.1.0 - twofactor_backupcodes: 1.3.1 - updatenotification: 1.4.1 - weather: 1.5.4 - workflowengine: 1.4.0 Disabled: - bookmarks - circles - encryption - files_antivirus - files_downloadactivity - files_external - user_external - user_ldap ```Configuration (config/config.php)
``` { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "cloud.mydomain.com" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "14.0.3.0", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpauth": 1, "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "maintenance": false, "memcache.local": "\\OC\\Memcache\\APCu", "theme": "", "loglevel": 0, "default_language": "en", "enable_previews": true, "preview_max_x": 2048, "preview_max_y": 2048, "preview_max_scale_factor": 10, "preview_max_filesize_image": 50, "preview_libreoffice_path": "\/usr\/bin\/libreoffice", "preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir ", "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\SVG", "OC\\Preview\\TIFF", "OC\\Preview\\MP3", "OC\\Preview\\TXT", "OC\\Preview\\Font", "OC\\Preview\\MarkDown", "OC\\Preview\\Movie", "OC\\Preview\\MSOffice2003", "OC\\Preview\\MSOffice2007", "OC\\Preview\\MSOfficeDoc", "OC\\Preview\\OpenDocument", "OC\\Preview\\PDF", "OC\\Preview\\Postscript", "OC\\Preview\\StarOffice" ], "tempdirectory": "\/home\/adminuser\/domains\/cloud.mydomain.com\/tmp\/nextcloudtemp", "overwrite.cli.url": "https:\/\/cloud.mydomain.com", "updater.release.channel": "stable" } ```Are you using external storage, if yes which one:
Are you using encryption: true
Are you using an external user-backend, if yes which one:
Client configuration
Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Operating system: Ubuntu 18.04 Workstation