search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.12k stars 4.03k forks source link

modified .user.ini integrity check fails #19935

Open domrim opened 4 years ago

domrim commented 4 years ago

How to use GitHub

Steps to reproduce

  1. Install Nextcloud 18.0.2 with nginx and php-fpm7.3 as webserver/php-runner
  2. modify .user.ini with own custom settings, as described in https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html for example, my new .user.ini: 
    
mbstring.func_overload=0
always_populate_raw_post_data=-1
default_charset='UTF-8'
output_buffering=0

upload_max_filesize = 10G post_max_size = 10G memory_limit = 512M max_execution_time = 300


### Expected behaviour
`occ integrity:check-core` should succeed if i don't modify any predefined values.

### Actual behaviour
`occ integrity:check-core` fails with:

I know this was an issue before (#115) so i thought this should work, or am i missing something?

Server configuration

Operating system: Debian 10 stable Web server: nginx 1.14.2 Database: postgresql 11.6 PHP version: php-fpm Nextcloud version: (see Nextcloud admin page) 18.0.2.2 Updated from an older Nextcloud/ownCloud or fresh install: Updated from 18.0.1. But is also reproducible on a new installation. Where did you install Nextcloud from: Nextcloud website download Signing status:

Signing status ``` Technical information ===================== The following list covers which files have failed the integrity check. Please read the previous linked documentation to learn more about the errors and how to fix them. Results ======= - core - INVALID_HASH - .user.ini Raw output ========== Array ( [core] => Array ( [INVALID_HASH] => Array ( [.user.ini] => Array ( [expected] => 4843b3217e91f8536cb9b52700efb20300290292cf6286f92794d4cec99df286afeb7dd6c91b1be20bc55eda541eef230a5c5e7dcd46c189edd0ed1e80c6d3f5 [current] => c2b18102f728f15c56050a6d12e1aed9e4e4b2ad2be4864cbe22f919837698ad785051feb62275098effa6e99952a3ae44dce88496050064657a0f23cb4876f1 ) ) ) ) ```

List of activated apps:

App list ``` Enabled: - accessibility: 1.4.0 - activity: 2.11.0 - bruteforcesettings: 1.5.0 - calendar: 2.0.2 - camerarawpreviews: 0.7.3 - checksum: 0.4.4 - cloud_federation_api: 1.1.0 - comments: 1.8.0 - contacts: 3.2.0 - dav: 1.14.0 - documentserver_community: 0.1.5 - federatedfilesharing: 1.8.0 - federation: 1.8.0 - files: 1.13.1 - files_pdfviewer: 1.7.0 - files_rightclick: 0.15.2 - files_sharing: 1.10.1 - files_trashbin: 1.8.0 - files_versions: 1.11.0 - files_videoplayer: 1.7.0 - firstrunwizard: 2.7.0 - logreader: 2.3.0 - lookup_server_connector: 1.6.0 - nextcloud_announcements: 1.7.0 - notifications: 2.6.0 - oauth2: 1.6.0 - onlyoffice: 4.1.4 - password_policy: 1.8.0 - photos: 1.0.0 - privacy: 1.2.0 - provisioning_api: 1.8.0 - recommendations: 0.6.0 - serverinfo: 1.8.0 - settings: 1.0.0 - sharebymail: 1.8.0 - support: 1.1.0 - survey_client: 1.6.0 - systemtags: 1.8.0 - tasks: 0.12.1 - text: 2.0.0 - theming: 1.9.0 - twofactor_backupcodes: 1.7.0 - updatenotification: 1.8.0 - viewer: 1.2.0 - workflowengine: 2.0.0 Disabled: - admin_audit - encryption - files_external - user_ldap ```

Nextcloud configuration:

Config report ``` { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "cloud.drimpf.de" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "pgsql", "version": "18.0.2.2", "overwrite.cli.url": "https:\/\/cloud.drimpf.de", "htaccess.RewriteBase": "\/", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "skeletondirectory": "", "memcache.local": "\\OC\\Memcache\\APCu", "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "dbindex": 1 }, "maintenance": false, "loglevel": 2, "mail_smtpmode": "sendmail", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpsecure": "tls", "mail_smtpauth": 1, "mail_smtpauthtype": "LOGIN", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "theme": "", "updater.release.channel": "beta", "updater.secret": "***REMOVED SENSITIVE VALUE***" } } ```

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Firefox 73.0.1 Operating system: macOS 10.15.3 (19D76)

Logs

Web server error log

Web server error log ``` empty ```

Nextcloud log (data/nextcloud.log)

Nextcloud log ``` empty ```
kesselb commented 4 years ago

Changing .user.ini is not supported anymore: https://github.com/nextcloud/server/pull/14430. Use the appropriate place to configure that limit. Good to know: Nextcloud's clients use chunking to upload files anyway. Such high limits are not required. Only if you plan to upload files with 3rdparty dav clients.

dartcafe commented 4 years ago

Such high limits are not required. Only if you plan to upload files with 3rdparty dav clients.

At least it is required for the installation of the community document server. Otherwise it fails.

domrim commented 4 years ago

also using the web-updater is critical, because the backuping step can take longer than 30 secs. and i don't want to adjust the settings for all fpm-runners.

kesselb commented 4 years ago

You can adjust those settings for each fpm pool.

cc @nextcloud/server-triage opinions? I'm fine with the current state. We might update the documentation because it's a bit unclear.

domrim commented 4 years ago

ah, i didn't read the php documentation completely 🙈 i'm fine with the current state, maybe the php-docu page could be linked?

skjnldsv commented 4 years ago

If this issue is opened, it probably means we can improve the doc :) Please provide any desired change to https://github.com/nextcloud/documentation so we can have a look and review :)

dartcafe commented 4 years ago

Such high limits are not required. Only if you plan to upload files with 3rdparty dav clients.

At least it is required for the installation of the community document server. Otherwise it fails.

Here I was referring to the memory_limit. It must be raised to 512M especially with some share hosters.

skjnldsv commented 4 years ago

Which we already recommend, no? Closing then?

dartcafe commented 4 years ago

The suggested solutions are not satisfying for users without administrative access to the machine or OS.

I have a few instances running on shared webspace (not changeable php memory limit 256M) . Everytime I update NC I have to edit the user.ini or .htaccess (adding php_value memory_limit 512M) in the middle of the update process to prevent the update from failing. In the result, the integrity scan fails.

After that the integrity check fails.

kesselb commented 4 years ago

https://github.com/nextcloud/server/issues/18655 is about the memory limit. I agree as long term goal we should not longer ship a .user.ini.

I think we can ship the .user.ini as .user.ini.example and exclude the .user.ini file from the integrity check. Maybe a small patch for the updater is needed to keep (or copy) a .user.ini during the upgrade.

dartcafe commented 4 years ago

I think we can ship the .user.ini as .user.ini.example and exclude the .user.ini file from the integrity check.

Sounds like a good solution to me. So the .htaccess file can stay untouched and users don't have to edit config files after updates.

nemihome commented 1 year ago

Well this issue is 2,5 years old. But the problem seems to be the same with latest version. Furthermore I don't thing that creating a fpm-pool for every single web app (and that's the result of the purposal which is mentioned above) on a server is something which is a good use of resources. Furthermore there are global settings which can't be set for a pool.

kesselb commented 1 year ago

Well this issue is two years old. But the problem seems to be the same with latest version.

True. I'm afraid nobody had time to work on this.

Furthermore I don't thing that creating a fpm-pool for every single web app (and that's the result of the purposal which is mentioned above) on a server is something which is a good use of resources.

I think you overestimate the resource usage for another fpm pool. To configure one pool to serve three web apps or three pools to serve one web app shouldn't be much difference. Would you mind doing some tests and share the results with us?

Furthermore there are global settings which can't be set for a pool.

False.

nemihome commented 1 year ago

I already have a php pool for nextcloud for security reasons and it takes more ressouces than before with one single pool for all apps. Several pools are consuming more memory, running more processes and cumsuming more CPU power and overal it's more inefficient to run so many pools because you set limits on pool basis (e.g. dynamic mode, max, min server) and that means less ressources if one single application does need it because you have to create you combined pool setups in a way that the server will run with heavy load on all pools. You have to decide for every pool what ressources are assigned and if every app would request that model (which is nearly impossible without experience).

Continuing that way you are creating a pool for roundcube, a pool for rainloop, a pool for wordpress a pool for wordpresss staging, phpmyadmin, a pool for every url hosted on a server and basically for every single app running on the URLs. If you have 3 php apps per URL and 10 urls you have 30 pools as a result.

As far as I know the user.ini is the best way to limit the pool to folders because otherwise the global settings are used and they have to be set for all folders.

E.g. open_basedir is set in php.ini (global setting) open_basedir = "/tmp/:/var/nextcloud:/usr/share/php:/usr/share/phpmyadmin:/usr/share/phpmyadminkeys:/var/www/html/URL2:/var/www/html/URL3:/var/www/html/URL4:/var/www/html/URL5:/var/www/html/URL6:/var/www/html/URL7:/var/www/html/URL8:/var/www/html/URL:/var/www/html/URL9:/var/www/html/URL10"

In user.ini for nextcloud: open_basedir="/usr/share/php/:/var/www/html/URL/nextcloud:/var/nextcloud:/var/www/html/URL/.well-known"

Furthermore there are other settings which nextcloud requests or requires which make no sense a global setting in php.ini. For me this looks like nextcloud should run on a single physical server based on the requirements. But nextcloud is only just one app on the server.

kesselb commented 1 year ago

I already have a php pool for nextcloud for security reasons

Good choice!

that means less ressources if one single application does need it because

Try ondemand (https://community.webcore.cloud/tutorials/php_fpm_ondemand_process_manager_vs_dynamic/)

In user.ini for nextcloud: open_basedir="/usr/share/php/:/var/www/html/URL/nextcloud:/var/nextcloud:/var/www/html/URL/.well-known"

It's possible to define open_basedir in the pool configuration. Open https://www.php.net/manual/en/install.fpm.configuration.php and look for Example #1.

Thanks for your extensive feedback. It may help to prioritise this issue.

nemihome commented 1 year ago

It's possible to define open_basedir in the pool configuration. Open https://www.php.net/manual/en/install.fpm.configuration.php and look for Example #1.

Thank you. I checked that documentation but only searched for the list of pool directives and did not see the example. So that should solve the problem for Nextcloud. But you still need a pool for every web app which behaves the same way.

I will check that out.

szaimen commented 1 year ago

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

nemihome commented 1 year ago

Hello,

Thank you.

I updated to 25.03 via 24.09 and did remove the " 'integrity.check.disabled' => true" setting before. I did not run into any problems.

szaimen commented 1 year ago

Thanks for verifying!

dartcafe commented 1 year ago

@szaimen Closing this issue was maybe a little bit too fast, since a changed user.ini still fails the integretiy check and gets overwritten with every update (#23679).

This is still a very annoying thing, because the settings have to get reset after every update.

szaimen commented 1 year ago

Fine by me to open it again. On which version is this happening?

dartcafe commented 1 year ago

Still on 25.0.3. Checked minutes ago.

szaimen commented 1 year ago

Thanks!

nemihome commented 1 year ago

I can confirm that. But the original bug which I reported was because of integrity check. I did not run into that. The user.ini is not only overwritten, also the owner is changed. I have e.g. another owner (not the user nextcloud is running with).

dartcafe commented 1 year ago

occ integrity:check-core triggers the message.

makoehr commented 1 year ago

I have nextcloud installed with a web hoster, I have no access to php.ini, that's why I have to adapt .user.ini to increase the memory_limit to more than 256M.

With nextcloud version 25.0.5 I run into integrity check issues when I modify the .user.ini. I highly recommend to adapt the integrity check (or leave this files out) for this very file as it is mandatory to modify this file under some circumstances.

asheroto commented 1 year ago

Found this after searching for why .user.ini keeps getting overwritten.

Although the file integrity check can be adapted to workaround this issue, ideally there needs to be an option that enables a user to have a customized .user.ini file as well as .htaccess file.

I updated to Nextcloud 26 awhile back, and had .user.ini setup up. A few weeks back I performed an upgrade to a minor Nextcloud version, then today upgraded to Nextcloud 27. Ran into an issue on Step 5 of the web updater, mentioned here also mentioned here. Realized it was because my PHP memory limit was not set, even though I had the .user.ini setup in my versions prior to the last minor version upgraded.

I copied .user.ini file from a backup I had and completed the update, but then my .user.ini file was not kept again. 😊

Is there an option to keep those files?