DietPi-Software | NextCloudPi convergence

I just saw this thread comparing the two projects and see no reason you could not further integrate NextcloudPi to improve Nextcloud integration and deployment in DietPi.

The NextcloudPi curl script installation or docker image could be easily modified to give you best of both worlds. This would allow you to integrate the ncp-config and ncp-web tools to more easily run OCC commands, or just run your particular Nextcloud scripts from an attractive terminal / SSH / web interface. @nachoparker is the friendly lead dev.

@sunjam Thanks for your idea.

Indeed I was also thinking about this already. Generally you should be able to run NCP install script on any DietPi system, although I didn't try it so far, maybe a task for the evening.

But as far as I can think of, the way, Nextcloud is implemented, it should break other web ui installations via dietpi-software. We heavily take care, that our implementations are as modular as possible and never do core changes/block core paths (e.g. webroot), so you can install/run everything in parallel.
Also many system- and Nextcloud configuration options should be doubled with NCP, including data directory, possible backups, HTTPS and much more, I guess, and adjusting via NCP web ui might break the DietPi way and the other way round.

But these are assumptions so far and what I remember from visiting NCP features and code the last time 😉, as said, I need to test.

The docker image of course would work, but if you are already on a SBC and DietPi actively integrates several software besides each other, so that you don't need to run them in separate containers, these would be a large overhead and not optimal, especially performance/resource wise. Generally I am no big friend of containers. For me they should be considered as last resort, if you don't have to take care resource usage and/or need an extremely easy to maintain and stable instance, while not taking care too much about regular updates and/or individual configuration...

To integrate NCP nicely it surely would need some adjustment to reduce/avoid doubled settings and probably automate/set-fix some settings of NCP to match DietPi and/or the other way round.

Let me mark this for investigation. I am surely interested how it works out. Maybe @nachoparker has/had already some thoughts/opinion about this as well?

Thanks both,

I am in favour of two people looking and maintaining one code base nicely rather than two people poorly maintaining two codebases that do the same thing. I think it leads to superior code quality, less duplication of efforts, and safer implementation due to more eyes looking at it.

I would be happy to help, time permitting to make NCP even less intrusive if needed.

But as far as I can think of, the way, Nextcloud is implemented, it should break other web ui installations via dietpi-software. We heavily take care, that our implementations are as modular as possible and never do core changes/block core paths (e.g. webroot), so you can install/run everything in parallel.

NCP does a couple modifications to php and mysql only. Nothing that would be too hard to isolate a bit better. Some things like the tmp dir (dietpi leaves it in the SD card, NCP moves it along with the datadir to a USB drive if that option is activated) don't seem easy to completely decouple, neither for dietpi or for NCP.

Of course if there are more web services there is going to be a port clash, but that problem has nothing to do with NCP, because it will happen with any web based services that dietpi is going to install. I don't know how people deal with this in dietpi.

Armbian just included NCP in their software selection (not yet on master), so there's interest to keep things separate, independently of what happens with dietpi

ncp-armbian-tool

The docker image of course would work, but if you are already on a SBC and DietPi actively integrates several software besides each other, so that you don't need to run them in separate containers, these would be a large overhead and not optimal, especially performance/resource wise. Generally I am no big friend of containers. For me they should be considered as last resort, if you don't have to take care resource usage and/or need an extremely easy to maintain and stable instance, while not taking care too much about regular updates and/or individual configuration...

I don't think the docker image fits nicely with dietPi. I mean it is a clean option and indeed it provides isolation, but wouldnt' it be weird if everything else is not containerized and NC is?

Aside from this, in general, I wouldn't disregard docker so quickly for your personal stuff. It is a great step forward in technology. It allows you to install complicated stacks superfast and reliably, and it does not have a performance penalty. For instance, it's isolation is perfect for situations like the one where we are at right now, with many webservices interfering each other's configurations... only updating it is a different process and has to be learnt aside from apt-get upgrade.

But that's another topic... xD

In any case, I think that first you should flash the latest NCP image and try it for yourself to understand the added value beyond NC: moving everything to an external hard drive, automount, autoupdates, auto debian upgrades, auto backups, auto SSL, DDNS support, added security, ncp-web, postfix ...

Then you can decide for yourself ;)

Ask anything

nextcloudplus.com

@nachoparker Thanks for sharing your ideas. Yes, indeed it's a question if we can get NCP installed in a way, that an existing webserver stack is adjusted with modular configuration files only and a Nextcloud-only web UI, that doesn't include otherwise doubled system settings. Simply a Nextcloud-only version. Also we allow Nginx and Lighttpd as webservers.

Let me now start with NCP installation on my 3 DietPi VMs 😃.

Off-topic about Docker

@nachoparker > docker ... it does not have a performance penalty. To be true, it was more a logic assumption of me. And if it's about runtime performance only, you seem right: https://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container#26149994 - Less abstraction than I thought 👍. - Though network seems to be slower. But of course if you run individual webservers, databases, memory caches etc. in separate containers instead of having all your software sharing one of each, you will face a large impact on RAM and disk usage and overall CPU time/usage. The single environments may be nearly as fast as being natively implemented, but the benefits of having all your software within their own environment+dependencies implicates automatically the disadvantages. Of course doubled doesn't mean doubled always, as you can optimize every environment for it's specific use case, disable/remove unneeded modules/parts and of course have just the usage of the related single software inside the container. But two highly optimized databases/webservers/... will always cost more than one that is configured to handle two web applications. Besides there are of course several forcing use cases and RAM/disk usage is not always a bottle neck, for me it's also a quite personal taste and finally ideological question. Stop reading here in case, I just love to go into philosophical discussions by times 🤣: - Having everything within it's own individual environment for me in general is kinda resignation to being able to make things work together, share their needs etc. Of course it's always "easier" to have everything have it's own, optimized in how it's needed. But in the end seen from a wider perspective, sharing goods/resources will mostly be an overall(!) benefit, at least if it's done good, which might need some additional work and thinking. Of course this can be understood in political terms as well, if humanity will be able to finally work together, sharing resources in a fair overall beneficial way, or if finally everyone is looking just onto his/her own (smaller environment), building walls/clear separations, as it's the easier way than talking and finding solutions to work together instead against/aside each other. - Finally it's a question what currently "works". On Windows I see 10 to 20 separate visual C++ implementations aside each other, as every software brings it's own specific, often very outdated version. I read that this was different in the past, causing huge issues, as Microsoft obviously was not able to keep it backwards compatible for older software titles and software developers were not able to upgrade their software based on the new C++ version. Okay it's just a "small" library here and before taking huge effort somehow guaranteeing old and new software titles work with the most current version, just allow them bring their own... - But for me, all this, same as docker, is not "step forward" in technology. It's a (indeed well done) step backwards, that make certain stings easier. A step "forward" or better: modern technology design, in my opinion, should always developing into a direction, that implies different use cases, modularity and most compatible interfaces according to open standards to allow it, or parts of it being implemented and used by different software/users in a different way in parallel. - This finally is, why I was falling in love with Linux (compared to Windows or MacOS), FOSS in general and DietPi in particular, as we are working on running different software on an open standard OS/distribution driven by again very different hardware ❤️. - On the other hand, I like to have my own server at home handle data myself, instead of sharing server capacity and cloud solutions. Yeah, it's always a question of how the implementation is done and if you want to play and tinker around as well 😉. For development/tinkering of course an isolated environment totally makes sense. I currently run 3 VMs here, ready to install NextcloudPlus 😄!

Testing protocol

On fresh DietPi Stretch VirtualBox VM:
1. dietpi-software install 76 91 # LAMP stack + Redis
2. curl -sSL https://raw.githubusercontent.com/nextcloud/nextcloudpi/master/install.sh | bash
  - LAMP stack + Redis want's to be installed again
  - Apache is installed with php-fpm, where DietPi installs it with mod-php.
  - Some Apache modules are actively disabled~~, which could cause issues with other software.~~ Some kind of Apache(+PHP) module dependency system is anyway something I have on my list, although many other things with higher priority. €: Ah found re-enable code, modules are disabled just during installation 👍.
  - Apache configured with basic HTTPS functionality (snake oil certs), although within separate configuration/vhost and no forced redirection/HSTS, this should be no problem.
  - MariaDB configuration output seems to expect root password authentication. However since Debian Stretch, unix_socket authentication is default, allowing login only when logged in as unix root user, but without the need to enter password for MariaDB login again. Compatibility with this behaviour assured? 🈯️€: Ah just found mysql_secure_installation script as provided by MariaDB. Nice to have, but on Debian Stretch not needed, as stated above: root user present only with unix_socket authentication configured, which is great for my impression. However running the script does not damage something.
  - Afterwards NCP installation itself starts, maybe this is where we can start from, which DietPi.
  - Postfix is installed. Would be nice to have modules like this optional, allowing to e.g. skip installation (and of course potential UI elements, depending on it) via command option, global variable or setup configuration file.
  - Redis is being installed at last, consider moving installation to the beginning within/between LAMP stack installation etc. and just do final Nextcloud related needed configuration within final NCP install script. This would also avoid enabling the service, which does not work in combination with DietPi, controlling the services itself.
  - 🈴 Last output: Failed to stop mysqld.service: Unit mysqld.service not loaded. The service is called mysql (init.d) or optionally mariadb (systemd). ~~Is installation aborted here or finished anyway?~~ €: Script execution broken at this stage, attempt with workarount below...
  - 🈴 Installer expects PHP 7.0 to be installed. This will break installation with default PHP version on Jessie (PHP 5) and Buster (PHP 7.2).

Whole terminal output

``` root@DietPi:~# curl -sSL https://raw.githubusercontent.com/nextcloud/nextcloudpi/master/install.sh | bash >>> WARNING: existing mysqld configuration will be changed <<< Getting build code... Ign:1 https://cdn-aws.deb.debian.org/debian stretch InRelease Hit:2 https://cdn-aws.deb.debian.org/debian stretch-updates InRelease Hit:3 https://cdn-aws.deb.debian.org/debian-security stretch/updates InRelease Hit:4 https://cdn-aws.deb.debian.org/debian stretch-backports InRelease Hit:5 https://cdn-aws.deb.debian.org/debian stretch Release Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done ca-certificates is already the newest version (20161130+nmu1). sudo is already the newest version (1.8.19p1-2.1). wget is already the newest version (1.18-5+deb9u2). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. /tmp/nextcloudplus.n9eLwT ~ --2018-05-21 13:36:52-- https://github.com/nextcloud/nextcloudpi/archive/master/latest.tar.gz Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112 Connecting to github.com (github.com)|192.30.253.113|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/nextcloud/nextcloudpi/tar.gz/master [following] --2018-05-21 13:36:53-- https://codeload.github.com/nextcloud/nextcloudpi/tar.gz/master Resolving codeload.github.com (codeload.github.com)... 192.30.253.121, 192.30.253.120 Connecting to codeload.github.com (codeload.github.com)|192.30.253.121|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/x-gzip] Saving to: ‘STDOUT’ - [ <=> ] 712.20K 983KB/s in 0.7s 2018-05-21 13:36:54 (983 KB/s) - written to stdout [729296] /root Installing NextCloudPlus Installing lamp Ign:1 https://cdn-aws.deb.debian.org/debian stretch InRelease Hit:2 https://cdn-aws.deb.debian.org/debian stretch-updates InRelease Hit:3 https://cdn-aws.deb.debian.org/debian-security stretch/updates InRelease Hit:4 https://cdn-aws.deb.debian.org/debian stretch-backports InRelease Hit:5 https://cdn-aws.deb.debian.org/debian stretch Release Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done apt-utils is already the newest version (1.4.8). cron is already the newest version (3.0pl1-128+deb9u1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree Reading state information... Done apache2 is already the newest version (2.4.25-3+deb9u4). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'php7.0-common' instead of 'php7.0-fileinfo' php7.0-cli is already the newest version (7.0.27-0+deb9u1). php7.0-cli set to manually installed. php7.0-common is already the newest version (7.0.27-0+deb9u1). php7.0-common set to manually installed. php7.0-curl is already the newest version (7.0.27-0+deb9u1). php7.0-curl set to manually installed. php7.0-gd is already the newest version (7.0.27-0+deb9u1). php7.0-gd set to manually installed. php7.0-mbstring is already the newest version (7.0.27-0+deb9u1). php7.0-mbstring set to manually installed. php7.0-mcrypt is already the newest version (7.0.27-0+deb9u1). php7.0-mcrypt set to manually installed. php7.0-opcache is already the newest version (7.0.27-0+deb9u1). php7.0-opcache set to manually installed. php7.0-xml is already the newest version (7.0.27-0+deb9u1). php7.0-xml set to manually installed. php7.0-zip is already the newest version (7.0.27-0+deb9u1). php7.0-zip set to manually installed. Suggested packages: php-pear The following NEW packages will be installed: php7.0 php7.0-fpm php7.0-ldap 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 1,364 kB of archives. After this operation, 4,638 kB of additional disk space will be used. Get:1 https://cdn-aws.deb.debian.org/debian stretch/main amd64 php7.0-fpm amd64 7.0.27-0+deb9u1 [1,290 kB] Get:2 https://cdn-aws.deb.debian.org/debian stretch/main amd64 php7.0 all 7.0.27-0+deb9u1 [52.9 kB] Get:3 https://cdn-aws.deb.debian.org/debian stretch/main amd64 php7.0-ldap amd64 7.0.27-0+deb9u1 [21.0 kB] Fetched 1,364 kB in 2s (598 kB/s) Selecting previously unselected package php7.0-fpm. (Reading database ... 20727 files and directories currently installed.) Preparing to unpack .../php7.0-fpm_7.0.27-0+deb9u1_amd64.deb ... Unpacking php7.0-fpm (7.0.27-0+deb9u1) ... Selecting previously unselected package php7.0. Preparing to unpack .../php7.0_7.0.27-0+deb9u1_all.deb ... Unpacking php7.0 (7.0.27-0+deb9u1) ... Selecting previously unselected package php7.0-ldap. Preparing to unpack .../php7.0-ldap_7.0.27-0+deb9u1_amd64.deb ... Unpacking php7.0-ldap (7.0.27-0+deb9u1) ... Setting up php7.0-ldap (7.0.27-0+deb9u1) ... Creating config file /etc/php/7.0/mods-available/ldap.ini with new version Setting up php7.0 (7.0.27-0+deb9u1) ... Processing triggers for libapache2-mod-php7.0 (7.0.27-0+deb9u1) ... Setting up php7.0-fpm (7.0.27-0+deb9u1) ... Creating config file /etc/php/7.0/fpm/php.ini with new version php_invoke: Enable module curl for fpm sapi php_invoke: Enable module posix for fpm sapi php_invoke: Enable module xsl for fpm sapi php_invoke: Enable module iconv for fpm sapi php_invoke: Enable module gd for fpm sapi php_invoke: Enable module ctype for fpm sapi php_invoke: Enable module fileinfo for fpm sapi php_invoke: Enable module simplexml for fpm sapi php_invoke: Enable module dom for fpm sapi php_invoke: Enable module tokenizer for fpm sapi php_invoke: Enable module gettext for fpm sapi php_invoke: Enable module sockets for fpm sapi php_invoke: Enable module sysvsem for fpm sapi php_invoke: Enable module shmop for fpm sapi php_invoke: Enable module phar for fpm sapi php_invoke: Enable module json for fpm sapi php_invoke: Enable module mysqli for fpm sapi php_invoke: Enable module xmlwriter for fpm sapi php_invoke: Enable module opcache for fpm sapi php_invoke: Enable module exif for fpm sapi php_invoke: Enable module apcu for fpm sapi php_invoke: Enable module readline for fpm sapi php_invoke: Enable module zip for fpm sapi php_invoke: Enable module ftp for fpm sapi php_invoke: Enable module calendar for fpm sapi php_invoke: Enable module redis for fpm sapi php_invoke: Enable module pdo for fpm sapi php_invoke: Enable module mbstring for fpm sapi php_invoke: Enable module xml for fpm sapi php_invoke: Enable module sysvmsg for fpm sapi php_invoke: Enable module igbinary for fpm sapi php_invoke: Enable module xmlreader for fpm sapi php_invoke: Enable module pdo_mysql for fpm sapi php_invoke: Enable module mysqlnd for fpm sapi php_invoke: Enable module wddx for fpm sapi php_invoke: Enable module sysvshm for fpm sapi php_invoke: Enable module mcrypt for fpm sapi NOTICE: Not enabling PHP 7.0 FPM by default. NOTICE: To enable PHP 7.0 FPM in Apache2 do: NOTICE: a2enmod proxy_fcgi setenvif NOTICE: a2enconf php7.0-fpm NOTICE: You are seeing this message because you have apache2 package installed. Created symlink /etc/systemd/system/multi-user.target.wants/php7.0-fpm.service → /lib/systemd/system/php7.0-fpm.service. Processing triggers for systemd (232-25+deb9u3) ... Reading package lists... Done Building dependency tree Reading state information... Done mariadb-server is already the newest version (10.1.26-0+deb9u1). php7.0-mysql is already the newest version (7.0.27-0+deb9u1). php7.0-mysql set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Enabling module http2. To activate the new configuration, you need to run: systemctl restart apache2 Enabling conf http2. To activate the new configuration, you need to run: systemctl reload apache2 Considering dependency proxy for proxy_fcgi: Enabling module proxy. Enabling module proxy_fcgi. Module setenvif already enabled To activate the new configuration, you need to run: systemctl restart apache2 Enabling conf php7.0-fpm. To activate the new configuration, you need to run: systemctl reload apache2 Enabling module rewrite. To activate the new configuration, you need to run: systemctl restart apache2 Enabling module headers. To activate the new configuration, you need to run: systemctl restart apache2 Module dir already enabled Module mime already enabled Considering dependency setenvif for ssl: Module setenvif already enabled Considering dependency mime for ssl: Module mime already enabled Considering dependency socache_shmcb for ssl: Enabling module socache_shmcb. Enabling module ssl. See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates. To activate the new configuration, you need to run: systemctl restart apache2 Module status disabled. Module reqtimeout disabled. Module env disabled. Module autoindex disabled. Module access_compat disabled. Module auth_basic disabled. Module authn_file disabled. Module authn_core disabled. Module alias disabled. Module access_compat already disabled To activate the new configuration, you need to run: systemctl restart apache2 Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: openssl-blacklist The following NEW packages will be installed: ssl-cert 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 20.8 kB of archives. After this operation, 64.5 kB of additional disk space will be used. Get:1 https://cdn-aws.deb.debian.org/debian stretch/main amd64 ssl-cert all 1.0.39 [20.8 kB] Fetched 20.8 kB in 1s (10.7 kB/s) Preconfiguring packages ... Selecting previously unselected package ssl-cert. (Reading database ... 20767 files and directories currently installed.) Preparing to unpack .../ssl-cert_1.0.39_all.deb ... Unpacking ssl-cert (1.0.39) ... Setting up ssl-cert (1.0.39) ... NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY! In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here. stty: 'standard input': Inappropriate ioctl for device Enter current password for root (enter for none): stty: 'standard input': Inappropriate ioctl for device OK, successfully used password, moving on... Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorisation. You already have a root password set, so you can safely answer 'n'. Change the root password? [Y/n] stty: 'standard input': Inappropriate ioctl for device New password: Re-enter new password: stty: 'standard input': Inappropriate ioctl for device Password updated successfully! Reloading privilege tables.. ... Success! By default, a MariaDB installation has an anonymous user, allowing anyone to log into MariaDB without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? [Y/n] ... Success! Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? [Y/n] ... Success! By default, MariaDB comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? [Y/n] - Dropping test database... ... Success! - Removing privileges on test database... ... Success! Reloading the privilege tables will ensure that all changes made so far will take effect immediately. Reload privilege tables now? [Y/n] ... Success! Cleaning up... All done! If you've completed all of the above steps, your MariaDB installation should now be secure. Thanks for using MariaDB! Installing nc-nextcloud Ign:1 https://cdn-aws.deb.debian.org/debian stretch InRelease Hit:2 https://cdn-aws.deb.debian.org/debian stretch-updates InRelease Hit:3 https://cdn-aws.deb.debian.org/debian-security stretch/updates InRelease Hit:4 https://cdn-aws.deb.debian.org/debian stretch-backports InRelease Hit:5 https://cdn-aws.deb.debian.org/debian stretch Release Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done iputils-ping is already the newest version (3:20161105-1). The following NEW packages will be installed: lbzip2 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 75.5 kB of archives. After this operation, 172 kB of additional disk space will be used. Get:1 https://cdn-aws.deb.debian.org/debian stretch/main amd64 lbzip2 amd64 2.5-2 [75.5 kB] Fetched 75.5 kB in 2s (36.9 kB/s) Selecting previously unselected package lbzip2. (Reading database ... 20776 files and directories currently installed.) Preparing to unpack .../lbzip2_2.5-2_amd64.deb ... Unpacking lbzip2 (2.5-2) ... Setting up lbzip2 (2.5-2) ... Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libdbus-1-3 libldb1 libpython-stdlib libpython2.7 libpython2.7-minimal libpython2.7-stdlib libsmbclient libtalloc2 libtdb1 libtevent0 libwbclient0 python python-minimal python-talloc python2.7 python2.7-minimal samba-libs Suggested packages: cups-common python-doc python-tk python2.7-doc binutils binfmt-support Recommended packages: dbus The following NEW packages will be installed: libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libdbus-1-3 libldb1 libpython-stdlib libpython2.7 libpython2.7-minimal libpython2.7-stdlib libsmbclient libtalloc2 libtdb1 libtevent0 libwbclient0 php-smbclient python python-minimal python-talloc python2.7 python2.7-minimal samba-libs 0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded. Need to get 11.7 MB of archives. After this operation, 46.6 MB of additional disk space will be used. Get:1 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libpython2.7-minimal amd64 2.7.13-2+deb9u2 [389 kB] Get:2 https://cdn-aws.deb.debian.org/debian stretch/main amd64 python2.7-minimal amd64 2.7.13-2+deb9u2 [1,382 kB] Get:3 https://cdn-aws.deb.debian.org/debian stretch/main amd64 python-minimal amd64 2.7.13-2 [40.5 kB] Get:4 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libpython2.7-stdlib amd64 2.7.13-2+deb9u2 [1,896 kB] Get:5 https://cdn-aws.deb.debian.org/debian stretch/main amd64 python2.7 amd64 2.7.13-2+deb9u2 [285 kB] Get:6 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libpython-stdlib amd64 2.7.13-2 [20.0 kB] Get:7 https://cdn-aws.deb.debian.org/debian stretch/main amd64 python amd64 2.7.13-2 [154 kB] Get:8 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libtalloc2 amd64 2.1.8-1 [36.3 kB] Get:9 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libtevent0 amd64 0.9.31-1 [29.3 kB] Get:10 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libavahi-common-data amd64 0.6.32-2 [118 kB] Get:11 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libavahi-common3 amd64 0.6.32-2 [52.0 kB] Get:12 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libdbus-1-3 amd64 1.10.26-0+deb9u1 [194 kB] Get:13 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libavahi-client3 amd64 0.6.32-2 [55.3 kB] Get:14 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libcups2 amd64 2.2.1-8+deb9u1 [306 kB] Get:15 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libtdb1 amd64 1.3.11-2 [43.2 kB] Get:16 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libldb1 amd64 2:1.1.27-1+b1 [114 kB] Get:17 https://cdn-aws.deb.debian.org/debian stretch/main amd64 libpython2.7 amd64 2.7.13-2+deb9u2 [1,072 kB] Get:18 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main amd64 libwbclient0 amd64 2:4.5.12+dfsg-2+deb9u2 [124 kB] Get:19 https://cdn-aws.deb.debian.org/debian stretch/main amd64 python-talloc amd64 2.1.8-1 [11.9 kB] Get:20 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main amd64 samba-libs amd64 2:4.5.12+dfsg-2+deb9u2 [5,212 kB] Get:21 https://cdn-aws.deb.debian.org/debian-security stretch/updates/main amd64 libsmbclient amd64 2:4.5.12+dfsg-2+deb9u2 [150 kB] Get:22 https://cdn-aws.deb.debian.org/debian stretch/main amd64 php-smbclient amd64 0.8.0-3 [19.8 kB] Fetched 11.7 MB in 5s (2,220 kB/s) Selecting previously unselected package libpython2.7-minimal:amd64. (Reading database ... 20786 files and directories currently installed.) Preparing to unpack .../0-libpython2.7-minimal_2.7.13-2+deb9u2_amd64.deb ... Unpacking libpython2.7-minimal:amd64 (2.7.13-2+deb9u2) ... Selecting previously unselected package python2.7-minimal. Preparing to unpack .../1-python2.7-minimal_2.7.13-2+deb9u2_amd64.deb ... Unpacking python2.7-minimal (2.7.13-2+deb9u2) ... Selecting previously unselected package python-minimal. Preparing to unpack .../2-python-minimal_2.7.13-2_amd64.deb ... Unpacking python-minimal (2.7.13-2) ... Selecting previously unselected package libpython2.7-stdlib:amd64. Preparing to unpack .../3-libpython2.7-stdlib_2.7.13-2+deb9u2_amd64.deb ... Unpacking libpython2.7-stdlib:amd64 (2.7.13-2+deb9u2) ... Selecting previously unselected package python2.7. Preparing to unpack .../4-python2.7_2.7.13-2+deb9u2_amd64.deb ... Unpacking python2.7 (2.7.13-2+deb9u2) ... Selecting previously unselected package libpython-stdlib:amd64. Preparing to unpack .../5-libpython-stdlib_2.7.13-2_amd64.deb ... Unpacking libpython-stdlib:amd64 (2.7.13-2) ... Setting up libpython2.7-minimal:amd64 (2.7.13-2+deb9u2) ... Setting up python2.7-minimal (2.7.13-2+deb9u2) ... Setting up python-minimal (2.7.13-2) ... Selecting previously unselected package python. (Reading database ... 21533 files and directories currently installed.) Preparing to unpack .../00-python_2.7.13-2_amd64.deb ... Unpacking python (2.7.13-2) ... Selecting previously unselected package libtalloc2:amd64. Preparing to unpack .../01-libtalloc2_2.1.8-1_amd64.deb ... Unpacking libtalloc2:amd64 (2.1.8-1) ... Selecting previously unselected package libtevent0:amd64. Preparing to unpack .../02-libtevent0_0.9.31-1_amd64.deb ... Unpacking libtevent0:amd64 (0.9.31-1) ... Selecting previously unselected package libavahi-common-data:amd64. Preparing to unpack .../03-libavahi-common-data_0.6.32-2_amd64.deb ... Unpacking libavahi-common-data:amd64 (0.6.32-2) ... Selecting previously unselected package libavahi-common3:amd64. Preparing to unpack .../04-libavahi-common3_0.6.32-2_amd64.deb ... Unpacking libavahi-common3:amd64 (0.6.32-2) ... Selecting previously unselected package libdbus-1-3:amd64. Preparing to unpack .../05-libdbus-1-3_1.10.26-0+deb9u1_amd64.deb ... Unpacking libdbus-1-3:amd64 (1.10.26-0+deb9u1) ... Selecting previously unselected package libavahi-client3:amd64. Preparing to unpack .../06-libavahi-client3_0.6.32-2_amd64.deb ... Unpacking libavahi-client3:amd64 (0.6.32-2) ... Selecting previously unselected package libcups2:amd64. Preparing to unpack .../07-libcups2_2.2.1-8+deb9u1_amd64.deb ... Unpacking libcups2:amd64 (2.2.1-8+deb9u1) ... Selecting previously unselected package libtdb1:amd64. Preparing to unpack .../08-libtdb1_1.3.11-2_amd64.deb ... Unpacking libtdb1:amd64 (1.3.11-2) ... Selecting previously unselected package libldb1:amd64. Preparing to unpack .../09-libldb1_2%3a1.1.27-1+b1_amd64.deb ... Unpacking libldb1:amd64 (2:1.1.27-1+b1) ... Selecting previously unselected package libpython2.7:amd64. Preparing to unpack .../10-libpython2.7_2.7.13-2+deb9u2_amd64.deb ... Unpacking libpython2.7:amd64 (2.7.13-2+deb9u2) ... Selecting previously unselected package libwbclient0:amd64. Preparing to unpack .../11-libwbclient0_2%3a4.5.12+dfsg-2+deb9u2_amd64.deb ... Unpacking libwbclient0:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Selecting previously unselected package python-talloc. Preparing to unpack .../12-python-talloc_2.1.8-1_amd64.deb ... Unpacking python-talloc (2.1.8-1) ... Selecting previously unselected package samba-libs:amd64. Preparing to unpack .../13-samba-libs_2%3a4.5.12+dfsg-2+deb9u2_amd64.deb ... Unpacking samba-libs:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Selecting previously unselected package libsmbclient:amd64. Preparing to unpack .../14-libsmbclient_2%3a4.5.12+dfsg-2+deb9u2_amd64.deb ... Unpacking libsmbclient:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Selecting previously unselected package php-smbclient. Preparing to unpack .../15-php-smbclient_0.8.0-3_amd64.deb ... Unpacking php-smbclient (0.8.0-3) ... Setting up libtdb1:amd64 (1.3.11-2) ... Setting up libwbclient0:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Processing triggers for mime-support (3.60) ... Processing triggers for libc-bin (2.24-11+deb9u3) ... Setting up libtalloc2:amd64 (2.1.8-1) ... Setting up libpython2.7-stdlib:amd64 (2.7.13-2+deb9u2) ... Setting up libdbus-1-3:amd64 (1.10.26-0+deb9u1) ... Setting up libavahi-common-data:amd64 (0.6.32-2) ... Setting up python2.7 (2.7.13-2+deb9u2) ... Setting up libpython-stdlib:amd64 (2.7.13-2) ... Setting up libtevent0:amd64 (0.9.31-1) ... Setting up libpython2.7:amd64 (2.7.13-2+deb9u2) ... Setting up libavahi-common3:amd64 (0.6.32-2) ... Setting up python (2.7.13-2) ... Setting up python-talloc (2.1.8-1) ... Setting up libldb1:amd64 (2:1.1.27-1+b1) ... Setting up libavahi-client3:amd64 (0.6.32-2) ... Setting up libcups2:amd64 (2.2.1-8+deb9u1) ... Setting up samba-libs:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Setting up libsmbclient:amd64 (2:4.5.12+dfsg-2+deb9u2) ... Setting up php-smbclient (0.8.0-3) ... Processing triggers for libc-bin (2.24-11+deb9u3) ... Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: postfix-sqlite Suggested packages: procmail postfix-mysql postfix-pgsql postfix-ldap postfix-pcre postfix-lmdb sasl2-bin libsasl2-modules dovecot-common postfix-cdb mail-reader ufw postfix-doc Recommended packages: python3 The following NEW packages will be installed: postfix postfix-sqlite 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 1,755 kB of archives. After this operation, 4,378 kB of additional disk space will be used. Get:1 https://cdn-aws.deb.debian.org/debian stretch/main amd64 postfix-sqlite amd64 3.1.8-0+deb9u1 [318 kB] Get:2 https://cdn-aws.deb.debian.org/debian stretch/main amd64 postfix amd64 3.1.8-0+deb9u1 [1,437 kB] Fetched 1,755 kB in 2s (742 kB/s) Preconfiguring packages ... Selecting previously unselected package postfix-sqlite. (Reading database ... 21916 files and directories currently installed.) Preparing to unpack .../postfix-sqlite_3.1.8-0+deb9u1_amd64.deb ... Unpacking postfix-sqlite (3.1.8-0+deb9u1) ... Selecting previously unselected package postfix. Preparing to unpack .../postfix_3.1.8-0+deb9u1_amd64.deb ... Unpacking postfix (3.1.8-0+deb9u1) ... Processing triggers for systemd (232-25+deb9u3) ... Setting up postfix-sqlite (3.1.8-0+deb9u1) ... grep: /etc/postfix/dynamicmaps.cf: No such file or directory Adding sqlite map entry to /etc/postfix/dynamicmaps.cf Setting up postfix (3.1.8-0+deb9u1) ... Created symlink /etc/systemd/system/multi-user.target.wants/postfix.service → /lib/systemd/system/postfix.service. Adding group `postfix' (GID 115) ... Done. Adding system user `postfix' (UID 109) ... Adding new user `postfix' (UID 109) with group `postfix' ... Not creating home directory `/var/spool/postfix'. Adding group `postdrop' (GID 116) ... Done. postconf: warning: inet_protocols: disabling IPv6 name/address support: Address family not supported by protocol postconf: warning: inet_protocols: disabling IPv6 name/address support: Address family not supported by protocol postconf: warning: inet_protocols: disabling IPv6 name/address support: Address family not supported by protocol setting myhostname: DietPi setting alias maps setting alias database mailname is not a fully qualified domain name. Not changing /etc/mailname. setting destinations: $myhostname, DietPi, localhost.localdomain, , localhost setting relayhost: setting mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 setting mailbox_size_limit: 0 setting recipient_delimiter: + setting inet_interfaces: all setting inet_protocols: ipv4 /etc/aliases does not exist, creating it. WARNING: /etc/aliases exists, but does not have a root alias. Postfix (main.cf) is now set up with a default configuration. If you need to make changes, edit /etc/postfix/main.cf (and others) as needed. To view Postfix configuration values, see postconf(1). After modifying main.cf, be sure to run 'service postfix reload'. Running newaliases Processing triggers for systemd (232-25+deb9u3) ... Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'php-redis' instead of 'php7.0-redis' php-redis is already the newest version (3.1.1-1). redis-server is already the newest version (3:3.2.6-1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Synchronizing state of redis-server.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable redis-server Created symlink /etc/systemd/system/redis.service → /lib/systemd/system/redis-server.service. Failed to stop mysqld.service: Unit mysqld.service not loaded. ```

Workaround to fix mysqld service name: sed -i '/systemctl .* mysqld/s/mysqld/mysql/g' etc/ncp-config.d/nc-nextcloud.sh

Next attempt with fix as above goes on:

dnsmasq is installed, check compatibility with DietPi dnsmasq using software, Pi-hole, hostapd etc.
letsencrypt is installed, incompatible with DietPi-LetsEncrypt. Consider moving those and above into separate optional install step.
🈴 dnsmasq setup breaks name resolution on DietPi. Scripts goes on, continuously failing to connect to APT repos etc. Consider implementing some connectivity check after critical network related changes/installations and break script in case of error.
No-IP is installed, Samba, unattended-upgrades (DietPi currently avoids this, as it could break our own scripts randomly. Although I like automated APT upgrades, next item on my list.

dnsmasq starts up afterwards, no idea why this failed during script execution:

Setting up dnsmasq (2.76-5+deb9u1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/dnsmasq.service → /lib/systemd/system/dnsmasq.service.
Job for dnsmasq.service failed because a timeout was exceeded.
See "systemctl status dnsmasq.service" and "journalctl -xe" for details.
invoke-rc.d: initscript dnsmasq, action "start" failed.
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
   Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
   Active: failed (Result: timeout) since Mon 2018-05-21 15:14:31 CEST; 87ms ago
  Process: 9469 ExecStartPost=/etc/init.d/dnsmasq systemd-start-resolvconf (code=killed, signal=TERM)
  Process: 9462 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=0/SUCCESS)
  Process: 9459 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCCESS)
 Main PID: 9468 (code=exited, status=0/SUCCESS)

May 21 15:13:01 DietPi dnsmasq[9468]: started, version 2.76 cachesize 150
May 21 15:13:01 DietPi dnsmasq[9468]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify
May 21 15:13:01 DietPi dnsmasq[9468]: DNS service limited to local subnets
May 21 15:13:01 DietPi dnsmasq[9468]: read /etc/hosts - 2 addresses
May 21 15:13:01 DietPi dnsmasq[9468]: reading /run/dnsmasq/resolv.conf
May 21 15:13:01 DietPi dnsmasq[9468]: using nameserver 192.168.178.1#53
May 21 15:14:31 DietPi systemd[1]: dnsmasq.service: Start-post operation timed out. Stopping.
May 21 15:14:31 DietPi systemd[1]: Failed to start dnsmasq - A lightweight DHCP and caching DNS server.
May 21 15:14:31 DietPi systemd[1]: dnsmasq.service: Unit entered failed state.
May 21 15:14:31 DietPi systemd[1]: dnsmasq.service: Failed with result 'timeout'.

Next attempt, now that dnsmasq is already installed and running:

🈴 Script fails/breaks, if ncp user already exists at: https://github.com/nextcloud/nextcloudpi/blob/master/nextcloudplus.sh#L120
Fix with: sed -i 's|^[[:blank:]]*useradd|grep "^$WEBADMIN:" /etc/passwd || useradd|' nextcloudplus.sh

Next attempt with fixed user creation:

🈺 Script tries to access config.php before it's created, but I could absolutely not find the exact code line, where this is done... After avahi-daemon installation (nextcloudplus.sh) but before nc-init 🤔However does not seem to break something:

Setting up avahi-daemon (0.6.32-2) ...
Created symlink /etc/systemd/system/dbus-org.freedesktop.Avahi.service → /lib/systemd/system/avahi-daemon.service.
Created symlink /etc/systemd/system/multi-user.target.wants/avahi-daemon.service → /lib/systemd/system/avahi-daemon.service.
Created symlink /etc/systemd/system/sockets.target.wants/avahi-daemon.socket → /lib/systemd/system/avahi-daemon.socket.
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for systemd (232-25+deb9u3) ...
Processing triggers for dbus (1.10.26-0+deb9u1) ...
grep: /var/www/nextcloud/config/config.php: No such file or directory
Installing nc-backup
Installing nc-restore
grep: /var/www/nextcloud/config/config.php: No such file or directory
NextCloudPlus updated to version v0.56.6
Activating nc-init
Setting up a clean Nextcloud instance... wait until message 'NC init done'

Result

🈯️ With the two fixes/workarounds above, installation goes through nicely.
🈺 Some setup steps break other DietPi software currently, webserver and database installation + configuration is split over different scripts. Skipping the first, breaks the later ones, thus is quite some work to make it compatible.
🈺 nc-config/the web ui contains many system settings, including file server, swap file, ramlog, dnsmasq, fail2ban, letsencrypt, ... which we already handle within DietPi. We would need a "version" that really just contains Nextcloud configuration, backup, data moving, auto update/info (Nextcloud-only), file scanning and such things. Everything else conflicts DietPi implementations.

As far as I can see now, it is quite a huge work and would need a huge adjustment to NCP to fit into DietPi concept. NCP is not just a Nextcloud installation, it is a whole image/system implementation that configures everything according to run Nextcloud nicely and offer many Nextcloud-related system features in a user-friendly way. It's script is a bid comparable with our DietPi-PREP, adjusting as well up to kernel settings. I think to underlying ideas of both projects are hard to combine currently 🤔. It is indeed a different approach if you wan't to build a system around a Nextcloud installation or Nextcloud, as one of many application into an otherwise fully developed system.

great job. Give me a few days to go through these points, as I am now preparing the next release

@nachoparker About mysql service name. Should be possible for compatibility to just try one by one all possible service names, mysql/mqsqld/mariadb and break loop after one was able to be restarted without error code.

About ncp user name. It's maybe good to let user choose if this existing user should be used/permission set or not, just in case if it exists for another reason/application. Or if unattended install is wanted, do some error handing/give user feedback about ncp user is needed/wanted for ncp and should be removed before next run.

Hi again,

Apologies for taking so long but I have not had the time to sit down for this before.

Indeed, the approach is different in both projects, and some modifications would need to be made, which might be fine. Still, we could share the maintenance burden so I think it would be nice for both projects.

If you look at the code, NCP installation boils down to the following steps

install_script lamp.sh
install_script nc-nextcloud.sh           # install Postfix and Redis
activate_script nc-nextcloud.sh          # install NC itself
install_script ncp.sh                    # install all NCP extras 
install_script nc-init.sh                # preconfigure NC

I think that the first logical step would be to skip ncp.sh and initially share the rest of the installation between projects. Then we could look at what NCP really adds and how it can be combined with what DietPi does.

Also, NCP only supports Stretch ATM, and the main reason is really small manpower. What is the case with DietPi? do you guys support several different releases?

To your points:

Apache is installed with php-fpm, where DietPi installs it with mod-php.

This is one thing that would need to be changed on one side... well, not necessarily. Which one is best? did a little search.

https://www.cloudways.com/blog/php-fpm-on-cloud/ http://fastjoomlahost.com/mod_php-fastcgi-php-fpm-server

Thoughts?

Apache configured with basic HTTPS functionality (snake oil certs), although within separate configuration/vhost and no forced redirection/HSTS, this should be no problem.

HSTS is enabled by default code. Also HTTP2, OCSP Stapling and other stuff.

MariaDB configuration output seems to expect root password authentication. However since Debian Stretch, unix_socket authentication is default, allowing login only when logged in as unix root user, but without the need to enter password for MariaDB login again. Compatibility with this behaviour assured? u6307€: Ah just found mysql_secure_installation script as provided by MariaDB. Nice to have, but on Debian Stretch not needed, as stated above: root user present only with unix_socket authentication configured, which is great for my impression. However running the script does not damage something.

mysql_secure_installation does other things too, it's nice to keep I think ;)

the unix_socket behaviour should be no problem. I didn't even notice the change and didn't have to adapt it to Stretch.

Postfix is installed. Would be nice to have modules like this optional, allowing to e.g. skip installation (and of course potential UI elements, depending on it) via command option, global variable or setup configuration file.

It is a different step, ( nc-nextcloud.sh "activation", meaning the configure() step ), so it would be easy to separate, but I wonder what's the benefit of removing it. Does DietPi also install postfix?

Redis is being installed at last, consider moving installation to the beginning within/between LAMP stack installation etc. and just do final Nextcloud related needed configuration within final NCP install script. This would also avoid enabling the service, which does not work in combination with DietPi, controlling the services itself.

Redis is installed after LAMP, and before Nextcloud (same with Postfix), so I think we agree here. I didn't understand the problem with enabling/disabling the service.

Last output: Failed to stop mysqld.service: Unit mysqld.service not loaded. The service is called mysql (init.d) or optionally mariadb (systemd). Is installation aborted here or finished anyway?

In my Raspbian Stretch all three exist: mysql, mysqld, mariadb, so I am surprised. Are you in Stretch? In any case, no problem to change this.

Installer expects PHP 7.0 to be installed. This will break installation with default PHP version on Jessie (PHP 5) and Buster (PHP 7.2).

Again, ATM only Stretch is supported. There are people starting to ask for PHP > 7.0, so one option would be to move to PHP 7.2 (by bringing sources from Buster to Stretch). That could potentially cause problems though, for instance that is what I did in Jessie to get PHP 7.0 support, but it would break php-smbclient.

Also, PHP 5.0 is just slow

Script fails/breaks, if ncp user already exists at: https://github.com/nextcloud/nextcloudpi/blob/master/nextcloudplus.sh#L120

The ncp user is not a login user. It is only used to log in to ncp-web. I think it is better that is separated from the login user, and has no permissions whatsoever. In any case, this is part of ncp.sh

In lamp.sh, another thing that should be done is to stop modifying /etc/mysql/mariadb.conf.d/50-server.cnf and create separate files, like /etc/mysql/mariadb.conf.d/90-ncp.cnf. Same for php ini and apache files where possible.

So the "curl installer" is a very simple one in terms of robustness: it expects a clean Stretch system. If there is already a LAMP stack or if you try to install it twice some things might fail. This would be another point for improvement.

I'd actually like the curl installer to allow "frugal"? installs on any preexisting and externally managed debian installation.

The LAMP and system settings would have to be checked and adjusted in a finer grained way, when there is no modular conf.d directory available.

May something like a "nc-provisioning-service" work?:

A script that:

Makes completed ncp installs detectable by moving a temporary ncp config state directory to a permanent place on success.
Instead of replacing (default or external) config files, always greps config files for the individual required ncp settings or stanzas and if not present:
- If it is a required configuration, insert/comment out or change it in the existing files (together with a comment "# Following line gets udpated by NCP (last on $date):".
- If the preexisting config file differs from the version shiped (debsums -e | grep FAILED), log the changes made to file and notify the admin.
Keeps separate copies of the managed config files and inotify-watches the config files to redo the grep/edit on any external changes, and sends the diff to the admin.

Well, maybe in the end above "nc-provisioning-service" could get installed with a .deb package.

install_script lamp.sh install_script nc-nextcloud.sh # install Postfix and Redis install_script ncp.sh # install all NCP extras -> debian package with package dependencies that installs the nc-provisioning-service script

activate_script nc-nextcloud.sh # install NC itself install_script nc-init.sh # preconfigure NC -> configs watched by nc-provisioning-service

hi @testbird .

I don't think I understand the point of this new script. I think that if possible, it would be nice to enhance the current scripts (lamp.sh specially) to

be more modular, by creating conf.dfiles instead of modifying the global configuration files
try to minimize changes, because of the fact that it won't be the only thing installed in the system necessarily anymore
try to detect whenever something might get broken (hard) by the script and make it more robust so that it can be, for instance, run twice in a system without problems. For instance, don't run nc-init if a NC instance already exists.
backup any file that is changed

In any case, I still think that integration should start by the lamp.sh + nc-nextcloud.sh + nc-init.sh, skipping the rest of NCP software. When that is completed we can move on to see how and if we add the rest of the software.

Hi, the reason I thought of a new "script" may just be that I may not have understood completely the reasons for the way you had split things up currently.

The idea was just wrapping things into some (service?) file that is started on boot to check the configs and starts watching for changes. (May just call the scripts as split up currently.)

Do you think this would solve your points 2-4?:

Each action on an existing file would have to 1) be conditional (skipped if the outcome already exists) 1) copy the old and new version of the file to a backup location 1) get logged.

For particular settings the condition test could, for example, be a regexp grep.

How would you control skipping the rest of NCP software, for now?

@testbird @nachoparker Sorry also from my side to leave this issue unanswered for such a long time. Actually I got the impression, that really the approaches and initial aims (that formed how both projects install and configure on the system) are too different to easily merge in some way. And limited time made me concentrate on other topics. But I still find this interesting, also as a learning project about how create projects most compatible, less intrusive and caring about existing user installs and adjustments. Especially in open source world, this makes the fork and use of code in different environments easier. But yeah, hard to achieve this after the project has grown for a while 😄.

Generally great is:

Find a good share between optimized and more general compatible/not touched settings. But yeah, we also simply pre-configure software best for our users, so...
Use separate conf.d settings file for own adjustments, where possible. This is also something we are more and more aiming for with DietPi, though still much to do. This also allows APT upgrades with updated package maintainer config files, where our conf.d/* just overwrites what we really want.
If no conf.d possible, carefully apply the needed settings within existing settings file and yeah inform user about it. We use some config injection function for this: https://github.com/Fourdee/DietPi/blob/testing/dietpi/func/dietpi-globals#L2223
- It scans for the setting first, allowing leading spaces, optionally leaves existing setting in place, otherwise overwrites it with chosen value. If setting was not found, it scans for commented ones, which is often the case within default config files, and "activates" the commented setting. By this it will be at the right place of the file, preserving explaining comments etc. Only if the setting was not found at all, if adds it to the end of the file, optionally after a certain line, which is e.g. needed to add settings to nextcloud/config/config.php that must not be outside of the braces. User output is done about everything, and much debug/error handling, to handle special characters as best as possible. With input string > function > grep > sed|redirect > echo all need to show/edit exactly the same, this is actually a horrible task 🤣. But when possible input is known, this can be simplified very much.
Doing updates of configuration files before editing them is actually a great idea. I should add this to our function: Copy file to some backup location (within our userdata), if backup does not yet exist. After everything is done, inform user about backup location to possibly clean up or recover custom settings 👍 .

About Apache `mod-php` vs `php-fpm`

This is a really difficult question and clearly there is no better without a certain case. I just had a conversation on https://help.nextcloud.com about it with someone that has much more insight than me. If it's about features, there is no real reason any more to use mod-php. The only thing it allows, which php-fpm does not, is using php-(admin-)value/flag to set PHP values within webserver configuration. By this you can apply values, which cannot be directly applied directory wise via .user.ini (and such, e.g. opcache.memory_consumption to satisfy Nextcloud admin panel) still directory wise, by placing them within a certain vhosts location statement. php-fpm (mpm_worker/event) on the other hand allows HTTP/2 and all the php-fpm specific configuration/features. But Apache from the ground up for my impression does no handle php-fpm too good/natively, as Nginx and Lighttpd do. There will remain several Apache processes (to handle static requests) according to the mpm_worker/event settings, besides the php-fpm processes. Nginx/Lighttpd just spawn a single process here. But that's maybe just a visual. According to Apache defaults I finally concluded that it makes more sense, if according to high access rates php-fpm is wanted, to directly choose Nginx or Lighttpd, which allow this more natively and with less RAM usage. On low traffic/concurrent access servers, Apache with mod-php is most likely a tiny bid faster, as everything about a request is handled within a single process.

But ideally this is left as (optional) user choice. In case of NextcloudPlus it would be very great if webserver installation could be fully skipped. But then there need to be vhosts available for every (supported) webserver and optimizations need to be done per webserver. This is what we try with our install script, working good for Apache and Nginx. But Lighttpd e.g. lacks example Nextcloud configuration and there seem to be nearly no users that chose Nextcloud+Lighttpd (besides DietPi default installation), so we do not change much there, also as Lighttpd + Nextcloud runs simply out of the box without any config edit 😄.

@nachoparker

HSTS is enabled by default code. Also HTTP2, OCSP Stapling and other stuff.

For Nextcloud this also makes much sense. In our case many users use their system within local network only for media system management and such, without access from www. There HTTPS is not necessarily required and then could be seen as avoidable overhead. For this reason we do not configure HTTPS by default, but then allow to do this afterwards via dietpi-letsencrypt (CertBot + manual additions based on webserver / software used).

the unix_socket behaviour should be no problem. I didn't even notice the change and didn't have to adapt it to Stretch.

Yeah, on Jessie, with unix_socket installed/used, mysql/mariadb login fails, if a password is given. On Stretch it is just ignored. If a password is applied (mysql > ... identified by 'password'), then unix_socket is automatically disabled and user + password is needed from then on for this user. If the for my impression reasonable feature is wanted (preserved), then the only issue is, that no other user can login via mysql -uroot, thus sudo is required then or a new mysql user with admin permissions, matching the default login user name.

It is a different step, ( nc-nextcloud.sh "activation", meaning the configure() step ), so it would be easy to separate, but I wonder what's the benefit of removing it. Does DietPi also install postfix?

I like, and this goes along with the aim of DietPi, to have installs by default as clean/slim as possible to reduce SDcard writes, disk and RAM usage as much as possible. When manually choosing a certain option or feature, additional packages can be installed and configured on demand. E.g. I don't use my server as mail server, have no static IP anyway and don't use any other feature that requires an MTA. Thus I would see postfix just as unnecessary baggage 😉. I personally would add on demand installation and pre-configuration of postfix to an init script or hint when accessing certain ncp-config or web ui options respectively.

Redis is installed after LAMP, and before Nextcloud (same with Postfix), so I think we agree here. I didn't understand the problem with enabling/disabling the service.

Yeah, this is something where DietPi perhaps needs some rework to behaviour more usual. Most software services are disabled (systemctl disable ...) by DietPi and started/stopped by our dietpi-software script before and after maintenance/install tasks. That they are disabled (not autostarted by systemd) allows defined startups after all boot stages and in preferred order (databases before PHP before webservers, backends before frontends etc). But I was already thinking, if we could leave them enabled. The only effect is that on boot they are started by systemd according to unit dependencies. This should be no issue and it would be less confusing for end users, which find e.g. systemctl status mysql being disabled, but still starting up on boot.

In my Raspbian Stretch all three exist: mysql, mysqld, mariadb, so I am surprised. Are you in Stretch? In any case, no problem to change this.

I tried this on a Stretch VirtualBox VM. There /etc/init.d/mysql and systemd mariadb.service is available. On my RPi it is the same, but that one is on Raspbian Buster 😅, to might be different on default Raspbian Stretch. DietPi uses the init.d service, to support Jessie as well, as there is no systemd service available. Although I am thinking about adding the systemd service to Jessie instead 🤔.

Again, ATM only Stretch is supported. There are people starting to ask for PHP > 7.0, so one option would be to move to PHP 7.2 (by bringing sources from Buster to Stretch). That could potentially cause problems though, for instance that is what I did in Jessie to get PHP 7.0 support, but it would break php-smbclient.

Ah okay, Stretch-only, this is clear then. Yeah, I was also thinking about allowing PHP7 install on Stretch or generally allowing to install custom PHP version (via different repos) and apply settings to whatever /etc/php(5)[7.X] folder we find on the system. But this could lead to much additional bugreports/support/testing due to largely increased possible setup combinations. For now at least we stay with the default APT version each repo.

The ncp user is not a login user. It is only used to log in to ncp-web. I think it is better that is separated from the login user, and has no permissions whatsoever. In any case, this is part of ncp.sh

Jep, I thought so. I just though that you should handle it gracefully, if the user already exists, otherwise the script fails (again). Either is is trusted, that ncp must be the NextcloudPlus user from a prior installation or failed (my case) install attempt, or it is removed and recreated (to assure wanted creation options) or choice about this is left to user, to not remove/overwrite a user that way maybe created for another task 😃.

In lamp.sh, another thing that should be done is to stop modifying /etc/mysql/mariadb.conf.d/50-server.cnf and create separate files, like /etc/mysql/mariadb.conf.d/90-ncp.cnf. Same for php ini and apache files where possible.

👍 Jep, we also did this not long time ago.

We use some config injection function for this: https://github.com/Fourdee/DietPi/blob/testing/dietpi/func/dietpi-globals#L2223

Very nice thing to share, too bad you said "All mine!" :D :-)

we stay with the default APT version each repo.

Sounds very reasonable, a more generic depenency on "php" in NCP would allow installing on stable, testing and unstable, even if initially not supported and working yet, it will allow users to test and fix things.

@nachoparker ~~Maybe could you introduce a frugal install detection mechanism (not ncp-build?, not armbian?) and a option to install.sh that causes only the basic things to get installed.~~

@MichaIng Maybe could you add the backup creation to G_CONFIG_INJECT() in a way that allows it to also be used in https://github.com/nextcloud/nextcloudpi/blob/master/etc/library.sh (to only make a check first and allow skiping the corresponding ncp install step, if the configuration already exists)

With that basic infrastructure in place it could be possible to adapt and test NCP install.sh on different systems.

Does DietPi also install postfix?

I like... to have installs by default as clean/slim as possible

Maybe using a non-spooling, ram-only forwarder is all that is needed to send out notifications. (Not sure msmtp, esmtp, or ssmtp?)

Edit: May require storing smtp login on the device?

I guess the status that some service or setting was already found (preinstalled) initally will have to get saved, to later skip any corresponding updates.

Is there some specific way you would imagine this to work? Append lines naming preinstalled or external options to some file?

I have made some things optional in the https://github.com/testbird/nextcloudpi/ branch, and after php-redis is installed manually from sid (because it is currently not available in buster) the installer is now able to run through in the freedombox debian-testing image. (https://ftp.freedombox.org/pub/freedombox/testing-latest/freedombox-testing-free_latest_all-amd64.vdi.xz)

To test the adaptions in a debian testing dev image:

wget http://ftp.debian.org/debian/pool/main/p/php-redis/php-redis_3.1.6-1+b1_amd64.deb
dpkg -i php-redis_3.1.6-1+b1_amd64.deb
apt install -f
curl -sSL https://raw.githubusercontent.com/testbird/nextcloudpi/master/install.sh | bash

However, I am not able to find out why it broke the web access. I don't know enough how all the parts are interrelated. Would be great if you could improve on my patches.

@MichaIng It would be really great if you could find the time to update your G_CONFIG_INJECT() with prior-state tests and backup keeping of the prior and the configured state, in a way that it could also be used in the NCP installer.

Edit: basic pointer https://github.com/nextcloud/nextcloudpi/wiki/ncp-app-developer-guide

nginx

One of the reasons why I use Apache is because of mod_security. This is harder to install in nginx. I definitely don't have the manpower for supporting three different LAMP stacks, but if I had more help it would be doable. For the same reason, I only support Stretch, and again if there was someone adapting the system to Jessie I would gladly merge that.

HTTPS

In the days of insecure IoT I believe that even in your local network you need HTTPS. Maybe you can skip signing your certificates, but HTTPS is a must. In any case, both Letsencrypt and HTTPS are optional in NCP, but by default NCP works with HTTPS only and you have to disable that manually, which I think it's the right thing to do in this case.

Postfix

I add postfix because that way the user receives notifications about shared files and other activity in their cloud. It is quite standard in Nextcloud and I think it is nice to include it. It is true that it is a bit heavy for what it does. I would be happy to replace but something lighter, we don't need a full fledged email server to send some informative emails.

Again here our projects have different goals. NC aims to be an all-included ready to use solution, and tries to minimize the amount of obscure steps that a non-technical user has to go through. Most people don't even know what postfix is, but they appreciate that the system sends them emails automatically out of the box. That being said, it could be a good improvement to have some option to disable it, and maybe install it separatedly from NC itself (in a separate step).

PHP

For now at least we stay with the default APT version each repo.

Agreed.

NCP user

Agreed, we should detect this, and probably just throw a big warning in case it already exists. I created https://github.com/nextcloud/nextcloudpi/issues/579

Config modularity

Created https://github.com/nextcloud/nextcloudpi/issues/580

More robust installer

In order to be able to run in systems that are not under our control, it would be nice to add a few functions to the NCP library and use that during build/update/installation. We can take the config injection function from DietPi, and potentially others as well.

In the past I wanted each single .sh file to be as self contained as possible, so I was trying to avoid sourcing other files, but in the new situation at hand I think it would be a nice move. This would make the code more readable as it is full of seds and greps, we could log actions better, backup configs and more.

https://github.com/nextcloud/nextcloudpi/issues/581

I think that @testbird agrees on this ;)

Sadly I won't have both time and a computer for a few weeks, but it is good that we start creating items. Feel free to open other ones in NCP github.

BTW: feel free to join our Telegram group, it's quite active and we share feedback and ideas.

https://t.me/NextCloudPi

Updated G_CONFIG_INJECT: https://github.com/Fourdee/DietPi/pull/2042

Added grep error catching of the if statements. This is needed to prevent e.g. adding setting to end of file, if grep checks fail due to error instead of string not found.
Added BACKUP=1 handling to create backup /path/to/file.bak before editing the file.

Only issue with direct NCP implementation is that it uses G_DIETPI-NOTIFY and G_WHIP_MSG, which would need to be replaced by echo -e and whiptail --msg.

Feel free to use and adjust for own needs of course. If you know the settings you want to add, all the error handling could be skipped, to have a lightweight version. I added this just, since we use the function much throughout DietPi and source it to current terminal session, so end users can use it as well as global function. Errors can happen if []$"{}() and such are not escaped properly (see function comments), also since extended regular expressions are handled (grep/sed -E).

Is PHP 7 coming soon? Nextcloud 14 requires PHP 7 so checking. Thanks.

@kevkha

Is PHP 7 coming soon?

We offer Debian Stretch images, thus PHP7, for all our supported devices, besides Odroid C1.

Nextcloud 14 requires PHP 7

Are you sure??? Can't believe it, haven't heard of it and it would break compatibility for a very large user space. Do you have a link for this info? In case this would mean that we need to manually install NC13 on Jessie images and inform users that they can't use Nextcloud 14. Flash new Stretch image would be needed otherwise, or manual steps to dist-upgrade to Debian Stretch or install PHP7 🤔.

https://github.com/nextcloud/server/commit/ace96a406a4fea0dcc07abd003495da34b5fa71c Indeed PHP5 support dropped with Nextcloud 14. Not sure what is best to do. I hope on Jessie/PHP5 systems, WebUI update attempt shows meaningful output and is prevented, so we can resolve it by forcing NC13 on Jessie. Otherwise we need to inform Jessie users that they shall not update and remove Nextcloud install option for Jessie to avoid unexpected issues. Test needed. @Fourdee This should be fixed/worked around before v6.15 release, as Nextcloud 14 is already released. Today official launch is planned, a fresh install already pulls v14, just the WebUI upgrade is not yet offered in my case.

indeed a situation. Until when are you guys going to support Jessie? I guess you will have to support two versions of NC :S

in NCP due to shortage of manpower we only support the latest stable debian

@nachoparker At least with PHP5 in use, NC14 update is not offered and we do no NC14-only install steps. So we just need to download/install the latest NC13 on Jessie systems.

The reason to still support Jessie is, besides existing Jessie users from older times, that the Odroid Stretch pre-image we use does not have full GPU support to play e.g. Kodi smoothly. And Kodi is one of the more used software on DietPi systems. Also we have no Stretch image for Odroid C1 available :thinking:. We can just hope that Meveric finds a fix for his Odroid Stretch images soon or have to disappoint many of our Odroid users, sooner or later.

@MichaIng Yes, pretty sure. I received a recommendation from my previous NC 13.0.6 upgrade. More details here https://nextcloud.com/blog/changes-for-developers-in-nextcloud-14-beta-4-ready-for-testing/

@kevkha Thanks for additional verification. Solution from our side will be as above to simply install https://download.nextcloud.com/server/releases/latest-13.zip on Jessie systems. Damn, wanted to add it to my open PR today, but simply to busy with other stuff... Tomorrow, at latest on Thursday it will be finished, including some other enhancements. I hope Fourdee has enough patience to wait with v6.15 release for this, to prevent Jessie users run into unusable Nextcloud installs.

I am still surprised by the decision to drop PHP5.6 support, as e.g. all the RHEL users already had problems to get PHP5.6 when PHP5.4 support was dropped. But the step to PHP7.0 is indeed a larger one, if one wants to stick with LTS distros and their native software repos.

Too bad for boards with shitty support. IIRC all Armbian boards NCP supports are on Stretch already

Another thought. Switching to use docker container https://ownyourbits.com/2017/06/08/nextcloudpi-docker-for-raspberry-pi/

@nachoparker Odroids are actually well supported. Sadly official images just include Ubuntu and Android 😭. But one developer Meveric offers clean Debian images, including an own large APT software and kernel repository with regular updates.

The missing Kodi (rechecked, it's really just Kodi, Chromium GPU acceleration works) GPU support is the left Stretch-only issue I am aware of.

@kevkha We offer docker and everyone is free to install docker containers of course, but we will not add docker container installs to dietpi-software or take special care about it. DietPi tries to integrate software hierarchically flat into the system. Docker encapsulates software and has a hierarchically vertical system structure. This has great benefits, e.g. avoiding the need to take care of compatibility and dependencies of different software titles, but the abstraction/encapsulation also has disadvantages of multiple resource (e.g. drive space, ram) usage and in some areas less performance (although not 100% sure which areas, also in comparison to snaps and such). Configuration abilities are mostly reduced as well.

I mark this issue as closed. I think it has been discussed enough that both projects have their own stranges and aims and should be seen as either/or for now. Making DietPi(-Software) more flexible and compatible with 3rd party installers is a general long term aim that is followed by many small steps with every DietPi update. I'll have a look back at NextCloudPi, as I am following it anyway with great interest, so as fast as we can think of an integration, we'll open it as new software request.

Aside from that, the NextCloudPi installer CAN run on DietPi, it is just the DietPi-Software installs and things like dietpi-letsencrypt, that currently assume and/or establish an incompatible webserver setup, which must be kept in mind from then on.

MichaIng / DietPi