Usage of Apps with bundled nginx

[pc-fan]

If I install for example the owncloud calender app and try to access the calendar (at index.php/apps/calendar/) several ressource could not be loaded. The the containers are set up like described in the quickstart section of the README. The /var/log/nginx/error.log of the owncloud-nginx container shows that the apps folder in the nginx container does not contain the installed apps, e.g.:

*1116 open() "/var/www/owncloud/apps/calendar/css/style.css" failed (2: No such file or directory)
*1140 open() "/var/www/owncloud/apps/calendar/js/on-event.js" failed (2: No such file or directory)
*1142 open() "/var/www/owncloud/apps/calendar/js/settings.js" failed (2: No such file or directory)
*1144 open() "/var/www/owncloud/apps/calendar/js/share.js" failed (2: No such file or directory)

It seems like the apps are only installed inside the fpm container.

File listing in the fpm container:

root@b2a13359bab9:/var/www/owncloud# ls apps
activity  contacts    external  files_external   files_sharing     files_trashbin  files_videoviewer  gallery        provisioning_api  updater        user_ldap
calendar  encryption  files     files_pdfviewer  files_texteditor  files_versions  firstrunwizard     notifications  templateeditor    user_external

File listing in the nginx container:

root@02ee105a58b2:/var/www/owncloud# ls apps
activity  encryption  external  files  files_external  files_pdfviewer  files_sharing  files_texteditor  files_trashbin  files_versions  files_videoviewer  firstrunwizard  gallery  notifications  provisioning_api  templateeditor  updater  user_external  user_ldap

Maybe the apps directory could become a volume or a part of the /var/lib/owncloud volume and thus be shared between the fpm and the nginx container? The nginx container could then be started using the --volumes-from owncloud directive.

[pc-fan]

It is even possible to specify more than one apps directory, see https://doc.owncloud.org/server/8.2/admin_manual/installation/apps_management_installation.html#using-custom-app-directories So i think the default directory could be kept read-only with the default apps and a second apps directory could be placed at /var/lib/owncloud/apps.

[sameersbn]

@pc-fan your findings are absolutely spot-on.

You mentioned the use of --volumes-from owncloud, this techinically can already be done. While starting the owncloud container add -v /var/lib/owncloud and use --volumes-from owncloud while starting the nginx container. This should work without any changes, but as you can tell this approach is very messy.

I was considering starting nginx and fpm in a single container instead of starting two containers for each of them. fpm and nginx should be tightly coupled so this isolation does not make complete sense and results in issues like the one you have reported. It would also make the startup scripts a lot less confusing :smile:

To achieve this i would need to use a supervisor (eg. supervisord) to manage launching the nginx and fpm processes. At the moment this is not very high on my priority list.

It is even possible to specify more than one apps directory

Thanks for this information. I was not aware of this option

[pc-fan]

If my understanding is correct, the apps currently are not located inside the /var/lib/owncloud dir and thereby at the moment are not included in the volume.

The long term plans about using supervisord sound good. As I am not yet familiar enough with supervisord to try starting implementing this, I created #5 as an attempt to make it possible to use apps in the current setup. Maybe you could have a look @sameersbn? Currently im kind of stuck, as I am not able to find out why the config.php does not get updated in the way I excpected, i described the problem in the pull request. Thank you in advance!

[r2evans]

On the discussion of allowing Custom App Directories, would you be interested in a PR to support it?

Two options: always make a localapps dir, and allow the user to decide if they want a volume; or allow an optional custom app dir. The former is straight-forward to implement, it just takes you deciding if you want it and then hard-coding some stuff in the config.php. I'll assume the latter, making it an option.

Here's my thinking:

Currently, there is no explicit setting of apps_paths in the config.php. To do so with your current get/set functions can be done safely iff the value is always on a single line (which it is not), otherwise your sed script does not get it all. To do so, we'd need a method for intelligently parsing the $CONFIG array. Since this is currently not direct (php occ config:list requires the database be already up and available, for example), less-direct methods might work: JSON, though having more than we need, is something we can easily parse both in php (json_encode and json_decode) and bash (either JSON.sh or jq).

For instance, I can do the following (lacking plenty of checks that should be in the code):

<?php
# getter
$n = count($argv);
if ($n > 1) {
    include $argv[1];
    $obj = ($n > 2) ? $CONFIG[$argv[2]] : $CONFIG;
    echo json_encode($obj, JSON_UNESCAPED_SLASHES) . "\n";
}

allows me to do the following:

$ php getter.php config.php datadirectory
"/var/www/owncloud/data"

$ php getter.php config.sample.php apps_paths
[{"path":"/var/www/owncloud/apps","url":"/apps","writable":true}]

$ php getter.php config.sample.php
{"instanceid":"","passwordsalt":"","trusted_domains":["demo.example.org","otherdomain.example.org"],"datadirectory":"/var/www/owncloud/data","version":"","dbtype":"sqlite","dbhost":"","dbname":"owncloud","dbuser":"","dbpassword":"","dbtableprefix":"","installed":false,"default_language":"en","defaultapp":"files","knowledgebaseenabled":true,"enable_avatars":true,"allow_user_to_change_display_name":true,"remember_login_cookie_lifetime":1296000,"session_lifetime":86400,"session_keepalive":true,"skeletondirectory":"/path/to/owncloud/core/skeleton","user_backends":[{"class":"OC_User_IMAP","arguments":["{imap.gmail.com:993/imap/ssl}INBOX"]}],"lost_password_link":"https://example.org/link/to/password/reset","mail_domain":"example.com","mail_from_address":"owncloud","mail_smtpdebug":false,"mail_smtpmode":"sendmail","mail_smtphost":"127.0.0.1","mail_smtpport":25,"mail_smtptimeout":10,"mail_smtpsecure":"","mail_smtpauth":false,"mail_smtpauthtype":"LOGIN","mail_smtpname":"","mail_smtppassword":"","overwritehost":"","overwriteprotocol":"","overwritewebroot":"","overwritecondaddr":"","overwrite.cli.url":"","proxy":"","proxyuserpwd":"","trashbin_retention_obligation":"auto","versions_retention_obligation":"auto","appcodechecker":true,"updatechecker":true,"has_internet_connection":true,"check_for_working_webdav":true,"check_for_working_wellknown_setup":true,"check_for_working_htaccess":true,"config_is_read_only":false,"log_type":"owncloud","logfile":"/var/log/owncloud.log","loglevel":2,"syslog_tag":"ownCloud","log.condition":{"shared_secret":"57b58edb6637fe3059b3595cf9c41b9","users":["sample-user"],"apps":["files"]},"logdateformat":"F d, Y H:i:s","logtimezone":"Europe/Berlin","log_query":false,"cron_log":true,"cron.lockfile.location":"","log_rotate_size":false,"3rdpartyroot":"","3rdpartyurl":"","customclient_desktop":"http://owncloud.org/sync-clients/","customclient_android":"https://play.google.com/store/apps/details?id=com.owncloud.android","customclient_ios":"https://itunes.apple.com/us/app/owncloud/id543672169?mt=8","appstoreenabled":true,"appstoreurl":"https://api.owncloud.com/v1","appstore.experimental.enabled":false,"apps_paths":[{"path":"/var/www/owncloud/apps","url":"/apps","writable":true}],"enable_previews":true,"preview_max_x":2048,"preview_max_y":2048,"preview_max_scale_factor":10,"preview_max_filesize_image":50,"preview_libreoffice_path":"/usr/bin/libreoffice","preview_office_cl_parameters":" --headless --nologo --nofirststartwizard --invisible --norestore -convert-to pdf -outdir ","enabledPreviewProviders":["OC\\Preview\\PNG","OC\\Preview\\JPEG","OC\\Preview\\GIF","OC\\Preview\\BMP","OC\\Preview\\XBitmap","OC\\Preview\\MP3","OC\\Preview\\TXT","OC\\Preview\\MarkDown"],"ldapUserCleanupInterval":51,"comments.managerFactory":"\\OC\\Comments\\ManagerFactory","systemtags.managerFactory":"\\OC\\SystemTag\\ManagerFactory","maintenance":false,"singleuser":false,"openssl":{"config":"/absolute/location/of/openssl.cnf"},"enable_certificate_management":false,"memcache.local":"\\OC\\Memcache\\APCu","memcache.distributed":"\\OC\\Memcache\\Memcached","redis":{"host":"localhost","port":6379,"timeout":0,"password":"","dbindex":0},"memcached_servers":[["localhost",11211]],"cache_path":"","objectstore":{"class":"OC\\Files\\ObjectStore\\Swift","arguments":{"username":"facebook100000123456789","password":"Secr3tPaSSWoRdt7","container":"owncloud","autocreate":true,"region":"RegionOne","url":"http://8.21.28.222:5000/v2.0","tenantName":"facebook100000123456789","serviceName":"swift","urlType":"internal"}},"sharing.managerFactory":"\\OC\\Share20\\ProviderFactory","dbdriveroptions":{"1012":"/file/path/to/ca_cert.pem"},"sqlite.journal_mode":"DELETE","supportedDatabases":["sqlite","mysql","pgsql","oci"],"tempdirectory":"/tmp/owncloudtemp","hashingCost":10,"blacklisted_files":[".htaccess"],"share_folder":"/","theme":"","cipher":"AES-256-CFB","minimum.supported.desktop.version":"1.7.0","quota_include_external_storage":false,"filesystem_check_changes":0,"part_file_in_storage":true,"asset-pipeline.enabled":false,"assetdirectory":"/var/www/owncloud","mount_file":"/var/www/owncloud/data/mount.json","filesystem_cache_readonly":false,"secret":"","trusted_proxies":["203.0.113.45","198.51.100.128"],"forwarded_for_headers":["HTTP_X_FORWARDED","HTTP_FORWARDED_FOR"],"max_filesize_animated_gifs_public_sharing":10,"filelocking.enabled":true,"memcache.locking":"\\OC\\Memcache\\Redis","debug":false,"copied_sample_config":true}

Similarly:

<?php
# setter
$n = count($argv);
if ($n == 4) {
    include $argv[1];
    $CONFIG[$argv[2]] = json_decode($argv[3]);
    $v = "<?php\n\$CONFIG = " . var_export($CONFIG, true) . ";\n";
    file_put_contents($argv[1], $v);
}

allows us to do:

$ cp config.sample.php config2.php

$ php setter.php config2.php datadirectory '"/foo/bar"'

$ head config2.php
<?php
$CONFIG = array (
  'instanceid' => '',
  'passwordsalt' => '',
  'trusted_domains' =>
  array (
    0 => 'demo.example.org',
    1 => 'otherdomain.example.org',
  ),
  'datadirectory' => '/foo/bar',

This method is a little less fragile than the current method of using sed, which though great for single-line settings, does not as easily deal with multi-line configuration items (e.g., arrays). Doing this would require changing the current get/set functions, but it should only need to be done once.

From here, allowing setting an alternative apps directory involves grabbing the current value (which is likely a list of one), appending to it, then writing it back to the file. Using JSON.sh:

$ php getter.php config2.php apps_paths | ./JSON.sh
[0,"path"]      "/var/www/owncloud/apps"
[0,"url"]       "/apps"
[0,"writable"]  true
[0]     {"path":"/var/www/owncloud/apps","url":"/apps","writable":true}
[]      [{"path":"/var/www/owncloud/apps","url":"/apps","writable":true}]

$ php getter.php config2.php apps_paths | ./JSON.sh | grep '\[[0-9]\]'
[0]     {"path":"/var/www/owncloud/apps","url":"/apps","writable":true}

and internally append a manually-combined string, perhaps something like:

$ php getter.php config2.php apps_paths | ./JSON.sh \
>     | sed -ne 's/^\[\][[:space:]]*//p' \
>     | sed -e 's/"writable":true/"writable":false/g' \
>           -e 's|\]$|,{"path":"/var/www/owncloud/'${OWNCLOUD_LOCALAPPS}'","url":"/'${OWNCLOUD_LOCALAPPS}'","writable":true}]|g'
[{"path":"/var/www/owncloud/apps","url":"/apps","writable":false},{"path":"/var/www/owncloud/apps2","url":"/apps2","writable":true}]

Thoughts?

[r2evans]

A quick fix to the original problem (using nextcloud instead of owncloud):

... the apps folder in the nginx container does not contain the installed apps.

I edited docker-nextcloud/Dockerfile so that the exported volumes included the install dir:

VOLUME ["${NEXTCLOUD_DATA_DIR}","${NEXTCLOUD_INSTALL_DIR}"]

and in my docker-compose.yml, I added to the nginx section:

  nginx:
    #...
    volumes_from:
    - nextcloud

After that, any reference to files within newly-installed apps should be resolved without error.

Side note: this means that the nginx image in the docker-compose.yml no longer needs to inherit from nextcloud. I have not yet changed my compose file to use sameersbn/docker-nginx yet, but there should technically be no reason it should not work. (With the same nginx service in this compose yaml, I'm also importing an exported volume from a LTB self-service password php-fpm image I built; now the nginx process it inheriting from both and it works great. My config has each of them in subdirs so there are no file collisions.)

@pc-fan, did you find a better (or different) solution that met your needs?