owncloud / core

:cloud: ownCloud web server core (Files, DAV, etc.)
https://owncloud.com
GNU Affero General Public License v3.0
8.37k stars 2.06k forks source link

OC-8.1.3 Update: External Storage Support (Local) Stopped Working #19283

Closed kneiser closed 8 years ago

kneiser commented 9 years ago

Is there a workaround to get this to work or do we have to wait for some time for it to be fixed?

karlitschek commented 9 years ago

can you post a full bug report as described here please? https://github.com/owncloud/core/blob/master/CONTRIBUTING.md

kneiser commented 9 years ago

Here is the issue filled out. If more information required contact me please.

Steps to reproduce

  1. Go to admin user "adminstration" and setup external storage as indicated below. Green light appears once configured, as always.
  2. Admin user goes to Files app, no external storage "shrMedia" folder. Regular ownCloud user has same experience.
  3. "External Storage" tag on left does display "shrMedia" folder, but does not work.

Expected behaviour

When user clicks on the "shrMedia" folder the folder list in the local external storage should appear. User tries to access the "External Storage" tag in Files, shrMedia folder appears, the click on the "shrMedia" folder, and the folder list in the local external storage should appear.

Actual behaviour

User logs in to ownCloud and the "shrMedia" folder is not visible in the Files app. Users tries to access the "External Storage" tag in Files, shrMedia folder appears, click on it and the Files home page is displayed not the contents of the shrMedia folder. Same happens for 'admin' user.

Server configuration

Operating system: Fedora 22 Web server: Apache 2.4.16 Database: MariaDB 10.0.21 PHP version: PHP 5.6.13 ownCloud version: (see ownCloud admin page) ownCloud 8.1.3 Updated from an older ownCloud or fresh install: ownCloud 8.1.1 List of activated apps: Enabled:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder

The content of config/config.php: <?php $CONFIG = array ( 'instanceid' => 'oc743916ebbb', 'passwordsalt' => '**', 'secret' => '***_', 'trusteddomains' => array ( 0 => '***', 1 => '***', ), 'datadirectory' => '/var/files/owncloud/data', 'overwrite.cli.url' => 'http://***/owncloud', 'dbtype' => 'mysql', 'version' => '8.1.3.0', 'dbname' => 'owncloud', 'dbhost' => 'localhost', 'dbtableprefix' => 'oc', 'dbuser' => '*****', 'dbpassword' => '_**', 'installed' => true, 'updatechecker' => true, 'has_internet_connection' => true, 'theme' => '', 'trashbin_auto_expire' => true, 'trashbin_retention_obligation' => 14, 'memcache.local' => '\OC\Memcache\APCu', 'maintenance' => false, 'appstore.experimental.enabled' => false, );

Note: Private information replaced with "***'s".

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system --public
from within your ownCloud installation folder

or 

Insert your config.php content here
(Without the database password, passwordsalt and secret)

Are you using external storage, if yes which one: local/smb/sftp/... local Configuration setup in admin page: Folder Name: shrMedia External Storage: Local Configuration: /var/files/media Available for: mediaRWGrp.

Are you using encryption: yes/no no Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... no

Client configuration

Browser: Firefox 40.0.30 Operating system: Fedora 22, Windows 7

Logs

Web server error log

Insert your webserver log here

[Wed Sep 23 12:44:38.604672 2015] [mpm_prefork:notice] [pid 681] AH00170: caught SIGWINCH, shutting down gracefully [Wed Sep 23 12:45:18.498901 2015] [core:notice] [pid 684] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0 [Wed Sep 23 12:45:18.615580 2015] [suexec:notice] [pid 684] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Wed Sep 23 12:45:18.656698 2015] [auth_digest:notice] [pid 684] AH01757: generating secret for digest authentication ... [Wed Sep 23 12:45:18.657315 2015] [lbmethod_heartbeat:notice] [pid 684] AH02282: No slotmem from mod_heartmonitor [Wed Sep 23 12:45:23.268567 2015] [mpm_prefork:notice] [pid 684] AH00163: Apache/2.4.16 (Fedora) PHP/5.6.13 configured -- resuming normal operations [Wed Sep 23 12:45:23.268594 2015] [core:notice] [pid 684] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

ownCloud log (data/owncloud.log)

Insert your ownCloud log here

{"reqId":"VgL0B4J3vbL09Ap14n0kKAAAAAE","remoteAddr":"10.25.3.121","app":"files_sharing","message":"Unable to get mount for shared storage '4' user 'admin'","level":3,"time":"2015-09-23T18:48:39+00:00"} {"reqId":"VgL0B4J3vbL09Ap14n0kKAAAAAE","remoteAddr":"10.25.3.121","app":"files_sharing","message":"Unable to get mount for shared storage '4' user 'admin'","level":3,"time":"2015-09-23T18:48:39+00:00"} {"reqId":"VgL0Qqn2cXkMfPcwJrLO2gAAAAo","remoteAddr":"10.25.3.121","app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-09-23T18:49:39+00:00"}

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log 
c) ...

This seems to be difficult to find. Is it important in this case?

RobinMcCorkell commented 9 years ago

I noticed you appear to be using suexec. Perhaps this is changing the user that ownCloud runs as? Although if you've set the file mode to 777, that can't be it...

The problem is very likely to be that files_sharing issue. cc @PVince81

eusonlito commented 9 years ago

Same here. I have a external disk mounted and added to OwnCloud (and used without problem by btsync) but owncloud say that is empty.

Steps to reproduce

  1. Added an existing local folder as "External Storage". This folder is mounted from external drive and working.
  2. OwnCloud say that contents is empty. 3.

Expected behaviour

Navigate using the web UI.

Actual behaviour

Show a folder icon and meesage "No files in here".

Server configuration

Operating system:

Distributor ID: Ubuntu
Description:    Ubuntu 14.04.3 LTS
Release:    14.04
Codename:   trusty

Web server:

nginx version: nginx/1.4.6 (Ubuntu)

Database:

Server version: 5.5.44-0ubuntu0.14.04.1 (Ubuntu)

PHP version:

fastcgi_pass unix:/var/run/php5-fpm.sock;

PHP 5.5.9-1ubuntu4.13 (cli) (built: Sep 29 2015 15:47:36) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies

ownCloud version: (see ownCloud admin page)

ownCloud 8.1.3 (stable)

Updated from an older ownCloud or fresh install:

fresh install from ppa

deb http://download.opensuse.org/repositories/isv:/ownCloud:/community/xUbuntu_14.10/ /

List of activated apps:

Enabled:
 - activity: 2.0.2
 - files: 1.1.10
 - files_external: 0.2.3
 - files_pdfviewer: 0.7
 - files_trashbin: 0.6.3
 - files_videoviewer: 0.1.3
 - gallery: 0.6.0
 - updater: 0.6
Disabled:
 - encryption
 - external
 - files_locking
 - files_sharing
 - files_texteditor
 - files_versions
 - firstrunwizard
 - provisioning_api
 - templateeditor
 - user_external
 - user_ldap
 - user_webdavauth

The content of config/config.php:

# sudo -u www-data php occ config:list system --public

  [InvalidArgumentException]                                
  There are no commands defined in the "config" namespace.  
<?php
$CONFIG = array (
  'instanceid' => 'XXXXXXXXXXXXXXXX',
  'passwordsalt' => 'XXXXXXXXXXXXXXXX',
  'secret' => 'XXXXXXXXXXXXXXXX',
  'trusted_domains' =>
  array (
    0 => 'owncloud.XXXXXXXXXXXXXXXX',
  ),
  'datadirectory' => '/var/www/owncloud/data',
  'overwrite.cli.url' => 'https://owncloud.XXXXXXXXXXXXXXXX',
  'dbtype' => 'mysql',
  'version' => '8.1.3.0',
  'dbname' => 'owncloud',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'XXXXXXXXXXXXXXXX',
  'dbpassword' => 'XXXXXXXXXXXXXXXX',
  'logtimezone' => 'Europe/Madrid',
  'memcache.local' => '\OC\Memcache\APC',
  'installed' => true,
);

Are you using external storage, if yes which one: local/smb/sftp/...

Yes: local

Are you using encryption: yes/no

No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

No

LDAP configuration (delete this part if not used)

Client configuration

Browser:

Any, but Version 46.0.2490.71 (64-bit) as default

Operating system:

Ubuntu

Logs

Web server error log

# tail /var/log/nginx/error.log
2015/10/20 11:19:32 [error] 1029#0: *122 upstream timed out (110: Connection timed out) while reading response header from upstream, client: XXXXXXXXXXXXXXXX, server: owncloud.XXXXXXXXXXXXXXXX, request: "GET /index.php/apps/files/?dir=%2Fbtsync HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "owncloud.XXXXXXXXXXXXXXXX"
2015/10/20 15:14:15 [error] 1029#0: *377 FastCGI sent in stderr: "PHP message: PHP Warning:  apc_store(): GC cache entry '5993e49134555c1232fdc9906f7f9e48/AutoloaderOC_User' was on gc-list for 13713 seconds in /var/www/owncloud/lib/private/memcache/apc.php on line 45
PHP message: PHP Warning:  apc_store(): GC cache entry '5993e49134555c1232fdc9906f7f9e48/AutoloaderOC\Group\Manager' was on gc-list for 13713 seconds in /var/www/owncloud/lib/private/memcache/apc.php on line 45
PHP message: PHP Warning:  apc_store(): GC cache entry '5993e49134555c1232fdc9906f7f9e48/AutoloaderOC\Log\ErrorHandler' was on gc-list for 13713 seconds in /var/www/owncloud/lib/private/memcache/apc.php on line 45" while reading response header from upstream, client: XXXXXXXXXXXXXXXX, server: owncloud.XXXXXXXXXXXXXXXX, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "owncloud.XXXXXXXXXXXXXXXX"

ownCloud log (data/owncloud.log)

{"reqId":"1yH24w7Wj18nIO0Y96KX","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-13T09:23:14+02:00"}
{"reqId":"q7Twb4pvwrqHGT9VqX08","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-14T13:27:36+02:00"}
{"reqId":"XIm\/0rXb5DezVZJJMhri","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-14T23:05:46+02:00"}
{"reqId":"3dJrHVU6I0mdpslcnqx+","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-15T00:14:05+02:00"}
{"reqId":"5Bhi6qKvTpKcZ281IThe","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-15T00:15:55+02:00"}
{"reqId":"oiTKNqHrLcRk+kt3KJy\/","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-20T11:06:59+02:00"}
{"reqId":"nC9Ovj8lQWrDpEEI7Mb+","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-20T11:23:58+02:00"}
{"reqId":"OCO5kKlZR0qiC3JVVXrv","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"Array to string conversion at \/var\/www\/owncloud\/lib\/private\/template\/functions.php#36","level":3,"time":"2015-10-20T11:24:20+02:00"}
{"reqId":"COfNRQfmGeYrmh71GcwH","remoteAddr":"","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/owncloud\/lib\/private\/util.php:438) at \/var\/www\/owncloud\/lib\/private\/response.php#254","level":3,"time":"2015-10-20T11:30:02+02:00"}
{"reqId":"COfNRQfmGeYrmh71GcwH","remoteAddr":"","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/owncloud\/lib\/private\/util.php:438) at \/var\/www\/owncloud\/lib\/private\/response.php#259","level":3,"time":"2015-10-20T11:30:02+02:00"}
{"reqId":"COfNRQfmGeYrmh71GcwH","remoteAddr":"","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/owncloud\/lib\/private\/util.php:438) at \/var\/www\/owncloud\/lib\/private\/response.php#260","level":3,"time":"2015-10-20T11:30:02+02:00"}
{"reqId":"COfNRQfmGeYrmh71GcwH","remoteAddr":"","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/owncloud\/lib\/private\/util.php:438) at \/var\/www\/owncloud\/lib\/private\/response.php#261","level":3,"time":"2015-10-20T11:30:02+02:00"}
{"reqId":"COfNRQfmGeYrmh71GcwH","remoteAddr":"","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/owncloud\/lib\/private\/util.php:438) at \/var\/www\/owncloud\/lib\/private\/response.php#262","level":3,"time":"2015-10-20T11:30:02+02:00"}
{"reqId":"YLthefTw1cDOuaWjewGC","remoteAddr":"XXXXXXXXXXXXXXXX","app":"PHP","message":"apc_store(): GC cache entry '5993e49134555c1232fdc9906f7f9e48\/AutoloaderOC\\Log\\ErrorHandler' was on gc-list for 13713 seconds at \/var\/www\/owncloud\/lib\/private\/memcache\/apc.php#45","level":3,"time":"2015-10-20T15:14:15+02:00"}
{"reqId":"TyMDWzAyNyGkQXX8qrsT","remoteAddr":"XXXXXXXXXXXXXXXX","app":"core","message":"Login failed: 'root' (Remote IP: 'XXXXXXXXXXXXXXXX)","level":2,"time":"2015-10-20T15:14:22+02:00"}

Browser log

Insert your browser log here, this could for example include:

At home, Ajax:

https://owncloud.XXXXXXXXXXXXXXXX/index.php/apps/files/ajax/list.php?dir=%2F&sort=name&sortdirection=asc

{"data":{"directory":"\/","files":[{"id":10,"parentId":"-1","date":"October 14, 2015 at 1:42:41 PM GMT+2","mtime":1444822961000,"icon":"\/core\/img\/filetypes\/folder.svg","name":"btsync","permissions":17,"mimetype":"httpd\/unix-directory","size":0,"type":"dir","etag":"561eb6141b71e"}],"permissions":31},"status":"success"}
Into btsync folder

https://owncloud.XXXXXXXXXXXXXXXX/index.php/apps/files/ajax/list.php?dir=%2Fbtsync&sort=name&sortdirection=asc

{"data":{"directory":"\/btsync","files":[],"permissions":17},"status":"success"}

https://owncloud.XXXXXXXXXXXXXXXX/index.php/apps/files/ajax/getstoragestats.php?dir=%2Fbtsync

{"data":{"uploadMaxFilesize":2097152,"maxHumanFilesize":"Upload (max. 2 MB)","freeSpace":296189239296,"usedSpacePercent":0,"owner":"root","ownerDisplayName":"root"},"status":"success"}

More details

$ mount
/dev/sda1 on /home/btsync/sync type ext4 (rw,nosuid,nodev,noatime,_netdev)

$ ls -al /home
drwxr-xr-x  3 btsync btsync 4096 Oct 14 13:16 btsync

$ ls -al /home/btsync
drwxr-xr-x  5 btsync btsync 4096 Oct 14 13:42 sync

$ ls -al /home/btsync/sync
drwxr-xr-x  6 btsync btsync 4096 Oct 14 13:42 backup
drwxr-xr-x  6 btsync btsync 4096 Oct 14 13:42 mobile

Folder added with name btsync and path /home/btsync/sync.

Cronjobs are working with Cron option: Cron Last cron job execution: 15 minutes ago.

kneiser commented 9 years ago

I just upgraded from ownCloud 8.1.3 to 8.2.0, same issue as before.

kneiser commented 8 years ago

I just updated one of my LAMP servers to ownCloud 8.1.4. External Storage is working again! Now we just have to see if 8.2.x will also have a fix.

PVince81 commented 8 years ago

@kneiser usually fixes are first applied to the latest version (for ex: 8.2) and backported to 8.1.4. If you say that your setup works fine with 8.1.4 but not 8.2, it is likely that there is something different going on. Another possibility is that something else broke it again in 8.2 (regression)

kneiser commented 8 years ago

i just update my ownCloud 8.2.0 LAMP test server to 8.2.1. This bug still exists in this new release. As stated before the v8.1.4 release works. Below is the owncloud.log file error from the v8.2.1 test instance: {"reqId":"VgL0B4J3vbL09Ap14n0kKAAAAAE","remoteAddr":"10.25.3.121","app":"files_sharing","message":"Unable to get mount for shared storage '4' user 'admin'","level":3,"time":"2015-09-23T18:48:39+00:00"} The owncloud logreader app shows the following: Level App Message Time Error files_sharing Unable to get mount for shared storage '4' user 'admin' a few seconds ago

PVince81 commented 8 years ago

Can you describe the way you set up the local storage and also the shares you used ?

I see you have a folder called "shrMedia" and that error message mentions a shared storage, which seems to imply that you might have shared a folder from inside the "Local" mount point ? This could maybe help reproducing the issue.

You could also try to unshare, then share again that folder.

kneiser commented 8 years ago

@PVince81 i am not sure how you can "unshare" an external storage support setup. i did look as you suggested, no sign of "unshare". I did however teardown the external storage support setup and removed the security group. then i used occ to rescan all files for all users. then setup external storage again with the security group. used occ again. same issue. i will post some screenshots later of my setup so that you can attempt to replicate the problem.

kneiser commented 8 years ago

Hello @PVince81 as you requested: Directories on Local Filesystem localdirs ownCloud is used to manage the files in these directories. The reason for this directory structure is to allow applications such as “miniDLNA” to access files to be steamed on a local network. Permissions and Ownership on “videos” directory: perms ownCloud External Storage Setup storage User External Storage Access users The group “ProjectXRWGrp” does not use the external storage setup. Files View for an ownCloud “admin” User admin-files Navigate in to “shrMedia” Folder admin-shrmedia As you can see this works for the “admin” user. Files for an ownCloud Normal User (test0) test0-files As you can see “shrMedia” is not visible here, but navigating through the “External storage” Menu. test0-shrmedia Navigating in to the “shrMedia”: test0-files User is taken back to the top level Files listing. ownCloud 8.1.3 had this same issue, 8.1.4 no longer has this issue.

PVince81 commented 8 years ago

I tried the following steps on v8.1.4 (admin user is called "root")

  1. Create a folder "/var/www/ocdata" and make this folder owned by the web server user
  2. Create a user "user1"
  3. Add user "root" to group "mediaRWGrp"
  4. Add user "user1" to group "mediaRWGrp" and "projectXRWGrp"
  5. Make "root" subadmin or group "mediaRWGrp" (just in case)
  6. Go to "Admin" page
  7. Mount external storage "/local" with type "Local" and specify the path "/var/www/ocdata", applicable to "mediaRWGrp"
  8. Login as "user1"
  9. Verify that "local" appears in the list
  10. Upgrade to 8.2.1
  11. Verify that "local" appears in the list

I can still see it.

I tried the same steps from scratch directly on v8.2.1, and I also see the mount point as "user1".

This is my ocdata folder:

drwxrwx--- 2 root www 4.0K Feb 24  2014 /var/www/ocdata

Maybe post your "mount.json" and "oc_storages" table ? oc_storages should have only a single entry for that local storage.

kneiser commented 8 years ago

@PVince81, I had a look at the oc_storages table and found multiple entries for "local::..". Some of them did not make sense so I removed them. Did my tests and now everything is fine. Here is what I have left in the table: oc_storages The "local::" entry pointing to the owncloud/data, is that supposed to be there? This path is in config.php. I've never configured the owncloud/data for local storage on this server.

PVince81 commented 8 years ago

Yes, just leave the one pointing to the data directory.

Which ones did you remove ? Just curious to see how they looked like.

kneiser commented 8 years ago

@PVince81 There were 3 "local::" entries: /var <= not sure how this got there /var/files <= not sure how this got there /var/file/mdeia <= this one was probably a typo on my part, I think this one was the problem. If I had known about this table in the past I would have looked. Too bad the occ doesn't have a cleanup that looks for disconnected items like this and remove them.

PVince81 commented 8 years ago

Hmmmm.... I wonder if it saved the entry while you were typing, before it was finished... Something to look into.

But even then, the existence of such bogus entries shouldn't affect the correct mount point.

kneiser commented 8 years ago

@PVince81 Well the entries did. I don't know but some sort of cleanup facility should be available, either manual via occ or automatic. One suggestion I have when setting up the external storage profiles, a "save" button rather than ownCloud's autosave.

kneiser commented 8 years ago

@PVince81 Recently I successfully upgrade 3 LAMP (2-Intel, 1-Raspberry Pi) stacks from ownCloud 8.1.4 to 8.2.1 the external storage problem did not recur. So I am considering this issue as solved. Thanks for your help!

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.