owncloud / core

:cloud: ownCloud web server core (Files, DAV, etc.)
https://owncloud.com
GNU Affero General Public License v3.0
8.4k stars 2.05k forks source link

expiry of a shared link leads to removal of remaining link with expiry in the future (multiple shared link feature) #32406

Open hmalessa opened 6 years ago

hmalessa commented 6 years ago
### Steps to reproduce 1. create/have one shared link with expiration date 2. create a second shared link (for same thing of course) with an expiration date after first link's expiry 3. wait until expiration date of first link ### Expected behaviour only the first link should be removed. The second link has to remain. ### Actual behaviour all links (for the same folder) are removed. database table OC_SHARE has no more rows with share_type=3. ### Server configuration **Operating system**: Ubuntu Server 18.04.1 LTS Kernel 4.4.0-133 x86_64 **Web server:** apache2 2.4.18-2ubuntu3.9 **Database:** mysql Ver 14.14 Distrib 5.7.23 **PHP version:** PHP 7.0.30-0ubuntu0.16.04.1 Redis server v=3.2.5 sha=00000000:0 malloc=jemalloc-4.0.3 bits=64 build=7a2c9bceaa71a5cb **ownCloud version:** (see ownCloud admin page) ownCloud 10.0.9 (stable) at the moment the version when the issue occured was 10.0.8.4 **Updated from an older ownCloud or fresh install:** ownCloud 9 **Where did you install ownCloud from:** ubuntu repo http://download.owncloud.org/download/repositories/production/Ubuntu_16.04/ **Signing status (ownCloud 9.0 and above):** ``` Login as admin user into your ownCloud and access http://example.com/index.php/settings/integrity/failed paste the results into https://gist.github.com/ and puth the link here. ``` https://gist.github.com/hmalessa/c2c90d277eac17f0fc3f9268bca6b478 **The content of config/config.php:** ``` Log in to the web-UI with an administrator account and click on 'admin' -> 'Generate Config Report' -> 'Download ownCloud config report' This report includes the config.php settings, the list of activated apps and other details in a well sanitized form. or If you have access to your command line run e.g.: sudo -u www-data php occ config:list system from within your ownCloud installation folder *ATTENTION:* Do not post your config.php file in public as is. Please use one of the above methods whenever possible. Both, the generated reports from the web-ui and from occ config:list consistently remove sensitive data. You still may want to review the report before sending. If done manually then it is critical for your own privacy to dilligently remove *all* host names, passwords, usernames, salts and other credentials before posting. You should assume that attackers find such information and will use them against your systems. ``` { "system": { "updatechecker": false, "instanceid": "oc7cabrr7t9d", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***", "***REMOVED SENSITIVE VALUE***" ], "datadirectory": "\/var\/www\/owncloud\/data", "overwrite.cli.url": "http:\/\/***REMOVED SENSITIVE VALUE***\/owncloud", "dbtype": "mysql", "version": "10.0.9.5", "dbname": "owncloud", "dbhost": "localhost", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "logtimezone": "UTC", "installed": true, "mail_smtpmode": "smtp", "mail_smtpsecure": "ssl", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "25", "activity_expire_days": 60, "memcache.local": "\\OC\\Memcache\\Redis", "filelocking.enabled": "true", "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "localhost", "port": 6379, "timeout": 0, "dbindex": 0 }, "theme": "", "loglevel": 2, "maintenance": false, "preview_max_scale_factor": 1, "enabledPreviewProviders": { "0": "OC\\Preview\\PNG", "1": "OC\\Preview\\JPEG", "2": "OC\\Preview\\GIF", "11": "OC\\Preview\\Illustrator", "12": "OC\\Preview\\Postscript", "13": "OC\\Preview\\Photoshop", "14": "OC\\Preview\\TIFF" } } } **List of activated apps:** ``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your ownCloud installation folder. ``` Enabled: - activity: 2.3.7 - comments: 0.3.0 - configreport: 0.1.1 - dav: 0.3.2 - federatedfilesharing: 0.3.1 - federation: 0.1.0 - files: 1.5.1 - files_external: 0.7.1 - files_pdfviewer: 0.9.0 - files_sharing: 0.10.1 - files_texteditor: 2.2.1 - files_trashbin: 0.9.1 - files_versions: 1.3.0 - files_videoplayer: 0.9.8 - firstrunwizard: 1.1 - gallery: 16.1.0 - market: 0.2.5 - notifications: 0.3.4 - provisioning_api: 0.5.0 - systemtags: 0.3.0 - templateeditor: 0.3.1 - updatenotification: 0.2.1 Disabled: - encryption - external - gallery.orig - user_external **Are you using external storage, if yes which one:** local/smb/sftp/... no **Are you using encryption:** yes/no no **Are you using an external user-backend, if yes which one:** LDAP/ActiveDirectory/Webdav/... no #### LDAP configuration (delete this part if not used) ``` With access to your command line run e.g.: sudo -u www-data php occ ldap:show-config from within your ownCloud installation folder Without access to your command line download the data/owncloud.db to your local computer or access your SQL server remotely and run the select query: SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap'; Eventually replace sensitive data as the name/IP-address of your LDAP server or groups. ``` ### Client configuration **Browser:** **Operating system:** ### Logs #### Web server error log ``` Insert your webserver log here ``` #### ownCloud log (data/owncloud.log) ``` Insert your ownCloud log here ``` link expiry of first shared link was 2018-08-15. OK: access to first shared link without password (I assume this was a search engine request) {"reqId":"Rim7OrExgRzGPfrZUbQh","level":3,"time":"2018-08-14T19:52:16+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"POST","url":"\/owncloud\/index.php\/apps\/gallery\/s\/pUnWVKTJNw6nlSW","message":"Exception: Wrong password (401)"} OK: same request like above but after expiration (shared link was removed) {"reqId":"2ejcSurExp8O8piw5x19","level":3,"time":"2018-08-15T09:51:27+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"GET","url":"\/owncloud\/index.php\/apps\/gallery\/s\/pUnWVKTJNw6nlSW","message":"Exception: Unspecified share exception (404)"} FAIL: DeleteOrphanedSharesJob with dead lock {"reqId":"cpRcMxZW0eDrUPD5Cbwf","level":3,"time":"2018-08-21T09:00:14+00:00","remoteAddr":"","user":"--","app":"core","method":"--","url":"--","message":"Error while running background job (class: OCA\\Files_Sharing\\DeleteOrphanedShar esJob, arguments: ): {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\DriverException\",\"Message\":\"An exception occurred while executing 'DELETE FROM `oc_share` WHERE `item_type` in ('file', 'folder') AND NOT EXISTS (SELECT `fileid ` FROM `oc_filecache` WHERE `file_source` = `fileid`)':\\n\\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\ \\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\ \/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(1015): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOMySql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'DELETE FROM `oc...', Array)\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(210): Doctrine\\\\DBAL\\\\Connection->executeUpdate('DELETE FROM `oc...', Array , Array)\\n#3 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_sharing\\\/lib\\\/DeleteOrphanedSharesJob.php(61): OC\\\\DB\\\\Connection->executeUpdate('DELETE FROM `oc...')\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/BackgroundJob \\\/Job.php(57): OCA\\\\Files_Sharing\\\\DeleteOrphanedSharesJob->run(NULL)\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/BackgroundJob\\\/TimedJob.php(53): OC\\\\BackgroundJob\\\\Job->execute(Object(OC\\\\BackgroundJob\\\\JobLi st), Object(OC\\\\Log))\\n#6 \\\/var\\\/www\\\/owncloud\\\/cron.php(121): OC\\\\BackgroundJob\\\\TimedJob->execute(Object(OC\\\\BackgroundJob\\\\JobList), Object(OC\\\\Log))\\n#7 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\ /composer\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractMySQLDriver.php\",\"Line\":115}"} FAIL: first request of second shared link addressing the same folder like first link. Link was created on 2018-07-23 with expiration in 2019 {"reqId":"A1QTyldRk8aHf3Cz4VMj","level":3,"time":"2018-08-21T18:20:27+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"GET","url":"\/owncloud\/index.php\/apps\/gallery\/s\/pSHcQ80GsuRCb4J","message":"Exception: Unspecified share exception (404)"} The query "select * from oc_share where share_type=3" has no results (tested at 2018-08-21 22:00) #### Browser log ``` Insert your browser log here, this could for example include: a) The javascript console log b) The network log c) ... ```
ownclouders commented 6 years ago

GitMate.io thinks the contributors most likely able to help are @ownclouders, and @PVince81.

Possibly related issues are https://github.com/owncloud/core/issues/22327 (Allow multiple link shares), https://github.com/owncloud/core/issues/7813 ([FEATURE REQUEST] Share link with multiple users has different link), https://github.com/owncloud/core/issues/11097 (Shared links expire with no expiry date set), https://github.com/owncloud/core/issues/15022 (Share link hangs), and https://github.com/owncloud/core/pull/31129 (Add missing public link share dialog features).

PVince81 commented 6 years ago

From what I see in the code the deletion should only affect shares that are actually expired: https://github.com/owncloud/core/blob/v10.0.9/lib/private/Share20/Manager.php#L1129

There is a loop that checks for each share.

@hmalessa how far apart were the expiration dates ? Is this reproducible consistently ? Are maybe time zones involved where maybe you believed that the second link was still valid but a time zone difference might have deleted it early ? (see https://github.com/owncloud/core/issues/8363 for the timezone issue)

I've set up a test case on my server (10.0.9) and will see whether the observed problem happens there as well.

PVince81 commented 6 years ago

I had set a link to expire today and another next week. Today I opened the share panel for the folder in question and I can see that today's link did disappear, but the one from next week is still there.

So no problem observed so far...

PVince81 commented 6 years ago

@hmalessa when you say "wait until expiration date of first link" there must be a step after that: are you opening the expired link ? or are you opening the share panel as I did ?

because the action you did after that is important as the expiry logic is triggered only the next time the link is accessed or listed

leblanc217 commented 3 years ago

@ownclouders, and @PVince81

Adding to this old thread. We also expirenced this behaviour today. We have directories with multiple public shares, one expiring the day of the issue (old link) and another expiring two weeks into the future (new link). When the old link expired, it also removed the new link we uploaded that day.

New to GitHub so any information required please let me know and I'll provide it.

Showing share activity, note two shares uploaded and only one expired: owncloud0

Showing no public links after first one expired. Should be at least one link as there are not two expire events: owncloud1

phil-davis commented 3 years ago

@dpakach we did some tests for link expiry recently. IMO we should now be able to add a test scenario where there are 2 public link shares of a resource. Then we put the expiry of one of them into the past, and access that "past-expired" link share. That should cause the "past-expired" link share to disappear. But the other link share that expires in the future should still exist.

Such a test scenario should be able to demonstrate this problem - and then someone can fix it.

I added this to the QA Test Automation project so that we can demonstrate the issue.

phil-davis commented 3 years ago

We added a test scenario in #39100 and that passes. I tried a few permutations of that locally, and could not find a sequence that failed. @SwikritiT tried locally to make public links with different expiry dates, and let one expire "naturally" overnight. And still we don't see the problem.

The other place that could be a problem is https://github.com/owncloud/core/blob/master/apps/files_sharing/lib/ExpireSharesJob.php - maybe if that runs, and finds a public link share that has expired, that it accidentally removes all the public link shares of the resource. But looking at the code, it explicitly queries the database only for shares with expiry before "today". And it loops through that list deleting the shares.

Before PR #38775 if there was a share that expired yesterday (1), and a share that expires today (2), and a share that expires tomorrow (3), then both (1) and (2) would have been deleted. After that PR (i.e. in 10.8.0 release) only (1) gets deleted.

Please try with 10.8.0 and let us know if you can still reproduce the problem.