search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
26.62k stars 3.99k forks source link

[Bug]: "Cannot delete the video part of a live photo" error message while deleting .mov file on server #46642

Closed squid-f closed 14 hours ago

squid-f commented 1 month ago

⚠️ This issue respects the following points: ⚠️

Bug description

After facing https://github.com/nextcloud/desktop/issues/6330#issuecomment-2239395618 I connected to my nextcloud account using Firefox to check whether the .mov file I deleted in Dolphin has been deleted from my account, as I got a client error notification. It appears the .mov file (created during the automatic transfer of an HEIC file by the Nextcloud on my iPhone) has been deleted locally on my desktop but is still present on the server. Even more weird, I cannot delete the .mov file using the web interface. I get Avertissement: delete process aborted

Steps to reproduce

  1. Transfer a live HEIC photo from my iPhone. Automatic transfer is on with the options "Most compatible" and "Live Photo".
  2. Two files are transferred to the server (a .jpg and a .mov) and synced to my desktop computer.
  3. Open Dolphin on the desktop computer and delete the .mov file
  4. The .mov file is deleted on the desktop computer and a notification error appears "Cannot delete the video part of a live photo" -> similar to https://github.com/nextcloud/desktop/issues/6330
  5. Connect to your account using a web browser.
  6. The .mov file which was deleted with Dolphin is still present.
  7. Try to delete it -> error , the file cannot be deleted

Expected behavior

The .mov file can be deleted on the server using a web browser connection

Installation method

Community Manual installation with Archive

Nextcloud Server version

29

Operating system

Other

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"**REMOVED SENSITIVE VALUE**"
],
"default_phone_region": "FR",
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "29.0.3.4",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"maintenance": false,
"mail_smtpmode": "smtp",
"mail_smtpsecure": "ssl",
"mail_sendmailmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtpauthtype": "LOGIN",
"mail_smtpauth": 1,
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"filelocking.enabled": true,
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"dbindex": 0,
"timeout": 0
},
"remember_login_cookie_lifetime": 0,
"session_keepalive": false,
"theme": "",
"logtimezone": "Europe\/Paris",
"loglevel": 2,
"log_rotate_size": 1100000,
"preview_libreoffice_path": "\/usr\/bin\/libreoffice",
"activity_expire_days": 1,
"trashbin_retention_obligation": "auto, 7",
"data-fingerprint": "9d938e709551177d36ec69695d8f9a93",
"overwrite.cli.url": "https:\/\/**REMOVED SENSITIVE VALUE**",
"encryption.key_storage_migrated": false,
"has_rebuilt_cache": true,
"allow_local_remote_servers": true,
"app_install_overwrite": [
"spreed",
"otpmanager"
],
"maintenance_window_start": 3,
"memories.db.triggers.fcu": true,
"memories.exiftool_no_local": true,
"memories.vod.path": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64",
"memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
"memories.vod.ffprobe": "\/usr\/bin\/ffprobe",
"enabledPreviewProviders": [
"OC\\Preview\\Image",
"OC\\Preview\\TIFF",
"OC\\Preview\\Movie"
],
"memories.gis_type": 1,
"preview_max_x": 2048,
"preview_max_y": 2048,
"memories.vod.disable": false,
"memories.vod.vaapi": true,
"memories.vod.nvenc": true
}
}

List of activated Apps

Enabled:
- activity: 2.21.1
- admin_audit: 1.19.0
- appointments: 2.1.6
- bookmarks: 14.2.2
- calendar: 4.7.12
- circles: 29.0.0-dev
- cloud_federation_api: 1.12.0
- comments: 1.19.0
- contacts: 6.0.0
- contactsinteraction: 1.10.0
- dashboard: 7.9.0
- dav: 1.30.1
- facerecognition: 0.9.51
- federatedfilesharing: 1.19.0
- files: 2.1.0
- files_downloadlimit: 2.0.0
- files_external: 1.21.0
- files_pdfviewer: 2.10.0
- files_reminders: 1.2.0
- files_sharing: 1.21.0
- files_trashbin: 1.19.0
- files_versions: 1.22.0
- fileslibreofficeedit: 1.1.0
- firstrunwizard: 2.18.0
- imageconverter: 2.0.2
- logreader: 2.14.0
- lookup_server_connector: 1.17.0
- memories: 7.3.1
- nextcloud_announcements: 1.18.0
- notifications: 2.17.0
- oauth2: 1.17.0
- otpmanager: 0.5.3
- password_policy: 1.19.0
- passwords: 2024.7.20
- phonetrack: 0.8.1
- photos: 2.5.0
- previewgenerator: 5.5.0
- privacy: 1.13.0
- provisioning_api: 1.19.0
- quota_warning: 1.19.0
- recognize: 7.0.3
- recommendations: 2.1.0
- related_resources: 1.4.0
- repod: 2.3.3
- richdocuments: 8.4.3
- richdocumentscode: 24.4.502
- serverinfo: 1.19.0
- settings: 1.12.0
- sharebymail: 1.19.0
- spreed: 19.0.7
- support: 1.12.0
- survey_client: 1.17.0
- systemtags: 1.19.0
- terms_of_service: 2.4.0
- text: 3.10.1
- theming: 2.4.0
- twofactor_admin: 4.5.0
- twofactor_backupcodes: 1.18.0
- twofactor_nextcloud_notification: 3.9.0
- twofactor_totp: 11.0.0-dev
- updatenotification: 1.19.1
- user_status: 1.9.0
- viewer: 2.3.0
- workflowengine: 2.11.0
Disabled:
- bruteforcesettings: 2.9.0 (installed 2.0.1)
- encryption: 2.17.0 (installed 2.5.0)
- federation: 1.19.0 (installed 1.15.0)
- guests: 3.1.0 (installed 3.1.0)
- integration_github: 2.0.7 (installed 2.0.7)
- maps: 1.4.0 (installed 1.4.0)
- suspicious_login: 7.0.0 (installed 4.4.0)
- talk_matterbridge: 1.26.0 (installed 1.23.2)
- user_ldap: 1.20.0
- weather_status: 1.9.0 (installed 1.2.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

[no app in context] Avertissement: delete process aborted
    DELETE /remote.php/dav/files/user/Photos/24-05-19%2019-10-58%203203.mov
    de 192.168.1.21 by user à 19 juil. 2024, 10:09:09

[no app in context] Avertissement: delete process aborted
    DELETE /remote.php/dav/files/user/Photos/24-05-19%2019-10-58%203203.mov
    de 192.168.1.21 by user à 19 juil. 2024, 10:09:09

[index] Erreur: Exception thrown: OCP\Files\GenericFileException
    GET /core/preview?fileId=380261&x=32&y=32&mimeFallback=true&a=0
    de 192.168.1.21 by user à 19 juil. 2024, 10:08:41

[PHP] Erreur: file_get_contents(/data/ncdata/appdata_ocjf3ojs7g1d/preview/3/b/3/6/4/6/5/380261/64-64-crop.jpg): Failed to open stream: No such file or directory at /var/www/nextcloud/lib/private/Files/Storage/Local.php#331
    GET /core/preview?fileId=380261&x=32&y=32&mimeFallback=true&a=0
    de 192.168.1.21 by user à 19 juil. 2024, 10:08:41

[index] Erreur: Exception thrown: OCP\Files\GenericFileException
    GET /core/preview?fileId=380261&x=32&y=32&mimeFallback=true&a=0
    de 192.168.1.21 by user à 19 juil. 2024, 10:08:41

[PHP] Erreur: file_get_contents(/data/ncdata/appdata_ocjf3ojs7g1d/preview/3/b/3/6/4/6/5/380261/64-64-crop.jpg): Failed to open stream: No such file or directory at /var/www/nextcloud/lib/private/Files/Storage/Local.php#331
    GET /core/preview?fileId=380261&x=32&y=32&mimeFallback=true&a=0
    de 192.168.1.21 by user à 19 juil. 2024, 10:08:41

Additional info

No response

solracsf commented 1 month ago

Scan the folder with occ files:scan --path and try again

squid-f commented 1 month ago

Scan the folder with occ files:scan --path and try again

Same error:

{"reqId":"JeXCrXVgGp87IXjQ70WY","level":2,"time":"2024-07-19T17:52:06+02:00","remoteAddr":"192.168.1.21","user":"user","app":"no app in context","method":"DELETE","url":"/remote.php/dav/files/user/Photos/24-07-17%2017-36-30%203256.mov","message":"delete process aborted","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","version":"29.0.3.4","exception":{"Exception":"OCP\\Exceptions\\AbortedEventException","Message":"Cannot delete the video part of a live photo","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/files/lib/Listener/SyncLivePhotosListener.php","line":89,"function":"handleDeletion","class":"OCA\\Files\\Listener\\SyncLivePhotosListener","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/ServiceEventListener.php","line":86,"function":"handle","class":"OCA\\Files\\Listener\\SyncLivePhotosListener","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":230,"function":"__invoke","class":"OC\\EventDispatcher\\ServiceEventListener","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":59,"function":"callListeners","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php","line":86,"function":"dispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php","line":98,"function":"dispatch","class":"OC\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/HookConnector.php","line":131,"function":"dispatchTyped","class":"OC\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Hook.php","line":105,"function":"delete","class":"OC\\Files\\Node\\HookConnector","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1282,"function":"emit","class":"OC_Hook","type":"::"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1154,"function":"runHooks","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":705,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":543,"function":"unlink","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":197,"function":"delete","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":281,"function":"delete","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpDelete","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":379,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":172,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/files/lib/Listener/SyncLivePhotosListener.php","Line":195,"message":"delete process aborted","exception":[],"CustomMessage":"delete process aborted"},"id":"669a8bc115dc7"}

Edit: I did a scan --all and I got:

+---------+-------+-----+---------+---------+--------+--------------+
| Folders | Files | New | Updated | Removed | Errors | Elapsed time |
+---------+-------+-----+---------+---------+--------+--------------+
| 3796    | 70147 | 0   | 44316   | 0       | 0      | 00:00:20     |
+---------+-------+-----+---------+---------+--------+--------------+
susnux commented 1 month ago

CC @artonge

squid-f commented 1 month ago

and now my Nextcloud journal is flood by messages like:

{"reqId":"ju95V155jnxyKuCtowHo","level":2,"time":"2024-07-21T07:57:18+02:00","remoteAddr":"192.168.1.21","user":"user","app":"no app in context","method":"DELETE","url":"/remote.php/dav/files/user/Photos/24-07-18%2010-59-07%203259.mov","message":"delete process aborted","userAgent":"Mozilla/5.0 (Linux) mirall/3.13.2git (Nextcloud, garuda-6.9.10-zen1-1-zen ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"29.0.3.4","exception":{"Exception":"OCP\\Exceptions\\AbortedEventException","Message":"Cannot delete the video part of a live photo","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/files/lib/Listener/SyncLivePhotosListener.php","line":89,"function":"handleDeletion","class":"OCA\\Files\\Listener\\SyncLivePhotosListener","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/ServiceEventListener.php","line":86,"function":"handle","class":"OCA\\Files\\Listener\\SyncLivePhotosListener","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":230,"function":"__invoke","class":"OC\\EventDispatcher\\ServiceEventListener","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":59,"function":"callListeners","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php","line":86,"function":"dispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php","line":98,"function":"dispatch","class":"OC\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/HookConnector.php","line":131,"function":"dispatchTyped","class":"OC\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Hook.php","line":105,"function":"delete","class":"OC\\Files\\Node\\HookConnector","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1282,"function":"emit","class":"OC_Hook","type":"::"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1154,"function":"runHooks","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":705,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":543,"function":"unlink","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":197,"function":"delete","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":281,"function":"delete","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpDelete","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":379,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":172,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/files/lib/Listener/SyncLivePhotosListener.php","Line":195,"message":"delete process aborted","exception":[],"CustomMessage":"delete process aborted"},"id":"669ca4ade99a1"}
squid-f commented 1 month ago

Maybe a hint here, looking at row 143 onwards? https://github.com/nextcloud/server/blob/a58ed05f1d19d591a1ed384e87915db952c5f163/apps/files/lib/Listener/SyncLivePhotosListener.php

artonge commented 1 month ago

I think it is expected, but could maybe be improved. Also, in the web UI, you should not be able to see the .mov file.

squid-f commented 1 month ago

I think it is expected, but could maybe be improved. Also, in the web UI, you should not be able to see the .mov file.

Well, both a .JPEG and a .mov are uploaded by the iOS app. On my desktop computer, I can see both of them synced. If I remove either the .jpeg or the .mov with my file manager, I do expect they are removed from the server as well. Why enforcing a different behavior compared to the removal of any other file type?

artonge commented 1 month ago

We can only display them as one file in the web UI and in mobile applications. So on those platform, there is no chance to wrongly delete one part or the other, which would "break" the live photo. On the desktop client, we are not able to show them as one file, so our logic was to block the deletion of the .mov file to prevent users from breaking their live photos.

squid-f commented 1 month ago

Understood, however can you prevent to delete the .mov on desktop ? What about not convertibg heic pictures and uploading them as heic ?

artonge commented 1 month ago

can you prevent to delete the .mov on desktop

You mean, blocking the delete action? I don't think that's possible. You can ask in the desktop repo.

What about not convertibg heic pictures and uploading them as heic ?

I am not sure to understand your question. I don't think we are converting anything.

squid-f commented 1 month ago

Hi. Indeed, desktop delete cannit be blocked, that's my point. So, If the user wants to delete the .mov, it should be reflected on server side.

Regarding conversion, my phone takes a heic picture and nextcloud upload it to the server, using a jpeg and a mov file. It is where the conversion happens. If one wants to prevent braking a live photo by deleting the mov file, I think uploading a heic format is a better option. Otherwise, it should be up to the user to decide which files to keep.

artonge commented 1 month ago

So, If the user wants to delete the .mov, it should be reflected on server side.

That's not the decision we made, as we think this is not what the user will expect.

From what I understand, the .heic does not contain the video part of a live photo. On iOS, you do not see the video file as Apple is hiding it, like we are doing on the web UI. So whenever you move the live photos out of iOS, you end up with two different files.

squid-f commented 1 month ago

My understanding is rather heic/ heif is a container including a series of images to create a live picture. So, one file only. I let experts confirming, as I am not.

artonge commented 1 month ago

My understanding is rather heic/ heif is a container including a series of images to create a live picture. So, one file only.

If that's true, then better ask the client's repositories.

squid-f commented 1 month ago

Hi. There is already another report targeting the client. We made a digression while talking about live pictures. This report is about a feature taking rights over the user, preventing to delete a file on the server. It cannot be that something has more rigths on the files than the user owning them, IMHO.

Edit: to summarize, I have now .mov files on the server which don't exist on my local computer and that I can't delete. In additing, it looks like there is an event coming back periodically, retrying to delete on the server, failing and flooding my log with errors.

susnux commented 1 month ago

I think this is a desktop client bug. The client should never try to delete the mov.

Basically this is one file, which because of limitations on the desktop, will be synced as two files. In the webui it is correctly shown as one file.

So if you delete any of it, "both" will be gone.

wotography commented 2 weeks ago

Fair enough you don't want to bother with people who delete the .mov files for what ever reason.

I think somewhere should at least be a introduction for this behaveiour and solution. So clients can know what they do.

I also think there should be a possibility to redownload the missing .mov files on desktop side, once deleted. I imagine something like a secondary click on a folder and selecting "resynchronize" or "redownload from server".

joshtrichards commented 17 hours ago

This looks to be probably a duplicate of #42237

susnux commented 14 hours ago

yes duplicate of https://github.com/nextcloud/server/issues/42237