SMB: Cannot Delete Media Files via GUI

jxmot commented 4 years ago

I borrowed the issue template from motionEyeOS and modified it slightly. Initially I thought the problem was located there. But after some digging in the log files and some testing I determined that motionEye is where the issue is located. But I had already started gathering info in the issue template and it seemed like a clear way to describe the problem.

Issue Description

Via the motionEye web GUI the camera snapshot and video files that are saved on an SMB share cannot be deleted via the web GUI after the first deletion of either type of file.

So far "fixes" I have tried:

Rasperry Pi / motionEye:
- Just a reboot, no effect.
- Verified permissions and user credentials
- Started "fresh" with newly imaged motionEyeOS 20200606
NAS:
- Deleted/Added user on the NAS and recreated permissions
- Deleted/Added share on the NAS and recreated permissions
- Updated NAS firmware

Error Message(s)

Received this error when attempting to delete files:

[Errno 13] Permission denied: u'/data/media/motioneye_192_168_0_100_dogcam_dogcam/.keep'

I have also seen a message with a minor differnce, the u is missing from the path:

[Errno 13] Permission denied: '/data/media/motioneye_192_168_0_100_dogcam_dogcam/.keep'

Reproduction Steps

The issue described here may only occur with my specific hardware setup. But here are the steps I use to reproduce it:

Conditions: Successfully installed & configured motionEyeOS 20200606, a camera is working and snapshot capture is working. In addition motionEye has been configured to save media files on an SMB share. Motion detection has been turned off and no MP4 files are present.

1) Take at least 2 snapshots. Result: Within the SMB share a date stamped folder is created, it contains two time stamped image files

2) View the pictures taken by the camera via the GUI. Result: Two image files are visible

3) Delete the oldest image. Result: The image is deleted, and the .keep file is created for the first time.

4) Delete the second image. Result: This error message is seen - [Errno 13] Permission denied: u'/data/media/motioneye_192_168_0_100_dogcam_dogcam/.keep'

No additional files can be deleted unless .keep is independently deleted first.

Log Files

I consider the following log files relevant to this issue:

motioneye.log

2020-06-14 12:04:39: [motioneye]     INFO: rebooting
2020-06-14 12:04:39: [motioneye]     INFO: executing "/sbin/reboot"
2020-06-14 12:04:42: [motioneye]     INFO: interrupt signal received, shutting down...
2020-06-14 12:04:42: [motioneye]     INFO: server stopped
2020-06-14 12:04:42: [motioneye]     INFO: tasks stopped
2020-06-14 12:04:43: [motioneye]     INFO: motion stopped
2020-06-14 12:04:43: [motioneye]     INFO: smb mounts stopped
2020-06-14 12:04:43: [motioneye]     INFO: bye!
2020-06-14 12:05:11: [motioneye]     INFO: hello! this is motionEye server 0.42.1
2020-06-14 12:05:15: [motioneye]     INFO: cleanup started
2020-06-14 12:05:15: [motioneye]     INFO: wsswitch started
2020-06-14 12:05:15: [motioneye]     INFO: tasks started
2020-06-14 12:05:15: [motioneye]     INFO: mjpg client garbage collector started
2020-06-14 12:05:15: [motioneye]     INFO: smb mounts started
2020-06-14 12:05:15: [motioneye]     INFO: server started

After boot up I try to delete a file via the web GUI:

2020-06-14 12:06:15: [motioneye]    ERROR: failed to cleanup media files: [Errno 13] Permission denied: '/data/media/motioneye_192_168_0_100_dogcam_dogcam/.keep'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/motioneye/cleanup.py", line 87, in _do_cleanup
  File "/usr/lib/python2.7/site-packages/motioneye/mediafiles.py", line 308, in cleanup_media
IOError: [Errno 13] Permission denied: '/data/media/motioneye_192_168_0_100_dogcam_dogcam/.keep'

Likely Cause?

At this point I feel that the NAS/SMB share is the problem. According to the Python documentation opening a file for write ("w" mode) should:

create the file if it does not exist (this works)
if it does exist open it and overwrite it (fails)

I have searched through the issues(motionEye, motionEyeOS) and have not been unable to find any other issues here that are similar.

I'm not sure where to look next, any suggestions are appreciated!

Working Environment

Versions Used

motionEye Version: 0.42.1 Motion Version: 4.2.2+gitUNKNOWN OS Version: motionEyeOS 20200606

motionEyeOS was installed using the instructions provided by the wiki. The only pre-installation change was to create wpa_supplicant.conf prior to first boot.

Board Model

I am using the following board/model: Raspberry 3 B+.

Camera

I am using the following type of camera: MMAL

My camera model is: Arducam Raspberry Pi Camera V2 8mp, IMX219.

Network Connection

My motionEyeOS unit is connected to the network via: WiFi 5ghz

SMB Version 3.1.1 (others behave the same)

Peripherals

I am using the following peripherals that I consider relevant to this issue:

Asustor NAS AS1002T, ADM 3.5.0.R5D3(latest update), Samba version 4.4.3

zhidkovu commented 3 years ago

Same problem!

My fix: In file: /usr/local/lib/python2.7/dist-packages/motioneye/mediafiles.py just comment all strings with word .keep and comment one associated string: if os.path.exists(target_dir)

This disable creating .keep files Then reboot system and forget about .keep files forever Without .keep files, videos are deleted from the SMB server without errors

Hope this helps!

dariodoc commented 6 months ago

Same problem!

My fix: In file: /usr/local/lib/python2.7/dist-packages/motioneye/mediafiles.py just comment all strings with word .keep and comment one associated string: if os.path.exists(target_dir)

This disable creating .keep files Then reboot system and forget about .keep files forever Without .keep files, videos are deleted from the SMB server without errors

Hope this helps!

I have the same problem, I tried to locate this path and couldn't find it.

dariodoc commented 4 months ago

Well.. In my case what worked was to change the smb version to the latest (I believe is 3.x) instead of smb1. Hope it helps

MichaIng commented 3 months ago

So the problem is that motionEye creates those .keep files, "to prevent the directory from being removed". But in SMB shares, depending on SMB server config, those files can be hidden a way that even the client which created it, cannot remove it anymore?

So when motionEye then tries to remove the directory of a media file, it fails to do so, because the recursive removal skips the hidden .keep file, hence fails on directory removal.

Samba can be configured to not hide dot files, as pointed out by @as-kholin in another issue: https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html But of course that should not be necessary for motionEye to work properly.

The main question it, what else would remove that directory, if it was left empty without .keep files? I am no SMB expert, but so far I do not understand this in the first place. I know and understand it from Git, since Git does not sync empty directories. You if you want to assure an otherwise empty directory to be synced across Git clones and servers, a convention is to add such a dot file. But is this the same with (some) SMB servers? And even if, what is the problem if actually transferring the directory only once a real media file has been written, and have it removed automatically once that media file is removed? @zhidkovu's workaround shows a case where just skipping the creation of this dot file does not cause issues.

It was first implemented here: https://github.com/motioneye-project/motioneye/commit/8135e6c So the purpose there was to not have a directory removed on media files cleanup. But still not sure why this would even happen, since the _remove_older_files should only loop through the contained files, not the directory itself. But we can of course test and assure that, else adjust the function to exclude the directory from the loop.

motioneye-project / motioneye