search

icloud-photos-downloader / icloud_photos_downloader

A command-line tool to download photos from iCloud
MIT License
6.6k stars 536 forks source link

1.17.3 introduces "Permission denied" on Synology NAS #764

Closed genegoykhman closed 8 months ago

genegoykhman commented 8 months ago

Overview

Running a pip install --upgrade icloudpd today updated my 1.17.2 installation to 1.17.3. Subsequently trying to run bin/icloudpd produces this error:

$ bin/icloudpd -d /volume1/homes/username/Photos -u username@icloud.com --set-exif-datetime --auto-delete --threads-num 2 --log-level error
Failed to execv() /tmp/staticx-geoCGJ/icloudpd-1.17.3-linux-amd64.dynamic: Permission denied

Downgrading using pip install icloudpd==1.17.2 resolves the issue.

Steps to Reproduce

pip install --upgrade icloudpd
icloudpd (parameters)

Expected Behavior

Sync should start normally.

Actual Behavior

Error:

Failed to execv() /tmp/staticx-geoCGJ/icloudpd-1.17.3-linux-amd64.dynamic: Permission denied

Context

Synology DS1522+ NAS

AndreyNikiforov commented 8 months ago

Sorry for broken experience. Compatibility tests show that pip runs fine on amd64 linuxes - I am curious how your setup on Synology is different... We run compatibility test in python virtual envs while your setup seems to be with it - may be that is the root cause...

genegoykhman commented 8 months ago

Thanks for the quick response Andrey. I'm not too familiar with Python or virtual environments, but I followed the Sam the Geek guide to set things up.

Here is a screenshot of the repro steps ... maybe I have a stale pip package cache or something? 2024-01-04 at 4 15 PM

AndreyNikiforov commented 8 months ago

"Permission denied" for /tmp folder suggests that user under which icloudpd is running does not have sufficient/necessary privileges for using /tmp folder. I suspect the fact that icloudpd self-extracts into tmp and then runs from there, complicate things.

I tried reproducing the situation using docker on linux and non-privileged account -- no lack. I don't have Synology hardware and seems that DSM is not available for installing into VM.

A number of options for your to proceed:

git-newbe commented 8 months ago

Hi Andrey, using docker image seems to be an issue as well. Please see my post #762. May I ask you to check what has been changed? Thanks a lot. - If you like we can also have a quick remote session to demonstrate ;)

genegoykhman commented 8 months ago

To test your theory that it might be a permissions issue I logged in with sudo -i and tried again:

2024-01-05 at 10 49 AM

Immediately after running this command I checked and there is no /tmp/staticx-lHadii in the file system, but maybe it was cleaned up as part of the script.

Is this expanding into /tmp new for 1.17.3? I noticed that there were some commits related to building and deployment between 1.17.2 and 1.17.3.

Anyway, thanks for looking into this and I'm happy to test or provide additional info.

AndreyNikiforov commented 8 months ago

Is this expanding into /tmp new for 1.17.3? I noticed that there were some commits related to building and deployment between 1.17.2 and 1.17.3.

We've been packaging python into single binary executable for some time now. In 1.17.3 I added validation and tweaks to the process that may have changed the behavior around /tmp folder.

Anyway, thanks for looking into this and I'm happy to test or provide additional info.

My thoughts on possible avenues for troubleshooting:

IIUC DSM allows running Virtual DSM (as VM). That may be better option for troubleshooting than your real setup.

gierschv commented 8 months ago

I got the same issue, it's happening because /tmp is mounted with noexec on Synology DSM 7.

sudo mount /tmp -o remount,exec will temporary fix this issue. I don't see it in the fstab, so I will probably try to run icloudpd differently if I don't see how to change this config.

genegoykhman commented 8 months ago

Yup @gierschv the sudo mount /tmp -o remount,exec allows 1.17.3 to work. Thanks!

I guess something in 1.17.3 now requires the ability to execute from within /tmp whereas that wasn't necessary in previous versions. I don't know much about DSM but if there is a better workaround in the future (like adding a line into fstab somewhere) I'd be interested in hearing it.

Or better yet, maybe icloudpd can be expanded and executed from $TMP instead @AndreyNikiforov ? But I'm not sure if that's feasible / possible / a good idea.

AndreyNikiforov commented 8 months ago

Or better yet, maybe icloudpd can be expanded and executed from $TMP instead @AndreyNikiforov ? But I'm not sure if that's feasible / possible / a good idea.

I'll keep looking for Synology with special permission on /tmp use case. I suspect that before 1.17.3 you were getting source code, while in 1.17.3 you started getting binaries and /tmp behavior is related to how self-extracting binaries work.

In general, I found binary distributions (docker, binary pip, binary npm) easier to support as there are less uncertainty and better compatibility (e.g. source icloudpd on arm needs rust compiler and plenty of libs during installation). There will be nuances like Synology /tmp use case and we'll have to figure out how to solve them.

Thanks for your patience

SamTheGeek commented 6 months ago

@genegoykhman I'm going to have to update my guide at some point, a few of the instructions are a little out of date.

It also appears that you can use export TMPDIR=$HOME/tmp; before running icloudpd which allows you to run it without needing sudo. This allows for use in scheduled tasks and unattended setup.

genegoykhman commented 6 months ago

@SamTheGeek I just tried it with icloudpd 1.17.3 and you're right, exporting TMPDIR=$HOME/tmp resolves the issue. Here is my updated procedure for logging into my Synology NAS and running an icloudpd sync:

ssh myuser@mynas.local
cd icloudpd
source bin/activate
pip install --upgrade icloudpd
export TMP=$HOME/icloudpd/tmp TMPDIR=$HOME/tmp
icloudpd -d /volume1/homes/myuser/Photos -u myuser@myappleid.com --set-exif-datetime --auto-delete --threads-num 2 --log-level error