version | system_name | percent | download_count |
---|---|---|---|
2.8.6 | Linux | 71.70% | 38 |
2.8.6 | Darwin | 13.21% | 7 |
2.8.7a1 | Linux | 7.55% | 4 |
2.8.6 | Windows | 5.66% | 3 |
2.8.6a9 | Linux | 1.89% | 1 |
THIS SCRIPT IS PROVIDED WITH NO WARRANTY WHATSOEVER. PLEASE REVIEW THE SOURCE CODE TO MAKE SURE IT WILL WORK FOR YOUR NEEDS. IF YOU FIND A BUG, PLEASE REPORT IT.
Consider this example to explain how files are uploaded into Sets/Albums on Flickr.
If you have the following folders and pics (the name of the flickr Sets/Albums depends on the uploadr.ini file setting FULL_SET_NAME, but I normally use it as False):
/home/user/media/pic00.jpg
/home/user/media/Album1/pic01.jpg
/home/user/media/Album2/pic02.jpg
/home/user/media/Album3/pic03.jpg
/home/user/media/folder/Album4/pic04.jpg
/home/user/media/folder/Album4/Sub/pic041.jpg
/home/user/media/newfolder/Album4/pic042.jpg
/home/user/media/folderAlbum5/pic01.jpg
/home/user/media/folderAlbum5/Sub/pic051.jpg
And you setup FILES_DIR
FILES_DIR=/home/user/media
You should get the following depending on how the setting FULL_SET_NAME is set:
FilePathName | Set/Album Name (FULL_SET_NAME=False) | Set/Album Name (FULL_SET_NAME=True) | Pic | Remarks |
---|---|---|---|---|
/home/user/media/pic00.jpg | media | . | pic00 | |
/home/user/media/Album1/pic01.jpg | Album1 | Album1 | pic01 | |
/home/user/media/Album2/pic02.jpg | Album2 | Album2 | pic02 | |
/home/user/media/Album3/pic03.jpg | Album3 | Album3 | pic03 | |
/home/user/media/folder/Album4/pic04.jpg | Album4 | folder/Album4 | pic04 | |
/home/user/media/folder/Album4/Sub/pic041.jpg | Sub | folder/Album4/Sub | pic041 | |
/home/user/media/newfolder/Album4/pic042.jpg | Album4 | newfolder/Album4 | pic042 | |
/home/user/media/Album5/pic01.jpg | Album5 | Album5 | pic01 | Same pic as in Album01 is loaded twice as it's part of a different Album |
/home/user/media/Album5/Sub/pic051.jpg | Sub | Album5/Sub | pic051 | With FULL_SET_NAME=False it will go into Album "Sub" |
$ cd
$ mkdir apps
$ mkdir apps/Python
$ export PYTHONPATH=~/apps/Python/lib/python2.7/site-packages
$ export PYTHONPATH=~/apps/Python/lib/python3.5/site-packages
dev
directory/folder to use as working area where to download/extract the files/packages prior to intstallation:
$ cd
$ mkdir dev
dev$ cd dev
python get-pip.py --prefix=~/apps/Python
$ cd
$ cd dev
dev$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1603k 100 1603k 0 0 3828k 0 --:--:-- --:--:-- --:--:-- 3827k
dev$ python get-pip.py --prefix=~/apps/Python
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% 1.3MB 495kB/s
Collecting setuptools
Downloading setuptools-36.6.0-py2.py3-none-any.whl (481kB)
100% 481kB 1.3MB/s
Collecting wheel
Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
100% 51kB 4.1MB/s
Installing collected packages: pip, setuptools, wheel
Successfully installed pip setuptools wheel
$ cd
$ cd dev
dev$ export PYTHONPATH=~/apps/Python/lib/python2.7/site-packages
dev$ pip install flickrapi --prefix=~/apps/Python
python setup.py install --prefix=~/apps/Python
$ cd dev
dev$ wget https://files.pythonhosted.org/packages/b1/f1/d10fa0872e4f781c2ed47e94e728ecd3c1998f8c8d12e78c7329a25d0727/flickrapi-2.4.0.tar.gz
dev$ tar tzvf flickrapi-2.4.0.tar.gz
flickrapi-2.4.0/
flickrapi-2.4.0/CHANGELOG.md
flickrapi-2.4.0/MANIFEST.in
flickrapi-2.4.0/.coveragerc
flickrapi-2.4.0/LICENSE.txt
flickrapi-2.4.0/tox.ini
flickrapi-2.4.0/README.md
(...)
dev$ cd flickrapi-2.4.0
dev/flickrapi-2.4.0$ python setup.py install --prefix=~/apps/Python
python setup.py install --prefix=~/apps/Python
running install
running bdist_egg
running egg_info
writing requirements to flickrapi.egg-info/requires.txt
writing flickrapi.egg-info/PKG-INFO
(...)
zip_safe flag not set; analyzing archive contents...
Moving chardet-3.0.4-py2.7.egg to /xxx/xxx/xxx/apps/Python/lib/python2.7/site-packages
Adding chardet 3.0.4 to easy-install.pth file
Installing chardetect script to /xxx/xxx/xxx/apps/Python/bin
Installed /xxxx/xxx/xxx/apps/Python/lib/python3.5/site-packages/certifi-2018.4.16-py3.5.egg Finished processing dependencies for flickrapi==2.4.0
### 5. Download and install flickr-uploader
#### 5.1 OPTION #1 (recommended): With PIP (installed in step #3 above)
- Now available on Pypi.org for installation also via PIP.
```bash
$ cd
$ cd dev
dev$ export PYTHONPATH=~/apps/Python/lib/python2.7/site-packages
dev$ pip install flickr-uploader --prefix=~/apps/Python
uploadr.ini
and uploadr.cron
$ cd
$ cd apps
apps$ wget https://github.com/oPromessa/flickr-uploader/releases/download/2.8.5/flickr-uploader-2.8.5.tar.gz
apps$ tar xzvf flickr-uploader-2.8.5.tar.gz
apps$ cd flickr-uploader-2.8.5
apps$ ./uploadr.py -a
~/apps/Python/bin
$ cd
$ cd apps
apps$ wget https://github.com/oPromessa/flickr-uploader/releases/download/2.8.5/flickr-uploader-2.8.5.tar.gz
apps$ tar xzvf flickr-uploader-2.8.5.tar.gz
apps$ cd flickr-uploader-2.8.5
apps$ python2.7 setup.py install --prefix=~/apps/Python --old-and-unmanageable
uploadr.ini
and uploadr.cron
Go to http://www.flickr.com/services/apps/create/apply and apply for an API key.
FILES_DIR = "YourDir"
FLICKR = {
"title" : "",
"description" : "",
"tags" : "auto-upload",
"is_public" : "0",
"is_friend" : "0",
"is_family" : "0",
"api_key" : "Yourkey",
"secret" : "YourSecret"
}
FLICKR["api_key"] = ""
FLICKR["secret"] = ""
EXCLUDED_FOLDERS = ["@eaDir","#recycle"]
IGNORED_REGEX = ['*[Ii][Gg][Nn][Oo][Rr][Ee]*', 'Private*']
ALLOWED_EXT = ["jpg","png","avi","mov","mpg","mp4","3gp"]
MANAGE_CHANGES = True
FULL_SET_NAME = False
Refer to https://www.flickr.com/services/api/upload.api.html for what each of the upload arguments above correspond to for Flickr's API.
Before running uploadr.py make sure you run the command below:
$ export PYTHONPATH=~/apps/Python/lib/python2.7/site-packages
On the first run you need to authenticate the applicaiton against Flickr.
-a
option$ cd dev
dev$ uploadr.py -a
Importing xml.etree.ElementTree...done. Continuing.
--------- (V2.7.7) Init: ---------
Python version on this system: 3.6.3 (default, Oct 3 2017, 21:45:48)
[GCC 7.2.0]
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] --------- (V2.7.7) Start time: 2018.04.16 23:55:09 ---------(Log:40)
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Setting up database:[/home/user/dev/flickrdb]
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Database version: [3]
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Completed database setup
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Checking if token is available... if not will authenticate
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Getting new token.
[2965][2018.04.16 23:55:09]:[12758 ][PRINT ]:[uploadr] Copy and paste following authorization URL in your browser to obtain Verifier Code.
https://www.flickr.com/services/oauth/authorize?oauth_token=xxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx&perms=delete
Verifier code (NNN-NNN-NNN):
Following runs can be launched unattended:
dev$ ./uploadr.py -v
Place the file uploadr.py in any directory and run via ssh (execution privs required). It will crawl through all the files from the FILES_DIR directory and begin the upload process.
$ ./uploadr.py
To check what files uploadr.py would upload and delete you can run the
script with option --dry-run
:
$ ./uploadr.py --dry-run
Run ./uploadr.py --help
for up to the minute information on arguments:
[961][2018.09.16 06:06:42]:[15221 ][PRINT ]:[uploadr] ----------- (V2.8.7) Start -----------(Log:40)
usage: uploadr.py [-h] [-C filename.ini] [-a] [-v] [-x] [-m] [-n] [-i TITLE]
[-e DESCRIPTION] [-t TAGS] [-l N] [-r] [-p P] [-u]
[--no-delete-from-flickr [nodelete]] [-d] [-b] [-c] [-s]
[-g] [--add-albums-migrate]
Upload files to Flickr. Uses uploadr.ini as config file.
optional arguments:
-h, --help show this help message and exit
Configuration related options:
-C filename.ini, --config-file filename.ini
Optional configuration file. Default
is:[/home/ruler/uploader/bin/uploadr.ini]
-a, --authenticate Performs/Verifies authentication with Flickr. To be
run on initial setup.Does not run any other option.
Verbose and dry-run options:
-v, --verbose Verbose output. Use -vv for more verbosity. See also
LOGGING_LEVEL value in INI file.
-x, --verbose-progress
Provides progress indicator on each upload. See also
LOGGING_LEVEL value in INI file.
-m, --mask-sensitive Masks sensitive data on log files like your pics
filenames and set/albums names. (Uses SHA1 hashing
algorithm)
-n, --dry-run Dry run. No changes are actually performed.
Information options:
-i TITLE, --title TITLE
Title for uploaded files. Overwrites title set in INI
config file. If not specified and not set in INI file,
it uses filename as title (*Recommended).
-e DESCRIPTION, --description DESCRIPTION
Description for uploaded filesOverwrites description
set in INI file.
-t TAGS, --tags TAGS Space-separated tags for uploaded files. It appends to
the tags defined in INI file.
-l N, --list-photos-not-in-set N
List as many as N photos (with tags) not in set.
Maximum listed photos is 500.
Processing related options:
-r, --drip-feed Wait a bit between uploading individual files.
-p P, --processes P Number of photos to upload simultaneously. Number of
process to assign pics to sets.
-u, --not-is-already-uploaded
Do not check if file is already uploaded and exists on
flickr prior to uploading. Use this option for faster
INITIAL upload. Do not use it in subsequent uploads to
prevent/recover orphan pics without a set.
--no-delete-from-flickr [nodelete]
Do not actually deletepics from flicr.com & mark them
with tag:[nodelete]
-d, --daemon Run forever as a daemon.Uploading every SLEEP_TIME
seconds. Please note it only performs upload/raw
convert/replace.
Handling bad and excluded files:
-b, --bad-files Save on database bad files to prevent continuous
uploading attempts. Bad files are files in your
Library that flickr does not recognize (Error 5) or
are too large (Error 8). Check also option -c.
-c, --clean-bad-files
Resets the badfiles table/list to allow a new
uploading attempt for bad files. Bad files are files
in your Library that flickr does not recognize (Error
5) or are too large (Error 8). Check also option -b.
-s, --list-bad-files List the badfiles table/list.
-g, --remove-excluded
Remove previously uploaded files, that are now being
excluded due to change of the INI file configuration
EXCLUDED_FOLDERS.NOTE: Option --remove-ignored was
dropped in favor of --remove-excluded.
Migrate to v2.7.0:
--add-albums-migrate From v2.7.0 onwards, uploadr adds to Flickr an album
tag to each pic. This option adds such tag to
previously loaded pics. uploadr v2.7.0 will perform
automatically such migration upon first run This
option is *only* available to re-run it, should it be
necessary.
by oPromessa, 2017, 2018
/full/path/to/uploadr.cron
# cron entry (runs at the top of every hour)
0 * * * * /full/path/to/uploadr.cron > /dev/null 2>&1
$ ./uploadr.py -v -d
Inspired by:
Makes dynamic use of the following libraries:
You may use this code however you see fit in any form whatsoever. And enjoy!!!
Q: Who is this script designed for?
Q: Why don't you use OAuth?
Q: Are you a python ninja?
Q: Is this script feature complete and fully tested?
Q: Do I need to install the "Python Module" from DSM Installation Package?
Q: How to automate it with a Synology NAS ?
Q: What if I have different folders to sync?
Q: "my understanding is that this is a sync script, which means when I later delete a pic from a synced folder, it will get deleted from Flickr"
Q: "What about previously existing folders (they didn't seem to get deleted)"
Q: What about when I sync a folder with the same name of a previously existing folder? (you mention getting existing sets from Flickr is managed also
Q: What about when I run the script on ~/pictures/parent_folder/folder_A and then later on ~/pictures/parentfolder will the script recongize the folder_A within parentfolder as being the one it uploaded before becaues its content will have matching checksums?
Q: I thought I read a mention of checksum as a way to detect file modification: what about the same file in 2 different folders, is it then upoad each time (in a set with the folder name) or only once?
Q: How to read the final report:
Initial Found Files:[ 757]
- Bad Files:[ 7] = [ 750]
Note: some Bad files may no longer exist!
Photos count:
Local:[ 750]
Flickr:[ 755] [ 5] Flickr-Local
Not in sets on Flickr:[ 0]
Q: What happens if the local control Database (flickrdb) is deleted?
Q: Is all sensitive information (albums and filenames) masked with the -u option?