search

richq / folders2flickr

Upload files to flickr
Other
102 stars 38 forks source link

Uploading without creating sets [OS X] #37

Open madswesti opened 9 years ago

madswesti commented 9 years ago

The upload runs perfectly, but when it tries to create a set, I get the following stack trace:

Traceback (most recent call last): File "./folders2flickr", line 8, in f2flickr.uploadr.main() File "/Users/username/folders2flickr/f2flickr/uploadr.py", line 628, in main tags2set.createSets(uploadedNow, HISTORY_FILE) File "/Users/username/folders2flickr/f2flickr/tags2set.py", line 88, in createSets uploaded = shelve.open( historyFile ) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shelve.py", line 239, in open return DbfilenameShelf(filename, flag, protocol, writeback) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shelve.py", line 223, in init Shelf.init(self, anydbm.open(filename, flag), protocol, writeback) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/anydbm.py", line 85, in open return mod.open(file, flag, mode) gdbm.error: (35, 'Resource temporarily unavailable')

The debug.log says: 2014-10-18 01:10:04,118 DEBUG successful. 2014-10-18 01:10:04,121 DEBUG tags2set: Started tags2set

EDIT I'm just realising that it may not be completely apparent for everybody. I'm on OS X Yosemite with python .7.8 installed via Homebrew.

richq commented 9 years ago

Hmm, looks like it can't write the shelve file. maybe there are some differences in the semantics if it's already opened on Mac? The stack trace looks promising, I'll take a look.

ghost commented 9 years ago

I can confirm this issue on Yosemite with the same python version (2.7.8_2 installed via homebrew), and with exactly the same stacktrace as in the original post by @MadsWesti.

Not sure if this has any relation to the root cause, but the tag that the script is generating contains hashes and forward slashes, i.e. #/Downloaded#Albums/100069383912684544889/Barcelona#13. Also further up in the log I see exceptions.UnicodeEncodeError, but those don't seem to be fatal.

Update: apparently it is a known issue on OSX: https://github.com/shazow/urllib3/issues/63 - that ticket contains a dirty workaround.

What I ended up doing was an even dirtier return in tags2sets.py:line 92:

try:
    uploaded = shelve.open( historyFile )
except Exception, e:
    print e
    print "Aborting createSets()..."
    return
madswesti commented 9 years ago

I don't think the hashes and forward slashes are the issue. I'm currently running the script on my Linux server and everything works with no issues and images are tagged as "#path/to/file/filename.JPG". I think it's related to gdbm, but thats just a hunch.

madswesti commented 9 years ago

@richq: I think I may have solved it. By adding "self.uploaded.close()" at the end of the "upload" function in uploadr.py. Both sets and tags seems to work.