search

stefrobb / folders2flickr

Automatically exported from code.google.com/p/folders2flickr
1 stars 0 forks source link

Too large objects are handled incorrectly #33

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Put large file into the upload queue (like 2.2GB video file)
2. Run the synchronization
3. Follow the debug/error log

What is the expected output? What do you see instead?
Either the object should be uploaded, or it should be skipped because of the 
size.
Instead, I see error messages within the error log like:
(<type 'exceptions.MemoryError'>, MemoryError(), <traceback object at 
0x8955dc4>)
(<type 'exceptions.MemoryError'>, MemoryError(), <traceback object at 
0x8955f7c>)
(<type 'exceptions.OverflowError'>, OverflowError('requested number of bytes is 
more than a Python string can hold',), <traceback object at 0xb6cf0be4>)

...and within the debug.log, I see additional:
2013-09-06 19:10:45,795 ERROR (<type 'exceptions.MemoryError'>, MemoryError(), 
<traceback object at 0x8955f7c>)
2013-09-06 19:10:46,135 DEBUG Getting EXIF for 
/data/fotky/flickr/2013/20130618/DSC_0022.MOV
2013-09-06 19:10:46,149 DEBUG NO_EXIF_HEADER for 
/data/fotky/flickr/2013/20130618/DSC_0022.MOV
2013-09-06 19:10:46,150 DEBUG Uploading image 
/data/fotky/flickr/2013/20130618/DSC_0022.MOV
2013-09-06 19:10:46,150 ERROR (<type 'exceptions.OverflowError'>, 
OverflowError('requested number of bytes is more than a Python string can 
hold',), <traceback object at 0xb6cf0be4>)

What version of the product are you using? On what operating system?
flickr.py reports Rev: 42 (hard to say, whether it's the revision number, or 
the correct answer to life, universe and everything...)
xmltramp.py reports version 2.16

Please provide any additional information below.
I run the script on linux server, ubuntu 13.04.

Original issue reported on code.google.com by rkla...@gmail.com on 6 Sep 2013 at 5:48

GoogleCodeExporter commented 9 years ago 
On a NAS box, I got the Memory Erro at about 350MB in size.   I reencoded the 
affected videos using a smarter encoder (FFMPEG) and reduced their size.

I modified the script to now not attempt upload over 300MB.   I also got it to 
skip windows created AlbumArt and upload more recognised filetypes.   See 
attachment of modified function.

Original comment by mark.gil...@gmail.com on 9 Oct 2013 at 6:48

Attachments: