Memory Error - EXIF

[johntwoshedsthomas]

I see a similar issue has already been reported here.... https://github.com/richq/folders2flickr/issues/21

... but sounded like you were working on a workaround?

I thought there was a setting just to ignore the EXIF routine entirely when running your python, is that available?

Below is the verbose debug output from the EXIF module...

2014-11-03 19:17:48,803 DEBUG Getting EXIF for D:\Users\dj concise\Pictures\Image Library\20020421\CIMG0064.JPG 2014-11-03 19:17:48,815 DEBUG JPEG format recognized data[0:2]=0xFFD8 2014-11-03 19:17:48,815 DEBUG data[2]=0xFF data[3]=0xE1 data[6:10]=Exif 2014-11-03 19:17:48,815 DEBUG Segment base 0x2 2014-11-03 19:17:48,815 DEBUG APP1 at base 0x2 2014-11-03 19:17:48,815 DEBUG Length: 0xA2 0x34 2014-11-03 19:17:48,815 DEBUG Code: Exif 2014-11-03 19:17:48,815 DEBUG Decrement base by 2 to get to pre-segment header (for compatibility with later code) 2014-11-03 19:17:48,815 DEBUG Endian format is M (Motorola) 2014-11-03 19:17:48,815 DEBUG IFD 0 (Image) at offset 8: 2014-11-03 19:17:48,815 DEBUG Make: (0x010F) ASCII=CASIO COMPUTER CO.,LTD. @ 146 2014-11-03 19:17:48,816 DEBUG Model: (0x0110) ASCII=EX-M2 @ 170 2014-11-03 19:17:48,816 DEBUG Orientation: (0x0112) Short=Horizontal (normal) @ 42 2014-11-03 19:17:48,816 DEBUG XResolution: (0x011A) Ratio=72 @ 178 2014-11-03 19:17:48,816 DEBUG YResolution: (0x011B) Ratio=72 @ 186 2014-11-03 19:17:48,816 DEBUG ResolutionUnit: (0x0128) Short=Pixels/Inch @ 78 2014-11-03 19:17:48,816 DEBUG Software: (0x0131) ASCII=1.01+R @ 194 2014-11-03 19:17:48,816 DEBUG DateTime: (0x0132) ASCII=2002:04:21 00:30:33 @ 216 2014-11-03 19:17:48,816 DEBUG YCbCrPositioning: (0x0213) Short=Centered @ 114 2014-11-03 19:17:48,816 DEBUG ExifOffset: (0x8769) Long=276 @ 126 2014-11-03 19:17:48,818 DEBUG PrintIM: (0xC4A5) Undefined=[80, 114, 105, 110, 116, 73, 77, 0, 48, 50, 53, 48, 0, 0, 0, 4, 0, 1, 0, 22, 0, 22, 0, 2, 1, 0, 0, 0, 1, 0, 5, 0, 0, 0, 1, 1, 1, 0, 0, 0] @ 236 2014-11-03 19:17:48,818 DEBUG Exif SubIFD at offset 276: 2014-11-03 19:17:48,818 DEBUG ExposureTime: (0x829A) Ratio=1/40 @ 654 2014-11-03 19:17:48,818 DEBUG FNumber: (0x829D) Ratio=16/5 @ 662 2014-11-03 19:17:48,818 DEBUG ExposureProgram: (0x8822) Short=Program Normal @ 310 2014-11-03 19:17:48,818 DEBUG ExifVersion: (0x9000) Undefined=0220 @ 322 2014-11-03 19:17:48,818 DEBUG DateTimeOriginal: (0x9003) ASCII=2002:04:21 00:30:33 @ 670 2014-11-03 19:17:48,818 DEBUG DateTimeDigitized: (0x9004) ASCII=2002:04:21 00:30:33 @ 690 2014-11-03 19:17:48,818 DEBUG ComponentsConfiguration: (0x9101) Undefined=YCbCr @ 358 2014-11-03 19:17:48,819 DEBUG CompressedBitsPerPixel: (0x9102) Ratio=12/25 @ 710 2014-11-03 19:17:48,819 DEBUG ExposureBiasValue: (0x9204) Signed Ratio=0 @ 718 2014-11-03 19:17:48,819 DEBUG MaxApertureValue: (0x9205) Ratio=16/5 @ 726 2014-11-03 19:17:48,819 DEBUG MeteringMode: (0x9207) Short=Pattern @ 406 2014-11-03 19:17:48,819 DEBUG LightSource: (0x9208) Short=Unknown @ 418 2014-11-03 19:17:48,819 DEBUG Flash: (0x9209) Short=Flash did not fire, auto mode @ 430 2014-11-03 19:17:48,819 DEBUG FocalLength: (0x920A) Ratio=15/2 @ 734 2014-11-03 19:17:48,982 DEBUG MakerNote: (0x927C) Undefined=[81, 86, 67, 0, 0, 0, 0, 35, 0, 2, 0, 3, 0, 0, 0, 2, 1, 64, 0, 240, ... ] @ 750 2014-11-03 19:17:48,982 DEBUG FlashPixVersion: (0xA000) Undefined=0100 @ 466 2014-11-03 19:17:48,982 DEBUG ColorSpace: (0xA001) Short=sRGB @ 478 2014-11-03 19:17:48,982 DEBUG ExifImageWidth: (0xA002) Long=1600 @ 490 2014-11-03 19:17:48,982 DEBUG ExifImageLength: (0xA003) Long=1200 @ 502 2014-11-03 19:17:48,982 DEBUG InteroperabilityOffset: (0xA005) Long=33158 @ 514 2014-11-03 19:17:48,982 DEBUG FileSource: (0xA300) Undefined=Digital Camera @ 526 2014-11-03 19:17:48,982 DEBUG CustomRendered: (0xA401) Short=Normal @ 538 2014-11-03 19:17:48,982 DEBUG ExposureMode: (0xA402) Short=Auto Exposure @ 550 2014-11-03 19:17:48,983 DEBUG WhiteBalance: (0xA403) Short=Auto @ 562 2014-11-03 19:17:48,983 DEBUG DigitalZoomRatio: (0xA404) Ratio=0 @ 742 2014-11-03 19:17:48,983 DEBUG FocalLengthIn35mmFilm: (0xA405) Short=36 @ 586 2014-11-03 19:17:48,983 DEBUG SceneCaptureType: (0xA406) Short=Standard @ 598 2014-11-03 19:17:48,983 DEBUG GainControl: (0xA407) Short=Low gain up @ 610 2014-11-03 19:17:48,983 DEBUG Contrast: (0xA408) Short=Normal @ 622 2014-11-03 19:17:48,983 DEBUG Saturation: (0xA409) Short=Normal @ 634 2014-11-03 19:17:48,983 DEBUG Sharpness: (0xA40A) Short=Normal @ 646 2014-11-03 19:17:48,983 DEBUG IFD 1 (Thumbnail) at offset 33188: 2014-11-03 19:17:48,983 DEBUG Compression: (0x0103) Short=JPEG (old-style) @ 33198 2014-11-03 19:17:48,983 DEBUG XResolution: (0x011A) Ratio=72 @ 33266 2014-11-03 19:17:48,983 DEBUG YResolution: (0x011B) Ratio=72 @ 33274 2014-11-03 19:17:48,983 DEBUG ResolutionUnit: (0x0128) Short=Pixels/Inch @ 33234 2014-11-03 19:17:48,983 DEBUG JPEGInterchangeFormat: (0x0201) Long=33282 @ 33246 2014-11-03 19:17:48,983 DEBUG JPEGInterchangeFormatLength: (0x0202) Long=8234 @ 33258 2014-11-03 19:17:48,983 DEBUG RecordingMode: (0x0001) Long= @ 196608 2014-11-03 19:17:48,984 DEBUG RecordingMode: (0x0001) ASCII='\x92''K''k''\x06''\x89''\xac''\xcd''\x06''\x99''8''\xbb''\x8a'')''\x19''\x16''\xe9''>''\xeb''\xe0''\x91''\xb8''~''U''\xd5''\x1a''\x8e''H''\xaa''\x92''N''\xd1''\xd8''\xfa''\xe7''\xc6''\x9e''\x15''\x9e''\xf7''\xe1''c''\xeb''d''\xc1''\x03''\xcd''9''L''\x9f''\x9b''\x01''\x86''s''\x8c''\xfb''\x1f''\xce''\xbe''Y''\xd2''\xd6''\xc3''\xc1''\xda''s''\xcd''\xa4''\xe1''/''d''\x1c''\xca''\x17''x''\x90''\xf6''\xc9''\xed''\x8a''\xb2''5''\t''\xb5''\x88''c''\x83''Y''\xba''\xbc''\x9e''\x14''\x91''@''L''\x13''\xdc''v''\x1c''v''\x15''\xb7''\xaf''%''\x9d''\xab''\xc1''o''\xa7''C''\xb1''U''z''\x10''\x0e''\x07''l''\xe2''\xb2''\x92''n''w''G''\x95''\x04''\x9d''\xae''\xad''c''\xc4''<''M''\xe2''[''\xbf''\x12''\xea''\x82''m''z''a''%''\xc1'']''\x88''\x14''m''\x18''\x1d''+''\xcc''S''K''\x96''y''\xa4''\xbb''\x11''\xb1''\x08''H''\xde''\xdc''\x03''^''\xc9''\xe2''{''\x0b''m''2''\xf5''%''\x11''\xc9''(''e'',''\x19''\x97' @ 196608 2014-11-03 19:17:48,984 DEBUG Quality: (0x0002) Ratio= @ 458752 2014-11-03 19:17:48,984 DEBUG RecordingMode: (0x0001) Byte= @ 196608 2014-11-03 19:17:48,984 DEBUG RecordingMode: (0x0001) ASCII='\x92''K''k''\x06''\x89''\xac''\xcd''\x06''\x99''8''\xbb''\x8a'')''\x19''\x16''\xe9''>''\xeb''\xe0''\x91''\xb8''~''U''\xd5''\x1a''\x8e''H''\xaa''\x92''N''\xd1''\xd8''\xfa''\xe7''\xc6''\x9e''\x15''\x9e''\xf7''\xe1''c''\xeb''d''\xc1''\x03''\xcd''9''L''\x9f''\x9b''\x01''\x86''s''\x8c''\xfb''\x1f''\xce''\xbe''Y''\xd2''\xd6''\xc3''\xc1''\xda''s''\xcd''\xa4''\xe1''/''d''\x1c''\xca''\x17''x''\x90''\xf6''\xc9''\xed''\x8a''\xb2''5''\t''\xb5''\x88''c''\x83''Y''\xba''\xbc''\x9e''\x14''\x91''@''L''\x13''\xdc''v''\x1c''v''\x15''\xb7''\xaf''%''\x9d''\xab''\xc1''o''\xa7''C''\xb1''U''z''\x10''\x0e''\x07''l''\xe2''\xb2''\x92''n''w''G''\x95''\x04''\x9d''\xae''\xad''c''\xc4''<''M''\xe2''[''\xbf''\x12''\xea''\x82''m''z''a''%''\xc1'']''\x88''\x14''m''\x18''\x1d''+''\xcc''S''K''\x96''y''\xa4''\xbb''\x11''\xb1''\x08''H''\xde''\xdc''\x03''^''\xc9''\xe2''{''\x0b''m''2''\xf5''%''\x11''\xc9''(''e'',''\x19''\x97' @ 196608 2014-11-03 19:17:48,986 DEBUG RecordingMode: (0x0001) Byte= @ 196608 2014-11-03 19:17:48,986 DEBUG RecordingMode: (0x0001) Short= @ 196608 2014-11-03 19:17:48,986 DEBUG RecordingMode: (0x0001) Byte= @ 196608 2014-11-03 19:17:48,986 DEBUG RecordingMode: (0x0001) Byte= @ 196608 2014-11-03 19:17:48,986 DEBUG Tag 0x0277: (0x0277) Byte=[] @ 86061318 2014-11-03 19:17:49,019 DEBUG Tag 0x0000: (0x0000) Long=[] @ 131079 2014-11-03 19:17:49,019 DEBUG Tag 0x0000: (0x0000) Long=[] @ 0 2014-11-03 19:17:49,086 DEBUG Tag 0x0121: (0x0121) ASCII= @ 33544704 2014-11-03 19:17:49,094 DEBUG Tag 0x8B48: (0x8B48) Long=[] @ 3846151549 2014-11-03 19:17:49,164 DEBUG Tag 0x0122: (0x0122) ASCII= @ 33544704 2014-11-03 19:17:49,216 ERROR error uploading Traceback (most recent call last): File "uploadr.py", line 327, in uploadImage exiftags = exif.process_file(f) File "C:\Users\awise\Downloads\folders2flickr v1\exifreadinit.py", line 221, in process_file hdr.decode_maker_note() File "C:\Users\awise\Downloads\folders2flickr v1\exifread\classes.py", line 401, in decode_maker_note tag_dict=makernote.CASIO) File "C:\Users\awise\Downloads\folders2flickr v1\exifread\classes.py", line 181, in dump_IFD values = self.file.read(count) MemoryError

[richq]

Yeah - I pushed a fix upstream, but they've gone off in the weeds porting to python 3 :(

An option to disable EXIF parsing is probably going to be the safest choice at this stage.

If you could provide an example of a crashy photo, it might help get the root cause problem fixed in the exif-py project too. I only fixed it by code inspection, I didn't actually test it with a known-bad input.

[johntwoshedsthomas]

[johntwoshedsthomas]

Just to verify, there is no way to disable exif processing with a switch in the current script?

[richq]

Thanks for the image. No, there's no way to selectively disable it - the EXIF check is hardcoded in there and would require a code change to disable.