Open gebailey opened 1 year ago
rprieto
commented
1 year ago Hi, thank you for raising this. Indeed some of the code regarding keyword management changed between 2.15.0 and 2.16.0. Are you using anything specific regarding keywords? For example picasa.ini files, or photos with embedded IPTC/XMP keywords?
gebailey
commented
1 year ago Are you using anything specific regarding keywords? For example
picasa.inifiles, or photos with embedded IPTC/XMP keywords?
Not to my knowledge, I recently migrated from an older version of thumbsup that didn't support them.
I'm not sure which photos may contain keywords; if there's a way to run thumbsup where it can tell me which photo caused the traceback, I'm happy to do that. But there's 30K+ photos to pick from....
rprieto
commented
11 months ago Sorry for the slow reply. Have you found the cause of the issue? One way you can identify the media file is:
-it) and with the /bin/sh command instead of thumbsupvi /thumbsup/node_modules/thumbsup/src/model/metadata.js// line 29, catch the exception
try {
this.keywords = keywords(exiftool, picasa)
} catch (ex) {
console.log('Exception', exiftool.SourceFile, exiftool, picasa)
throw(ex)
}thumbsup <args>
andybalaam
commented
2 months ago I got the same error today - here is the output of following the steps outlined above:
$ docker run -i -t -v /path/to/input:/input:ro -v "/path/to/output:/output" -u root ghcr.io/thumbsup/thumbsup /bin/sh
/ # vi /thumbsup/node_modules/thumbsup/src/model/metadata.js
/ # su node
/ $ thumbsup --input /input --output /output --entrypoint /bin/sh
⠋ Indexing folder
Resizing media
Creating website
Exception 2015/05/06/IMG_5605.JPG {
SourceFile: '2015/05/06/IMG_5605.JPG',
ExifTool: { ExifToolVersion: 12.6 },
File: {
FileName: 'IMG_5605.JPG',
Directory: '2015/05/06',
FileSize: '1758 kB',
FileModifyDate: '2024:03:24 03:39:33+00:00',
FileAccessDate: '2024:07:13 21:34:19+00:00',
FileInodeChangeDate: '2024:03:25 19:47:08+00:00',
FilePermissions: '-rw-r--r--',
FileType: 'JPEG',
FileTypeExtension: 'jpg',
MIMEType: 'image/jpeg',
ExifByteOrder: 'Little-endian (Intel, II)',
CurrentIPTCDigest: 'a517155d87735f05af4a22d2a7af790c',
ImageWidth: 3264,
ImageHeight: 2448,
EncodingProcess: 'Baseline DCT, Huffman coding',
BitsPerSample: 8,
ColorComponents: 3,
YCbCrSubSampling: 'YCbCr4:2:2 (2 1)'
},
EXIF: {
Make: 'Canon',
Model: 'Canon DIGITAL IXUS 80 IS',
Orientation: 'Horizontal (normal)',
XResolution: 180,
YResolution: 180,
ResolutionUnit: 'inches',
Software: 'Shotwell 0.18.0',
ModifyDate: '2015:05:06 15:11:56',
YCbCrPositioning: 'Centered',
ExposureTime: '1/60',
FNumber: 2.8,
ISO: 125,
ExifVersion: '0220',
DateTimeOriginal: '2015:05:06 15:11:56',
CreateDate: '2015:05:06 15:11:56',
ComponentsConfiguration: 'Y, Cb, Cr, -',
CompressedBitsPerPixel: 3,
ShutterSpeedValue: '1/60',
ApertureValue: 2.8,
ExposureCompensation: 0,
MaxApertureValue: 2.8,
MeteringMode: 'Multi-segment',
Flash: 'Auto, Did not fire',
FocalLength: '6.2 mm',
UserComment: '',
FlashpixVersion: '0100',
ColorSpace: 'sRGB',
ExifImageWidth: 3264,
ExifImageHeight: 2448,
InteropIndex: 'R98 - DCF basic file (sRGB)',
InteropVersion: '0100',
RelatedImageWidth: 3264,
RelatedImageHeight: 2448,
FocalPlaneXResolution: 14506.66667,
FocalPlaneYResolution: 14485.2071,
FocalPlaneResolutionUnit: 'inches',
SensingMethod: 'One-chip color area',
FileSource: 'Digital Camera',
CustomRendered: 'Normal',
ExposureMode: 'Auto',
WhiteBalance: 'Auto',
DigitalZoomRatio: 1,
SceneCaptureType: 'Standard',
Compression: 'JPEG (old-style)',
ThumbnailOffset: 3402,
ThumbnailLength: 5215,
ThumbnailImage: '(Binary data 5215 bytes, use -b option to extract)'
},
MakerNotes: {
MacroMode: 'Normal',
SelfTimer: 'Off',
Quality: 'Fine',
CanonFlashMode: 'Auto',
ContinuousDrive: 'Single',
FocusMode: 'Single',
RecordMode: 'JPEG',
CanonImageSize: 'Large',
EasyMode: 'Full auto',
DigitalZoom: 'None',
Contrast: 'Normal',
Saturation: 'Normal',
Sharpness: 0,
CameraISO: 'Auto',
MeteringMode: 'Evaluative',
FocusRange: 'Auto',
AFPoint: 'Face Detect',
CanonExposureMode: 'Easy',
LensType: 'n/a',
MaxFocalLength: '18.6 mm',
MinFocalLength: '6.2 mm',
FocalUnits: '1000/mm',
MaxAperture: 2.8,
MinAperture: 8,
FlashBits: '(none)',
FocusContinuous: 'Single',
AESetting: 'Normal AE',
ImageStabilization: 'On',
ZoomSourceWidth: 3264,
ZoomTargetWidth: 3264,
SpotMeteringMode: 'Center',
ManualFlashOutput: 'n/a',
FocalType: 'Zoom',
FocalLength: '6.2 mm',
FocalPlaneXSize: '5.84 mm',
FocalPlaneYSize: '4.39 mm',
AutoISO: 116,
BaseISO: 100,
MeasuredEV: 9.53,
TargetAperture: 2.8,
TargetExposureTime: '1/60',
ExposureCompensation: 0,
WhiteBalance: 'Auto',
SlowShutter: 'Off',
SequenceNumber: 0,
OpticalZoomCode: 0,
FlashGuideNumber: 0,
FlashExposureComp: 0,
AutoExposureBracketing: 'Off',
AEBBracketValue: 0,
ControlMode: 'Camera Local Control',
FocusDistanceUpper: '2.65 m',
FocusDistanceLower: '0 m',
FNumber: 2.9,
ExposureTime: '1/64',
BulbDuration: 0,
CameraType: 'Compact',
AutoRotate: 'None',
NDFilter: 'Off',
SelfTimer2: 0,
FlashOutput: 0,
CanonImageType: 'IMG:DIGITAL IXUS 80 IS JPEG',
CanonFirmwareVersion: 'Firmware Version 1.01',
FileNumber: '100-5605',
OwnerName: '',
ISO: 117,
Rotation: 0,
CameraTemperature: '24 C',
CanonModelID: 'PowerShot SD1100 IS / Digital IXUS 80 IS / IXY Digital 20 IS',
AFAreaMode: 'Face Detect AF',
NumAFPoints: 9,
ValidAFPoints: 1,
CanonImageWidth: 3264,
CanonImageHeight: 2448,
AFImageWidth: 320,
AFImageHeight: 240,
AFAreaWidths: '28 18 18 18 18 18 18 18 18',
AFAreaHeights: '28 18 18 18 18 18 18 18 18',
AFAreaXPositions: '12 0 18 -18 0 18 -18 0 18',
AFAreaYPositions: '-66 -18 -18 0 0 0 18 18 18',
AFPointsInFocus: 0,
PrimaryAFPoint: 0,
ThumbnailImageValidArea: '0 0 0 0',
DateStampMode: 'Off',
MyColorMode: 'Off',
FirmwareRevision: '1.01 rev 1.00',
Categories: 'People',
ImageUniqueID: '95d4177341e5511613357aa2e83a9160',
VRDOffset: 0
},
XMP: { XMPToolkit: 'XMP Core 4.4.0-Exiv2', Subject: [ 70 ] },
IPTC: {
Keywords: 70,
OriginatingProgram: 'Shotwell',
ProgramVersion: '0.18.0'
},
Composite: {
DriveMode: 'Single-frame Shooting',
ISO: 116,
Lens: '6.2 - 18.6 mm',
ShootingMode: 'Full auto',
Aperture: 2.8,
ImageSize: '3264x2448',
Megapixels: 8,
ScaleFactor35efl: 6.1,
ShutterSpeed: '1/60',
Lens35efl: '6.2 - 18.6 mm (35 mm equivalent: 37.5 - 112.6 mm)',
CircleOfConfusion: '0.005 mm',
DOF: '1.64 m (0.90 - 2.53 m)',
FOV: '51.2 deg',
FocalLength35efl: '6.2 mm (35 mm equivalent: 37.5 mm)',
HyperfocalDistance: '2.77 m',
LensID: 'Unknown 6-18mm',
LightValue: 8.6
}
} {}
Unexpected error value.split is not a function
┌───────────────────────────────────────────────────────────────────────────┐
│ │
│ Something went wrong! │
│ │
│ An unexpected error occurred and the gallery didn't build successfully. │
│ This is most likely an edge-case that hasn't been tested before. │
│ │
│ Please check the logs at /output/thumbsup.log. │
│ To help improve thumbsup and hopefully resolve your problem, │
│ you can raise an issue at https://github.com/thumbsup/thumbsup/issues. │
│ │
└─────────────────────────────────────Thank you for sharing the EXIF data.
The root cause seems to be that the keyword stored as IPTC/XMP is the value 70 (a number, not a string). I can already reproduce the issue with a unit test. I need to read the spec to understand if this is expected, and if we should safely convert it to a string ("70").
Do you know why the keyword might be a number?
gebailey
commented
2 months ago Sorry I forgot to circle back to this after the suggestion. The output of my run with the diagnostic logging:
thumbsup:info Exception 2021/20210727_GregBrendaCollage/1993-02 Brenda & Greg.jpg {
SourceFile: '2021/20210727_GregBrendaCollage/1993-02 Brenda & Greg.jpg',
ExifTool: { ExifToolVersion: 12.5 },
File: {
FileName: '1993-02 Brenda & Greg.jpg',
Directory: '2021/20210727_GregBrendaCollage',
FileSize: '68 kB',
FileModifyDate: '2021:07:27 17:35:26-07:00',
FileAccessDate: '2021:07:27 17:35:26-07:00',
FileInodeChangeDate: '2021:07:28 16:27:09-07:00',
FilePermissions: '-rw-r--r--',
FileType: 'JPEG',
FileTypeExtension: 'jpg',
MIMEType: 'image/jpeg',
ExifByteOrder: 'Little-endian (Intel, II)',
CurrentIPTCDigest: '8cb7b5afb3b75357bd16e6dbcfdd002c',
ImageWidth: 435,
ImageHeight: 464,
EncodingProcess: 'Baseline DCT, Huffman coding',
BitsPerSample: 8,
ColorComponents: 3,
YCbCrSubSampling: 'YCbCr4:2:0 (2 2)'
},
JFIF: {
JFIFVersion: 1.01,
ResolutionUnit: 'inches',
XResolution: 290,
YResolution: 290
},
EXIF: {
Software: 'Elements Organizer 8.0',
ModifyDate: '2010:09:17 13:16:52'
},
XMP: {
XMPToolkit: 'Adobe XMP Core 4.2.2-c063 53.360203, 2008/10/08-10:33:40 ',
CreatorTool: 'Elements Organizer 8.0',
ModifyDate: '2010:09:17 13:16:52-07:00',
MetadataDate: '2010:09:17 13:16:52-07:00',
NativeDigest: '256,257,258,259,262,274,277,284,530,531,282,283,296,301,318,319,529,532,306,270,271,272,305,315,33432;C452906BC33C5B3B1A74C8101C9440F7',
Subject: [ 1993 ]
},
IPTC: { ApplicationRecordVersion: 2, Keywords: 1993 },
Photoshop: { IPTCDigest: '8cb7b5afb3b75357bd16e6dbcfdd002c' },
Composite: { ImageSize: '435x464', Megapixels: 0.202 }
} {} +16s
thumbsup:error TypeError: value.split is not a function
thumbsup:error at makeArray (/thumbsup/node_modules/thumbsup/src/model/metadata.js:142:47)
thumbsup:error at arrayMap (/thumbsup/node_modules/lodash/lodash.js:653:23)
thumbsup:error at map (/thumbsup/node_modules/lodash/lodash.js:9622:14)
thumbsup:error at Function.flatMap (/thumbsup/node_modules/lodash/lodash.js:9325:26)
thumbsup:error at /thumbsup/node_modules/lodash/lodash.js:4430:28
thumbsup:error at arrayReduce (/thumbsup/node_modules/lodash/lodash.js:697:21)
thumbsup:error at baseWrapperValue (/thumbsup/node_modules/lodash/lodash.js:4429:14)
thumbsup:error at LodashWrapper.wrapperValue (/thumbsup/node_modules/lodash/lodash.js:9114:14)
thumbsup:error at keywords (/thumbsup/node_modules/thumbsup/src/model/metadata.js:104:53)
thumbsup:error at new Metadata (/thumbsup/node_modules/thumbsup/src/model/metadata.js:31:19) +0ms
thumbsup:error
thumbsup:error Unexpected error value.split is not a function +0ms
thumbsup:error
thumbsup:error ┌───────────────────────────────────────────────────────────────────────────┐
thumbsup:error │ │
thumbsup:error │ Something went wrong! │
thumbsup:error │ │
thumbsup:error │ An unexpected error occurred and the gallery didn't build successfully. │
thumbsup:error │ This is most likely an edge-case that hasn't been tested before. │
thumbsup:error │ │
thumbsup:error │ Please check the logs at /output/thumbsup.log. │
thumbsup:error │ To help improve thumbsup and hopefully resolve your problem, │
thumbsup:error │ you can raise an issue at https://github.com/thumbsup/thumbsup/issues. │
thumbsup:error │ │
thumbsup:error └───────────────────────────────────────────────────────────────────────────┘
thumbsup:error +7msHi @gebailey, thank you. The good news is that it's the same cause as @andybalaam: the IPTC/XML keyword is a number. Same question: does it make sense in your context?
I tried modifying Thumbsup to always parse keywords as a string and it removes the crash in my local test. I do want to make sure it's the appropriate fix.
gebailey
commented
2 months ago the IPTC/XML keyword is a number. Same question: does it make sense in your context?
I'm not sure. I don't know what that keyword represents or how Thumbsup would make use of it...
andybalaam
commented
2 months ago The root cause seems to be that the keyword stored as IPTC/XMP is the value
70(a number, not a string). ... Do you know why the keyword might be a number?
No idea - I don't know what it means at all :-) I didn't fiddle with this data so it must have been added by my phone.
rprieto
commented
2 months ago I'm not sure what those numeric keywords are, although in the second case it looks like a year.
In thumbsup you can use IPTC/XMP keywords to group photos into albums, such as one album per keyword. See https://thumbsup.github.io/docs/3-configuration/album-settings/.
In any case, the Exiftool docs confirm that numbers are not converted to strings regardless of the IPTC spec.
https://exiftool.org/exiftool_pod.html#Input-output-text-formatting
Note that ExifTool quotes JSON values only if they don't look like numbers (regardless of the original storage format or the relevant metadata specification).
I will push a fix to handle this properly.
Bug description
I just recently updated from an old thumbsup version. When I run it, I get the following traceback:
I'm not much of a JavaScript developer, but it seems related to the keyword handling. Version 2.15.0 runs without errors if I specify that version of the container.
Steps to reproduce
I'm using the provided docker image, and invoking it like this:
If I instead specify a different image
ghcr.io/thumbsup/thumbsup:2.15.0, the problem does not occur. The exception happens with both2.16.0andlatest.I'm not sure which image(s) are causing the problem; I just recently updated from an older version and hit this. Is there some way to see what image(s) is causing the traceback?