search

t2bot / matrix-media-repo

Highly configurable multi-domain media repository for Matrix.
https://docs.t2bot.io/matrix-media-repo
MIT License
210 stars 79 forks source link

Do some more intelligent guessing on bad EXIF headers #283

Closed Sorunome closed 4 years ago

Sorunome commented 4 years ago

Some images won't thumbnail due to bad exif information (zero-length tag value), perhaps it is a good idea to assume no orientation thin this cases.

Example image: https://sorunome.de/_matrix/media/r0/download/matrix.org/oIbpszMzXTbdQqTxNjMYroUT

Log output:

Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.140 Z] Received request                              contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.140 Z] Host is valid - processing request            contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.140 Z] Getting media record from database            allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: WARN[2020-09-17 11:10:40.140 Z] Attempted to animate a media record that isn't an animated type. Assuming animated=false  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.141 Z] Getting thumbnail record from database        allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.141 Z] Thumbnail does not exist, attempting to generate it  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.141 Z] Generating thumbnail                          allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.141 Z] Processing thumbnail request                  nocontext=true worker_animated=false worker_height=600 worker_media=matrix.org/oIbpszMzXTbdQqTxNjMYroUT worker_method=scale worker_requestId=thumbnail_matrix.org_oIbpszMzXTbdQqTxNjMYroUT_800_600_scale_false worker_width=800
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.142 Z] Using generator: jpgGenerator                 nocontext=true worker_animated=false worker_height=600 worker_media=matrix.org/oIbpszMzXTbdQqTxNjMYroUT worker_method=scale worker_requestId=thumbnail_matrix.org_oIbpszMzXTbdQqTxNjMYroUT_800_600_scale_false worker_width=800
Sep 17 13:10:40 sorunome media_repo[6077]: ERRO[2020-09-17 11:10:40.159 Z] Error generating thumbnail: jpg: error applying orientation: exif: error decoding orientation: loading EXIF sub-IFD: exif: sub-IFD ExifIFDPointer decode failed: zero length tag value  nocontext=true worker_animated=false worker_height=600 worker_media=matrix.org/oIbpszMzXTbdQqTxNjMYroUT worker_method=scale worker_requestId=thumbnail_matrix.org_oIbpszMzXTbdQqTxNjMYroUT_800_600_scale_false worker_width=800
Sep 17 13:10:40 sorunome media_repo[6077]: ERRO[2020-09-17 11:10:40.159 Z] Unexpected error locating media: jpg: error applying orientation: exif: error decoding orientation: loading EXIF sub-IFD: exif: sub-IFD ExifIFDPointer decode failed: zero length tag value  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=oIbpszMzXTbdQqTxNjMYroUT method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 requestedAnimated=true requestedHeight=500 requestedMethod=scale requestedWidth=500 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT server=matrix.org usingForwardedHost=false
Sep 17 13:10:40 sorunome media_repo[6077]: INFO[2020-09-17 11:10:40.159 Z] Replying with result: *api.ErrorResponse &{Code:M_UNKNOWN Message:Unexpected Error InternalCode:M_UNKNOWN}  contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=500&method=scale&width=500" remoteAddr=94.113.160.97 requestId=REQ-43450 resource=/_matrix/media/r0/thumbnail/matrix.org/oIbpszMzXTbdQqTxNjMYroUT usingForwardedHost=false
Sorunome commented 4 years ago

Another example is this: https://sorunome.de/_matrix/media/r0/download/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj

Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.950 Z] Received request                              contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.950 Z] Host is valid - processing request            contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.951 Z] Getting media record from database            allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: WARN[2020-09-19 13:08:17.951 Z] Attempted to animate a media record that isn't an animated type. Assuming animated=false  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.951 Z] Getting thumbnail record from database        allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.951 Z] Thumbnail does not exist, attempting to generate it  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.951 Z] Generating thumbnail                          allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.951 Z] Processing thumbnail request                  nocontext=true worker_animated=false worker_height=240 worker_media=ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj worker_method=scale worker_requestId=thumbnail_ma.neko.bar_svSwmSozpWcABtCEfWYcTgGj_320_240_scale_false worker_width=320
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.952 Z] Using generator: jpgGenerator                 nocontext=true worker_animated=false worker_height=240 worker_media=ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj worker_method=scale worker_requestId=thumbnail_ma.neko.bar_svSwmSozpWcABtCEfWYcTgGj_320_240_scale_false worker_width=320
Sep 19 15:08:17 sorunome media_repo[6077]: ERRO[2020-09-19 13:08:17.973 Z] Error generating thumbnail: jpg: error applying orientation: exif: error decoding orientation: exif: failed to find exif intro marker  nocontext=true worker_animated=false worker_height=240 worker_media=ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj worker_method=scale worker_requestId=thumbnail_ma.neko.bar_svSwmSozpWcABtCEfWYcTgGj_320_240_scale_false worker_width=320
Sep 19 15:08:17 sorunome media_repo[6077]: ERRO[2020-09-19 13:08:17.973 Z] Unexpected error locating media: jpg: error applying orientation: exif: error decoding orientation: exif: failed to find exif intro marker  allowRemote=true contentLength=0 contentType="" host=sorunome.de mediaId=svSwmSozpWcABtCEfWYcTgGj method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 requestedAnimated=true requestedHeight=200 requestedMethod=scale requestedWidth=200 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj server=ma.neko.bar usingForwardedHost=false
Sep 19 15:08:17 sorunome media_repo[6077]: INFO[2020-09-19 13:08:17.973 Z] Replying with result: *api.ErrorResponse &{Code:M_UNKNOWN Message:Unexpected Error InternalCode:M_UNKNOWN}  contentLength=0 contentType="" host=sorunome.de method=GET queryString="height=200&width=200" remoteAddr=94.113.160.97 requestId=REQ-51576 resource=/_matrix/media/r0/thumbnail/ma.neko.bar/svSwmSozpWcABtCEfWYcTgGj usingForwardedHost=false
Half-Shot commented 4 years ago

Ran into this today:

INFO[2020-10-20 11:32:16.578 Z] Downloading object from bucket hsmxmedia: 8f0c3156b25c5e5677dd2f2c451cebd3b816bbda 
INFO[2020-10-20 11:32:16.665 Z] Using generator: jpgGenerator                 nocontext=true worker_animated=false worker_height=96 worker_media=hacklab.fi/LBLtxAUQyrogqrfEpLYdRpwQ worker_method=scale worker_requestId=thumbnail_hacklab.fi_LBLtxAUQyrogqrfEpLYdRpwQ_96_96_scale_false worker_width=96
ERRO[2020-10-20 11:32:16.679 Z] Error generating thumbnail: jpg: error applying orientation: exif: error decoding orientation: exif: failed to find exif intro marker  nocontext=true worker_animated=false worker_height=96 worker_media=hacklab.fi/LBLtxAUQyrogqrfEpLYdRpwQ worker_method=scale worker_requestId=thumbnail_hacklab.fi_LBLtxAUQyrogqrfEpLYdRpwQ_96_96_scale_false worker_width=96
ERRO[2020-10-20 11:32:16.679 Z] Unexpected error locating media: jpg: error applying orientation: exif: error decoding orientation: exif: failed to find exif intro marker  allowRemote=true contentLength=0 contentType="" host=half-shot.uk mediaId=LBLtxAUQyrogqrfEpLYdRpwQ method=GET queryString="height=64&width=64" remoteAddr=95.216.6.98 requestId=REQ-121942 requestedAnimated=true requestedHeight=64 requestedMethod=scale requestedWidth=64 resource=/_matrix/media/r0/thumbnail/hacklab.fi/LBLtxAUQyrogqrfEpLYdRpwQ server=hacklab.fi usingForwardedHost=false