Some thumbnails aren't being shown in the web UI.

GoogleCodeExporter commented 9 years ago

Some (but not all) images aren't being previewed on the web UI.

Camlistored shows the same output when uploading both types of images.

In the permanode page for the broken permanode, the preview thumbnail is blank 
and has the url http://localhost:3179/ui/undefined?mh=375&tv=2. Asking for 
http://localhost:3179/ui/thumbnail/<correct hash>/foo.jpg?mh=500&tv=2 gets the 
right image though. The bad URL is all that's on the JS console.

I couldn't reliably reproduce bad images, but I have two samples, one that 
always works and one that always doesn't. They were taken with the same camera 
but with different settings. Maybe it's the metadata? I'm happy to share them 
privately.

cf. https://groups.google.com/forum/#!topic/camlistore/UDtN6XDY7cM

Original issue reported on code.google.com by salman.a...@gmail.com on 20 Jul 2014 at 4:20

Attachments:

screen.png

GoogleCodeExporter commented 9 years ago

Just updated to tip. Now the permanode view is blank for both images with or 
without working thumbs.

JS console says:
Uncaught TypeError: Cannot read property 'image' of null

(Different bug?)

Original comment by salman.a...@gmail.com on 20 Jul 2014 at 4:26

Attachments:

screen2.png

GoogleCodeExporter commented 9 years ago

yeah I can try and debug if you send them to me now, before my memory of the UI 
code becomes stale again

Original comment by mathieu....@gmail.com on 21 Jul 2014 at 5:09

GoogleCodeExporter commented 9 years ago

Yes, feel free to also email them to me.

Original comment by bradfitz on 21 Jul 2014 at 5:44

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

OK, nevermind the 1st comment, that fixed itself. Not sure what was going on.

Also, I found another image which does this. Attached!

Original comment by salman.a...@gmail.com on 22 Jul 2014 at 3:23

Attachments:

c.jpg

GoogleCodeExporter commented 9 years ago

Replying here instead of on https://camlistore-review.googlesource.com/#/c/3271/

I think the issue is the EXIF metadata is just too large with the 3D camera 
metadata in the appdata headers.

I think we can modify our EXIF code to return a special error type that says 
how long it expected it to be (in func newAppSec or somewhere around there) and 
then we can have our own version of image.DecodeConfig if we really need to.

But probably easiest is to just slurp something small to memory (e.g. 256 or 
512 KB) and if it fails, then use the schema.FileReader and pass that to 
images.DecodeConfig or our EXIF reader.  It might be a little slower, but it 
won't be truncated, and we won't have to keep tweaking the size.

Original comment by bradfitz on 22 Jul 2014 at 4:41

GoogleCodeExporter commented 9 years ago

Awesome, I'm happy to do this. I was thinking just pass a FileReader, but that 
sounds better.

Original comment by salman.a...@gmail.com on 22 Jul 2014 at 5:02

GoogleCodeExporter commented 9 years ago

Interesting, I have this gist that dumps the markers from JPEGs 
https://gist.github.com/wathiede/87d8d0458bedf9f60a99

And it seems like the metadata doesn't pass the 1MB boundary:

go run jpeginfo.go -offset -size c.jpg
c.jpg:SOI:0x0:0xe1ff
c.jpg:APP1:0x32f:0x5803
c.jpg:APP1:0x687:0xb4ff
c.jpg:APP1:0x1063b:0xb4ff
c.jpg:UNK0x67:0x10686:0x496f
c.jpg:APP1:0x205ed:0xb4ff
c.jpg:APP1:0x305a1:0xb4ff
c.jpg:APP1:0x40555:0xb4ff
c.jpg:APP1:0x50509:0xb4ff
c.jpg:APP1:0x604bd:0xb4ff
c.jpg:APP1:0x70471:0xb4ff
c.jpg:APP1:0x80425:0xb4ff
c.jpg:APP1:0x903d9:0xb4ff
c.jpg:APP1:0xa038d:0xb4ff
c.jpg:APP1:0xb0341:0xb4ff
c.jpg:APP1:0xc02f5:0xb4ff
c.jpg:APP1:0xd02a9:0x1f54
c.jpg:APP0:0xd56c8:0x1000
c.jpg:DQT:0xd56d8:0x4300
c.jpg:DQT:0xd571b:0x4300
c.jpg:SOF0:0xd575e:0x1100
c.jpg:DHT:0xd576f:0x1f00
c.jpg:DHT:0xd578e:0xb500
c.jpg:DHT:0xd5843:0x1f00
c.jpg:DHT:0xd5862:0xb500
c.jpg:SOS:0xd5917:0xc00
c.jpg:EOI:0x12cb24:0x0

DQT, starts at 874200, which should give us plenty of headroom. Something else 
funny might be going on, but Brad's suggestion would be a foolproof way to 
never have to worry about it again.  I worry not understanding what is going on 
here will cause us to miss a bug where we start slurping in the entire file 
just to read the metadata.

Original comment by couchmo...@gmail.com on 22 Jul 2014 at 6:11

GoogleCodeExporter commented 9 years ago

Sorry, that's because I'm a muppet. This is the bad image... :(

go run jpeginfo.go -offset -size p.jpg 
Desktop/p.jpg:SOI:0:65505
Desktop/p.jpg:APP1:815:856
Desktop/p.jpg:APP1:1671:65460
Desktop/p.jpg:APP1:67131:65460
Desktop/p.jpg:UNK0x67:67206:18480
Desktop/p.jpg:APP1:132589:65460
Desktop/p.jpg:APP1:198049:65460
Desktop/p.jpg:APP1:263509:65460
Desktop/p.jpg:APP1:328969:65460
Desktop/p.jpg:APP1:394429:65460
Desktop/p.jpg:APP1:459889:65460
Desktop/p.jpg:APP1:525349:65460
Desktop/p.jpg:APP1:590809:65460
Desktop/p.jpg:APP1:656269:65460
Desktop/p.jpg:APP1:721729:65460
Desktop/p.jpg:APP1:787189:65460
Desktop/p.jpg:APP1:852649:65460
Desktop/p.jpg:APP1:918109:65460
Desktop/p.jpg:APP1:983569:65460
Desktop/p.jpg:APP1:1049029:65460
Desktop/p.jpg:APP1:1114489:49247
Desktop/p.jpg:APP0:1163736:16
Desktop/p.jpg:DQT:1163752:67
Desktop/p.jpg:DQT:1163819:67
Desktop/p.jpg:SOF0:1163886:17
Desktop/p.jpg:DHT:1163903:31
Desktop/p.jpg:DHT:1163934:181
Desktop/p.jpg:DHT:1164115:31
Desktop/p.jpg:DHT:1164146:181
Desktop/p.jpg:SOS:1164327:12
Desktop/p.jpg:EOI:1673090:0

Original comment by salman.a...@gmail.com on 24 Jul 2014 at 11:12

Attachments:

p.jpg

GoogleCodeExporter commented 9 years ago

Ok, I definitely agree with Brad's suggestion.  These images are just nutty.  I 
think each of the APP1 markers is basically a base64 encoded image taken while 
you move the camera, and then everything after APP0 is the image you see when 
you open it in a viewer.  That is certainly atypical to say the least.  

I'm less worried that something is running off reading too much.  I think this 
images are just strange.

Original comment by couchmo...@gmail.com on 25 Jul 2014 at 3:07

GoogleCodeExporter commented 9 years ago

https://camlistore-review.googlesource.com/3271
e14c122c5255dc7853b07ae8f10d9c35b983a2cf

Original comment by mathieu....@gmail.com on 7 Oct 2014 at 2:19

Changed state: Fixed

GoogleCodeExporter commented 9 years ago

This issue has moved to https://camlistore.org/issue/477

Original comment by bradfitz on 14 Dec 2014 at 11:37

Added labels: IssueMoved, Restrict-AddIssueComment-Commit

google-code-export / camlistore

Some thumbnails aren't being shown in the web UI. #477