search

brunolojor / jbrout

Automatically exported from code.google.com/p/jbrout
0 stars 0 forks source link

import/refresh error on faulty/bad pictures #38

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Refresh thumbnails of a folder with a damaged picture (example join)
2.
3.

What is the expected output? What do you see instead?

Humm... Since the picture is damaged -> Jbrout should restore the picture
and fix every faulty pixel of the picture! :)

Seriouly, Jbrout should continue to refresh others thumbnails, and display
the error on a log file or on the terminal...

Instead of that, we see a Traceback which DO NOT display the file name...
So it's a long waste of time to isolate the faulty picture!

Traceback (most recent call last):
   File "/usr/lib/jbrout/jbrout.py", line 1898, in
on_selecteur_menu_select_plugin
    ret=callback(l)
   File "jbrout/plugins/__init__.py", line 203, in myCallBack
   File "jbrout/plugins/rotate/__init__.py", line 82, in rebuildThumb
   File "jbrout/jbrout/db.py", line 769, in rebuildThumbnail
   File "jbrout/jbrout/tools.py", line 490, in rebuildExifTB
   File "/usr/lib/python2.5/site-packages/PIL/Image.py", line 1523, in
thumbnail
    self.load()
   File "/usr/lib/python2.5/site-packages/PIL/ImageFile.py", line 207, in load
    raise IOError(error + " when reading image file")
 IOError: decoding error when reading image file

What version of the product are you using? On what operating system?
Ubuntu 159

Please provide any additional information below.

Original issue reported on code.google.com by gautier....@gmail.com on 12 Jan 2009 at 12:38

Attachments:

GoogleCodeExporter commented 9 years ago 
can't download the attachment ;-(
can you send me it by mail ?

Original comment by manat...@gmail.com on 12 Jan 2009 at 8:44

GoogleCodeExporter commented 9 years ago 
in r161 : it displays the full path of the faulted file

> Seriouly, Jbrout should continue to refresh others thumbnails, and display
> the error on a log file or on the terminal...

Since 0.3, and recents changes : there shouldn't be any errors on import (since 
tags
or comment are always decoded to unicode, and exif flash value are no more a 
problem)

The only exception, which will break the import process, should be this one :
importing a image that is not a image (or corrupted like yours)

It's an old debate too ;-)
Importing only valid images, and logging silently faulting pictures, is not a 
good
idea ... I think.
I really prefer that jBrout shows the trouble, "explicit is better than 
implicit".
(the user should correct (or delete) the bad picture)

But sure, the current messagebox is not sexy ;-), or user-friendly ;-)

Original comment by manat...@gmail.com on 12 Jan 2009 at 9:33

GoogleCodeExporter commented 9 years ago 
The very least jBrout can do is give the name of the file. This is not about 
sexiness
or user-friendliness, IMO this is plain politeness.

Original comment by davito...@gmail.com on 12 Jan 2009 at 10:05

GoogleCodeExporter commented 9 years ago 
... but I think jBrout now displays the name of the file, IIUC revision 161 :-)

Original comment by davito...@gmail.com on 12 Jan 2009 at 10:19

GoogleCodeExporter commented 9 years ago 
Also, if there is only one bad image, this behaviour is bearable, but imagine a
collection of thousands of pictures where a few bad pictures are scattered :-( 
This
is not far-fetched, obviously this has happened several times since you 
released 0.3...

The user goes through 1 minute of import and jBrout crashes. The user removes 
the
faulty picture and jBrout crashes at 1'06, the user removes the faulty picture 
and
jBrout crashes at 1'08, he removes the faulty picture and jBrout crashes at 
1'15...
You're lucky nobody invented the computer equivalent of the atomic bomb or of 
the
small pox, manatlan :-D 

So I strongly believe that if jBrout persists in refusing to import a folder 
where
there is a bad picture, jBrout should give a means to easily extract the 
pictures he
is going to refuse.

Original comment by davito...@gmail.com on 12 Jan 2009 at 10:22

GoogleCodeExporter commented 9 years ago 
Last but not least, you are cheating! You sound as if it was either push the bad
pictures under the rug or refuse to load at all. There are intermediate 
solutions,
solutions where jBrout loads all he can, but still tells openly to the user 
that some
pictures were wrong. BTW, isn't this what jBrout is doing currently? What are 
all
those pictures with zigzags and question marks?

Original comment by davito...@gmail.com on 12 Jan 2009 at 10:24

GoogleCodeExporter commented 9 years ago 
- "question marks" is for pictures without "internal exif thumbnail"
- "zigzag"  was for bad pictures (in jbrout 0.2)

you're right ... zigzags shouldn't appear in 0.3 !
(does anybody have zigzags in 0.3 ?)

like always, it's not easier as you say ...
if  there is an exception at import/refresh (and it should be very very very 
rare, ok
? (in my case, 35000 without any trouble)) ... some infos can't be stored in 
the db/xml.
In this particular case : it's about "resolution"

if you take a txt file, rename it with ".jpg" at the end. and try to import it 
: the
exception should be "bad jpeg header"
and no infos can be stored in db/xml.

If I continue to manage bad files (zigzags like in 0.2) : it will generate 
endless
troubles in other features.
I'd like to keep jbrout simple and maintanable.
jBrout is done to manage pictures (real pictures) ... which generally came from 
a cam
(a cam produce real pictures too)
I don't want to overbloat all the code for 0.000001%  of the cases, and one guy 
;-)
(keep in mind that I do that in my spare time, and I earn nothing)

zigzags was a bad idea ... but it's a historical thing. Because, in the past, 
python
libs (exif and iptc) were throwing a lot of unknown exceptions.

btw, a sexier messagebox could be done for this rare case.
in a better world : the messagebox could try to give info about the faulty 
picture
(and try to display it), letting the user the choice to :
- suppress this file, and continue current operation
- or stop the current operation (so the user should try to repair himself)
- or better : put this file in "quarantine" and continue

but I don't want to reintroduce zigzags

Original comment by manat...@gmail.com on 12 Jan 2009 at 11:04

GoogleCodeExporter commented 9 years ago 
Damaged pictures are not exceptional, it's often occur when you burn pictures 
to a
CD-R (which is a really bad idea)... I will try to send you the picture by 
email, but
I think you've just to take one picture and to modify 1 or 2 hex lines to have 
the
same result... :) 

Note that in my case, Jbrout import the file (because the thumbnail is 
correct), but
is not able to display the real picture (or to recreate a thumbnail)...

The only thing very disappointing for me is that jbrout ends the process and I 
have
to restart everything... 

I think zigzags are completely useless... IMO the better way for every task 
(import,
refresh, rotate etc.) is :
 - to log all errors on a file
 - to remove all faulty picture FROM THE jBrout DATABASE
 - to DISPLAY the log file at the end of the process... 
So it's not silent...

So It will be very easy to locate faulty pictures, to try to open it with others
softwares to fix them, or to locate these pictures on an old backup (it's my 
case,
but unfortunately, jBrout has renamed all my pictures :) )

I strongly disagree to propose to delete faulty files... Keeping everything is 
the
only way to be able to retreive data... a faulty picture for jBrout doesn't 
meant
that it's the case for all softwares (for example, EOG is able to open a part 
of the
picture of this issue)

Even Microsoft Word doesn't ask for deleting a corrupted word file!

"I don't want to overbloat all the code for 0.000001%  of the cases, and one 
guy ;-)
(keep in mind that I do that in my spare time, and I earn nothing)"

Hum, if the guy is me, this mean 0 guy, because I have isolated all my damaged
pictures :) But seriously, I'm pretty sure it's not 0.0000001% of the cases.

We all know the work you've done. But I also know that I can't give jBrout to my
mother because she will be lost with all these errors... I really want to help 
you to
improve this software, and it's the only reason why I post all these little 
bugs...

Original comment by gautier....@gmail.com on 13 Jan 2009 at 5:34

GoogleCodeExporter commented 9 years ago 
Ok, at least we know. But I suggest we try to find more meaningful pictograms.
Thumbnails are big enough :-) No hurry, but it would be nice.

Original comment by davito...@gmail.com on 13 Jan 2009 at 8:11

GoogleCodeExporter commented 9 years ago 
I am definitely not suggesting that you manage faulty pictures, sorry if I led 
you to
understand that. What I was suggesting is that when finding a faulty image 
jBrout
could store the file name and a flag saying "bad picture" in the "database". 
Period.
No other information, no <resolution>, no <date>, no <real> (whatever that 
means), no
<t>(ag). So that problematic pictures could be filtered out and they would not
trigger any problem in other features. gautier.avril's suggestion is another 
way to
handle the issue which would do the trick.

manatlan, you must understand that by going open source, your collection is not 
a
reference any more. There are a lot of users out there and some of them are 
doing
things with their pictures that maybe you don't even know are possible. 
Furthermore,
distributing a Windows version of jBrout will certainly bring more problematic
pictures in. One week after releasing 0.3, you have already two users 
complaining
about these issues, but how many have had the same problem but didn't report? 
How
many know enough English to discuss here?

I understand the time argument, and I really thank you for doing an application 
which
is not perfect but is going the right way! But please don't say: "I don't have 
the
time so it will never be done". Brout is Open Source, so make users "pay" for 
it: "I
don't have the time, so if you really want it, do it yourself" :-)

Original comment by davito...@gmail.com on 13 Jan 2009 at 8:35

GoogleCodeExporter commented 9 years ago 
Zigzags are indeed useless in the current state of things. So either remove them
completely or allow to select them. Anyhow, I agree that displaying them among 
valid
thumbnails is very ugly :-)

But I do have quite a few images which display a zigzag. And I happen to know 
exactly
which software I used to edit them: the Gimp! Now who was speaking of bad 
software ;-)

What is strange about those pictures is that asking for a thumbnail rebuild 
corrects
the issue. I checked, when doing a thumbnail rebuild, jBrout rebuilds the whole 
EXIF
data, putting it in a different order, but it as far as I could see nothing was 
lost.

Here is zigzag picture before thumbnail rebuild

Original comment by davito...@gmail.com on 13 Jan 2009 at 9:04

Attachments:

GoogleCodeExporter commented 9 years ago 
THanks ...

Your picture is interesting ... In fact, there is something in your Exif 
internal
thumbnail ... but it seems it's not a valid jpeg (for gtk/pixbuf).

perhaps it's a bug in pyexiv2/exiv2 which is not able to extract rightly this 
data.
perhaps it's a bug in gtk/pixbuf.

But in jBrout, we can't do something ;-(
except this zigzag

Original comment by manat...@gmail.com on 15 Jan 2009 at 10:18

GoogleCodeExporter commented 9 years ago 
Ok. Then I'll renew my request: allow to easily select all "bad" pictures 
(question
marks and zigzags) just as you can select all pictures from a date or all 
pictures
with a tag. This would allow the user to try to fix them in another tool.

Original comment by davito...@gmail.com on 16 Jan 2009 at 8:12

GoogleCodeExporter commented 9 years ago 
Not a bad idea ...
But really hard to do, because the fact that a picture is bad, is not stored in 
the
db/xml.
To do that, we'll need to refresh all pictures to find faulty ones. It can be a 
very
long process (at less as long as a "refresh operation"). Or need to save a new 
state
in the db/xml.

btw, these faulty pictures (which haven't got an exif thumb or a bad one) are 
rarely.
(all pictures which came from a cam are good)

Perhaps, jbrout should try to "rebuild thumb" for these ones at import/refresh
operations.

Original comment by manat...@gmail.com on 16 Jan 2009 at 8:50

GoogleCodeExporter commented 9 years ago 
Yes, you'd need a way to easily find the "bad" pictures. 

I thought about creating a "technical" tag (a tag which would only be in 
jBrout, not
in the pictures). This tag could be "<t>_jBrout: bad thumbnail</t>", or 
"</t>_jBrout:
bad exif</t>", depending on what the problem is. This would avoid changing the 
user
interface. To find the bad pictures, all the user would have to do would be 
select
this technical tag in the list. But I usually dislike those "tricks". I thought 
of a
way to make these technical tags cleaner: add a new <pseudo-tag> element 
instead of a
<t> element, or add a "technical=yes" attribute to the <t> element. Thus jBrout 
would
know for sure which tags come from the pictures and which tags are technical.

If you want to avoid to use tags, you could add a new attribute to the "photo"
element (for example jBroutPictureStatus). Either you add this element to all
pictures and give to it a value for each picure (usually "ok", but could be
"badExif", badThumb", "noThumb"...) or you add it only to "bad" pictures".

Of course populating the database with this new information will mean 
refreshing the
whole database: the information isn't currently in it, jBrout has no way of 
guessing
which pictures are wrong. But if the user does not refresh, all which will 
happen is
that he won't be able to see the wrong pictures. Nothing else will be broken, 
only
the new feature will not work.

Original comment by davito...@gmail.com on 16 Jan 2009 at 10:36

GoogleCodeExporter commented 9 years ago 
Yes, jBrout could try to rebuild automatically pictures with problems. This 
should be
an option, though. jBrout should let the user decide :-)

Original comment by davito...@gmail.com on 16 Jan 2009 at 10:40

GoogleCodeExporter commented 9 years ago 
one more error on import/refresh

Traceback (most recent call last):
   File "./jbrout.py", line 1479, in on_menu_refresh
    self.on_drop_folders_from_os(model,[path])
   File "./jbrout.py", line 1574, in on_drop_folders_from_os
    for nb in iterator:
   File "/home/kefiiir/doc/src/!svn/jbrout-read-only/jbrout/jbrout/db.py", line 128,
in add
    self.__addPhoto( file ,tags,filesInBasket)
   File "/home/kefiiir/doc/src/!svn/jbrout-read-only/jbrout/jbrout/db.py", line 191,
in __addPhoto
    importedTags=node.updateInfo( iii )
   File "/home/kefiiir/doc/src/!svn/jbrout-read-only/jbrout/jbrout/db.py", line 899,
in updateInfo
    nodeComment.text = pc.comment
   File "etree.pyx", line 741, in etree._Element.text.__set__
   File "apihelpers.pxi", line 344, in etree._setNodeText
   File "apihelpers.pxi", line 648, in etree._utf8
 AssertionError: All strings must be XML compatible, either Unicode or ASCII

and I even don't know what picture calls this error

Original comment by Oleg.Bla...@gmail.com on 21 Jan 2009 at 8:15

GoogleCodeExporter commented 9 years ago 
fixed in trunk : picture successfully loaded with a folder refresh

Original comment by tbenita on 12 Jul 2009 at 10:14