[2.1] BrowseFolders - archive support in development - problem with filenames with non ASCII char. in archives

[GoogleCodeExporter]

Model:PRS-505 / PRS+ version: 2.1.0 ?  (PRS+ builded with de last files of 
repository)

I think there are two errors in the new functions of BrowseFolders 
(decompression RAR and ZIP files):

- Files rar / zip on SD:
 I see  zip and rar files with the browseFolders. Are displayed as folders. Inside them I see the books. The options "Copy to Internal Memory & Open" and "Copy to Internal Memory" works well, but the option "preview book" does not work. Display the Message: "Unpacking archive failed".

-Files RAR / ZIP in the Internal Memory: 
Works well to see the books. The option "Preview book" opens fine the books, 
but when I go out the book, isn't displayed the another books and folders 
(Browsefolders not works). It is necesary connect to USB (or restart) to return 
to see the folders.

Original issue reported on code.google.com by surkizu@gmail.com on 28 Dec 2011 at 12:14

[GoogleCodeExporter]

Thanks for reporting.  But we are still working on BF/Archives for 505.  We 
know there are problems!  Please wait for the all clear from Mark Nord (who has 
a 505 and is doing the testing).

Original comment by ben.chen...@gmail.com on 28 Dec 2011 at 5:53

• Changed state: Invalid

[GoogleCodeExporter]

Lets have this open while testing for 300/505.

Original comment by Mark.N...@gentlemansclub.de on 28 Dec 2011 at 8:18

• Changed title: 300/505 nightly builds: BrowseFolders - archive support in development
• Changed state: Started

[GoogleCodeExporter]

fixed CB* support,
checked audio-support working,
not tested: books out of archives
open: cb* viewer - translation of "x of y" 

Original comment by Mark.N...@gentlemansclub.de on 28 Dec 2011 at 8:14

[GoogleCodeExporter]

Issue 259 has been merged into this issue.

Original comment by Mark.N...@gentlemansclub.de on 30 Dec 2011 at 8:07

[GoogleCodeExporter]

Issue 259 was:
Reported by surk...@gmail.com

PRS-505 / PRS+ version: 2.1.0 Installer made ​​with the latest files from 
the repository

I see the comic files on BrowseFolder.
The CBR files works perfectly, but the CBZ files will not open.

Original comment by Mark.N...@gentlemansclub.de on 30 Dec 2011 at 8:08

[GoogleCodeExporter]

Can't reproduce this with (correct named and compressed) file.
Tried archives with images in the root and in a folder, both are working.

But found several issues regarding file-extensions:
  * files with capitalized extensions are ignored,
  * files with the wrong extension e.g. *.cbz but actually it's a rar and should be *.cbr will show empty node-list. Probably we can add some kind of "mime-type" check (ifn't there is already something like this) by reading the first bytes of the file and check for "PK", "7z" and "rar!".

and some general problems
  * if the filenames within the archive contains umlauts, they will be shown with the correct icon, but without name in the nodelist (text-encoding!?)
  * rar-archives with very long filenames, containing a folder with long names, containing files with long names (as you get them usually form the dark grey sources of the net) fail with exit-code: 8 - not testes with zip, 7z atm

Original comment by Mark.N...@gentlemansclub.de on 30 Dec 2011 at 8:53

• Changed title: ALL: nightly builds: BrowseFolders - archive support in development

[GoogleCodeExporter]

* Although you already fixed it, we could actually get rid of 
Core.io.extractExtension. There is FileSystem.getExtension, which returns the 
extension in lowercase (available on all models).
* Unfortunately, the built-in FileSystem.getMIMEType won't help us if the 
extension is incorrect, as all it does is return mimetype from a list based on 
the extension (so it doesn't read the file at all). Reading the first bytes of 
a file would be possible, but tricky. Perhaps we could add a fallback to 
core.archiver to try the other unpackers if one fails?

* umlauts... no idea, sorry :/
* Do these archives fail on opening, or when opening a file inside? If the 
latter, we might be able to get around it by moving the ~temp~ folder to the 
root.

Original comment by quisvir on 30 Dec 2011 at 5:15

[GoogleCodeExporter]

Some updates:
  * capitalized file-extensions are now handled

  * RETURNED_LIST is encoded with CP-850; best solution would be a fix in the binaries to have them output UTF-8.

Original comment by Mark.N...@gentlemansclub.de on 2 Jan 2012 at 8:13

[GoogleCodeExporter]

Hi everyone,

I checked the source code and the 7za source code does use unicode for the 
returned list of files in an archive.  However, for some reason, by the time 
this list reaches PRS+ the unicode is messed up.  So the problem may lie in the 
way ARM Linux passes the output.  If so, there isn't anything we can do about 
this (as far as I know!)

Sincerely,
Ben.

Original comment by ben.chen...@gmail.com on 22 Feb 2012 at 11:02

[GoogleCodeExporter]

I'm completely out of my depth here, but I thought I'd posit the thought 
anyway: is this related to the busybox version used, and if so, is there any 
chance we could update it?

Otherwise, perhaps we could (?) change the binaries to write directly to a text 
file, instead of redirecting the output.

Original comment by quisvir on 22 Feb 2012 at 1:15

[GoogleCodeExporter]

Outputting directly to a text file is a good idea.  I will take a look...

Original comment by ben.chen...@gmail.com on 22 Feb 2012 at 2:53

[GoogleCodeExporter]

Changing pages is painfully slow (2.1.01 alpha, PRS950).

Pages load pretty quick when selecting them directly from archive. Changing 
pages takes few times as long. Also returning to archive list takes 
considerably long time (files are on memory card, comic is 130 images in cbz 
archive).

Moreover the times don't seem to be dependent on image size. I tried it on 
1280px wide and scaled down to 600px wide - exactly the same times for the same 
page. Times I got:
1280px: opening page: 2.5s, changing page: 15.4s, returning to images index: 
13.5s
600px: opening: 2.7s, changing: 15.7s, returning: 13.5s.

The times suggest that displaying the page itself doesn't take that long, but 
there is something time-consuming being done both when changing page and when 
returning to list.

Still I appreciate this function very much as it significantly improves comics 
quality. I've been converting comics to pdf earlier but due to no control over 
image size on the device image is slightly scaled and therefore poor (BTW, 
maybe someone knows how to deal with that). I attach screenshots of the same 
page as viewed in cbz and converted to pdf (with ImageMagick).
Still the time required to change pages makes it not very useful for now.

Original comment by kil...@gmail.com on 27 Feb 2012 at 1:00

Attachments:

• prs950cbz.jpg
• prs950pdf.jpg

[GoogleCodeExporter]

Hi kilwar,

When you change pages, what is happening behind the scenes is that the image 
and the thumbnail automatically generated by the system are being deleted.  You 
see, we load the image into the Sony reader's internal library so you can view 
it, but then when you change to the next image, we have to delete it otherwise 
your reader would fill up with images.

(Also, I think that the bigger the archive the slower it takes.  You could 
experiment by breaking the 130 images into four smaller archives and see if 
that makes a difference...)

An alternative would be to leave all the images on the reader (using the /tmp/ 
folder) which only gets cleaned out every time you reboot the reader.

Sincerely,
Ben.

Original comment by ben.chen...@gmail.com on 27 Feb 2012 at 4:44

[GoogleCodeExporter]

If you have control over the process I would suggest deleting the temporary 
copy AFTER page is changed. This way time consuming process would take place 
when one is busy reading a next page anyway. The only cost is having two 
temporary images in internal memory, but I don't think it's a big problem.

Still this makes it impossible to fast-browse a comic so an option to have a 
temp folder you mentioned might still be helpful in some cases.

best,
kilwar

Original comment by kil...@gmail.com on 27 Feb 2012 at 8:06

[GoogleCodeExporter]

I suspect kbook.root.update(kbook.model) is causing the delay here. kilwar, is 
your book cache quite large by any chance?

Ben: optimally, if the user is viewing comics/pictures, we should only update 
the pictures node (if any). The root update was a bit of a lazy suggestion of 
mine at the time ;)

Original comment by quisvir on 28 Feb 2012 at 1:29

[GoogleCodeExporter]

Hi quisvir,

It might be possible to only update the picture node.  But archives can also 
contain books, so any code changes would need to be carefully done!

Sincerely,
Ben.

Original comment by ben.chen...@gmail.com on 28 Feb 2012 at 4:27

[GoogleCodeExporter]

Yes, my book cache is quite large (I get a similar delay always when returning 
to home screen). So if it really is updated every page change that's what is 
responsible for the delay.

If it's only matter of updating the cache it would take roughly the same amount 
of time no matter how many temporary entries are to be deleted, would it? So it 
could be done when one stops reading a comic (likely by returning to home 
screen when the cache is parsed anyway as far as I know).

Another way might be to somehow cheat the cache system. Is it possible to 
create a permanent temporary file which is always listed in the cache and only 
update it when one changes a page? Might be a few such files (for images, for 
books, different file types) if necessary.

Original comment by kil...@gmail.com on 28 Feb 2012 at 6:34

[GoogleCodeExporter]

I pushed a change so that the reader update only happens when the archive is 
closed. Image files are still deleted when moving to the next image.  However, 
it was noticeably faster on my reader (and I have about 260 books on mine).

Original comment by ben.chen...@gmail.com on 28 Feb 2012 at 4:06

[GoogleCodeExporter]

Thank you. I'll test it if I manage to compile the source. I've been trying 
that for a while without any success.

Original comment by kil...@gmail.com on 28 Feb 2012 at 8:29

[GoogleCodeExporter]

I managed to compile my PRS+ build and after your push changing comic pages is 
ca. 5 times faster on my reader making the delay acceptable for casual reading. 
Thank you very much Ben!

Original comment by kil...@gmail.com on 4 Mar 2012 at 2:04

[GoogleCodeExporter]

Excellent!  Glad to be of assistance.

Original comment by ben.chen...@gmail.com on 4 Mar 2012 at 2:26

[GoogleCodeExporter]

Original comment by Mark.N...@gentlemansclub.de on 11 Mar 2012 at 6:59

• Changed title: [2.1] BrowseFolders - archive support in development - problem with filenames with non ASCII char. in archives