Optimize viewer and utilities module

[samehissam]

1- Comment code using java doc. 2- What is the main problems that take much time in execution. 3- Try to solve this problems with available code. 4- If no change in performance change the available code with other that achieve your goal.

[digitamo]

thumbnails performance report

• Problem statement: The thumbnails in large files takes time to load, If you jump to the end of the you will have to wait to all the previous thumbnails to load.
• Solutions:
  • Use "DefaultListModel" instead of array of panels to load the thumbnails changes automatically.
  • load the visible thumbnails only.
  • make sure that the thumbnail resolution is the smallest possible.

Overall notes and opinions:

• This optimization made the thumbnails more responsive and user-friendly.
• The memory performance is not affected by this change by this optimization since we buffer all the thumbnails.
• The user will have to wait a slight time to load thumbnails, But for sure it is not as long as waiting for loading all thumbnails. Suggestions use a fade in animation to make it even more appealing and user-friendly.

[digitamo]

Page Viewer Report

• Manually remove invisible rendered pages:

        commit # 759df774f0d8e35571cb9c03caa8bb401077047b.

  The previewed pages were set to null when it's not visible. The memory at the beginning of is about 50 Mb and increases by viewing more pages, It turned out that null are reference if the variable is assigned according to this question.

• Writing a cutom ListModel::

        commit # 773bded7a27580cc67277d56e2440f3087cf9efd.

  Wrote a custom list model to store data in with only one page to guarantee no more pages are buffered or unintentionally stored in memory, By extending AbstractListModel. Got the same previous result.

• Overall notes and opinions :
  • The problem is mostly a memory leak in original source code -lizardtch's-.
  • The pages that are previously viewed are stored on the memory and upon re-rendering (re-viewing) the page the page in reused from the memory, but this is not desirable because the memory usage will be dependent on the size of the book.
  • Upon inspecting the some classes I noticed that data chunks -like annotations and data other than the book itself- are disabled (commented) in DjVuPage.java class at line 1403. @mahmoudfouadbadwy @YoussefDeabes this may be helpful.
• Next steps:
  • inspect the source code more for the memory leak.
  • inspect the source code of the original software -lizardtech's-.
  • continue optimizing the module.

[digitamo]

Page Viewer Report

• Progress report: commit: 11ce7c0abc94add13fa5b4c3760bbbe336a8d6d1
  • old memory profile (128 pages loaded):
  • The memory is increasing as long as you are flipping pages and a large part of memory is used. In this particular case we have a 128 pages book which will use about 1000 MB memory and the heap will be around 1700 MB.
  • The memory was dependent on the number of book pages, It increases with the number of pages because of buffering all the pages.
  • current memory profile (128 pages loaded):
  • On the idle state the used memory is nearly 24 MB and when flipping pages maximum used memory is about 100 MB and the heap in often around 200Mb.
  • The memory now is not dependent on the number of the book's pages.
  • pages were cached in a CachedInputStream which contained a Datapool object, In other word all the pages that were rendered were cached. This maybe for CPU resources saving, Note that this software was written in late 90s so this was a big deal.
• Overall notes and observations:
  • The memory problem was solved elegantly using a flag called cached in Document class, So that we don't lose the original caching computed pages - Just in case we needed it again -.
  • There is a small Lag when flipping pages.
  • The heap is somehow a lot compared to actual memory used.
• Next steps:
  • solve the lagging problem. It may be simple but it's super annoying.
  • try to decrease the heap.

[digitamo]

Decreasing heap size report:

By default the min and max heap sizes is are 40%, 70% in order, Although you can change this but it's done by changing the JVM parameters in every machine that runs the application.

So a couple of spare mega bytes doesn't worth the effort or the time, And also using this solution may actually slow down other operations that uses a lot of memory suddenly.

• For further reading:
  • see -Xms parameter in this Oracle Doc
  • This Blog post or this

[digitamo]

Lagging Report:

commit : 31c48fe5580abdc3c057b51380fc1c0e0bf6cd68

• Problem statement: The main cause for the problem basically was that rendering pages was done in the same GUI thread, Therefore you can't flip pages unless all pages you've passed through are rendered.
• Old State: Pages were kept in a custom data model that extends AbstractListModel called "PagesModel" and renders each page upon the JList's call.

• Solution:

  • Tested Solutions:

  • Add a AdjustmentChagedListener to the JList's ScrollPane and start a thread to render pages when the user scrolls the page. Unfortunately I forgot to commit this solution and tried the next one.

    Result: The Lag was gone but the memory suffered significantly form this approach in case of flipping pages at a fast pace.

    Explanation: There is a thread created/started by flipping each page, Although the thread is interrupted It's Variables are kept in the memory until the JVM's garbage collector removes them even if there are removed form the used memory the heap is still affected by the significant memory change.

• Implemented Solution: Dedicated a thread for rendering page with a small buffer of pages -static size of 3 pages-.