search

sysalto / ReactiveReports

Reactive Reports - a framework developed in Scala, designed for generating reports from code.
Other
16 stars 4 forks source link

adding images performance issue by using drawImage() #11

Closed conniepe closed 5 years ago

conniepe commented 5 years ago

Comments copied from another issue:

conniepe commented on Oct 5 • I found a performance issue when adding a lot of images to the PDF report by using drawImage(). Without adding any images, the performance looks fine. When I tried to add about 20000 images, the size of image is not big (1KB each), the report generation took about 20 mins to finish. Do you have any idea how to improve the performance when adding images?

sysalto commented on Oct 22 • Issue 2 - Regarding adding 20,000 images, did you add the same image 20,000 times, or are they different ? How many different unique images do you have, in the 20,000 ?

sysalto commented on Oct 27 • Our newest version, 1.0.2 was released today. The issue with image optimization was fixed here. After the fix, a report having the same image on 20,000 pages takes only 4 minutes to generate, down from 20 minutes.

Scalaz, another open source software, was used for this optimization. Here is the license for scalaz: Scalaz license

conniepe commented 23 hours ago Thanks for your update. I have just tested the latest version 1.0.2 for the image optimization issue. I was trying to add 20,000 images to the PDF report and there are around 10 different unique images in the 20,000. The report is around 600 pages and each page has about 40 images added. By using 1.0.2 version, It took about 20 mins to generate the report. It doesn't look a good improvement on the performance. Could you help take a look this issue again? Thanks.

sysalto commented 5 years ago

Hi, right now we are working on the tags, which were requested from a different issue. Because it's a new feature, it takes precedence. We'll look at optimization when we find time, after we complete the tags.

conniepe commented 5 years ago

Thanks for the update. Just for you information, the same report (with 20,000 images added) generated by using BIRT (Business Intelligence and Reporting Tools) only takes about 2 mins. It's unacceptable if takes 20 mins or more time to generate the same report by using reactive. Please help take a look the optimization when you have time. Thanks.

sysalto commented 5 years ago

Hi, we are currently working on the tags and other features for accessibility, which you requested. We will handle the speed optimization afterwards. Now, we generated a report with 20,000 images in 4 minutes, having the same image, and only the image, no text or other items. Your report might have other items, and some aspects of your code might be not optimal. In any case, I saw an email from you about this. We can discuss the details on the email thread, and when we fix it, you can close this item.

conniepe commented 5 years ago

Hi, the image performance issue becomes more priority than other accessibility features because we have tried version 1.0.2 and 1.0.3, both them can't generate a report with 20,000 images and text. They do work for a report with only 20,000 images but not work with images and text together. Our report has less 10 different images, but at each row we need to print one of the images at the beginning and follow with some text. When we debug our code we found that the report.render() was taking most of time. If we generate a report with 20,000 images and some text by using version 1.0.2/1.0.3, it will hang on report.render() forever. If we use version 1.0.1, it takes about 20 mins to generate the report. If we use BIRT, it only takes around 2 mins as I mentioned before. We understand you are currently working on the accessibility features as we requested, but since this performance issue is blocking us to generate a large report. Hopefully you can take a look this issue at your earliest convenience. Thanks.

sysalto commented 5 years ago

Hi, The issue was fixed in the new 1.0.4 version. For 20,000 rows and 10 images the report runs in only 35 seconds. Please try again and let me know if the issue was fixed. Thank you for reporting this bug.

conniepe commented 5 years ago

Thanks for your quick response and fixing the image performance issue. I have tested the new 1.0.4 version, it does fix the image issue if the report only contains images without any text by using rocksDB, actually it's pretty fast so I'm happy with your fix, but I found another two issues:

  1. By using rocksDB, it doesn't work if the report contains both image and text for 20,000 rows , but it works for small report only contains 1000 rows with image and text. I just attached an example of our small report, give you some ideas how our report looks like.
  2. By using hashmap instead of rocksDB, it still takes a long time for 20,000 rows with images only.

Could you take these two issues? Thanks a lot. MetricsByCategory.pdf

sysalto commented 5 years ago

We tested in our local with both images and text for 20,000 rows and 10 images and it was around 4-5 secounds. We added our test in the latest commit: https://github.com/sysalto/ReactiveReports/commit/a4e8be54ab90e583b15049a54d60f33249ee1d6a.

conniepe commented 5 years ago

Verified issue has been fixed in the latest version 1.0.5-SNAPSHOT. Thanks for solving this performance issue.