quran / quran_android

a quran reading application for android
http://android.quran.com
GNU General Public License v3.0
1.99k stars 882 forks source link

Compress Quran Images #699

Closed tarekeldeeb closed 7 years ago

tarekeldeeb commented 7 years ago

The generated quran images (png) are not well compressed. I managed to use zopfli from google to compress all png images, reached 75% of the original size. Please have a look here. I already made a branch for width 1024. If, you agree, I can compress all widths for you.

ozbek commented 7 years ago

AFIAK, images are auto-generated on the server side. So, you should probably send patches to another repo.

tarekeldeeb commented 7 years ago

Sorry, I cannot understand. Quran Images has the scripts and fonts to generate the images. Quran_Android team does this job and upload them somewhere. Right?

I'm proposing to compress those generated images before uploading them to user apps. I can compress them for you.

Are we on the same page?

ozbek commented 7 years ago

Yes, most probably you are right. Let's wait for the maintainers to take a look.

ahmedre commented 7 years ago

masha'Allah, excellent suggestion, jazakAllah khairan! should most definitely do this on the existing images on the server (and also make a note of it or put it in quran/quran.com-images).

am curious as to how the win is 75% (since from what i understood reading about zopfli, the win is usually more around 7-8%, and since i had run these through pngcrush) - is that for the first page alone or overall?

do you think it makes more sense for me to just write a script and run it on the server directly (instead of you uploading and then me re-downloading for each set of images on each server?)

jazakAllah khairan!

ahmedre commented 7 years ago

let me answer my own question - the win is ~25% (the output file is ~75% of the original file).

i ran it on one page, and got:

❯ zopflipng --iterations=500 --splitting=3 --filters=01234mepb --lossy_8bit --lossy_transparent page300.png out300.png
Optimizing page300.png
Block splitting last was better
Filter strategy zero: 120580 bytes
Block splitting last was better
Filter strategy one: 147530 bytes
Filter strategy two: 158714 bytes
Block splitting last was better
Filter strategy three: 196259 bytes
Filter strategy four: 165827 bytes
Filter strategy minimum sum: 158585 bytes
Filter strategy entropy: 142677 bytes
Block splitting last was better
Filter strategy predefined: 120580 bytes
Block splitting last was better
Filter strategy brute force: 122713 bytes
Result is smaller
Input size: 165481 (161K)
Result size: 120580 (117K). Percentage of original: 72.866%

insha'Allah will run these on all of the pages (with explicitly using the block splitting last strategy). jazakAllah khairan.

tarekeldeeb commented 7 years ago

Dear Ahmed, Sorry for the late reply. Good for you to reach even better compression result. But I would not go with any lossy compression, though. Did you visually inspect the lossy output?

ahmedre commented 7 years ago

the two images are visually identical (i.e. using an image diffing tool, both are pixel wise identical).

i think the reason is that the two flags actually aren't really lossy:

--lossy_transparent: remove colors behind alpha channel 0. No visual difference, removes hidden information.
--lossy_8bit: convert 16-bit per channel image to 8-bit per channel
ATouhou commented 7 years ago

I love you guys :-)

tarekeldeeb commented 7 years ago

Perfect :-)

ahmedre commented 7 years ago

for future reference: i've updated the command to use 100 iterations instead of 500 (very minimal file size delta (a few bytes per image) between file sizes of 100 iterations versus 500, but a massive speed increase). i also explicitly used the "block splitting last" technique since that's what worked best in the test image.

~/zopfli/zopflipng --prefix=out/ --lossy_transparent --lossy_8bit --splitting=2 --iterations=100 *.png

i'll update this comment with running totals as i update them (this is the zip including all the images, database, etc):

width original size compressed size
1920 165mb 116mb
1280 98mb 73mb
1260 101mb 74mb
1024 75mb 58mb
800 55mb 43mb
512 31mb 25mb
480 29mb 24mb
320 17mb 15mb

todo list:

note that naskh and qaloon are already highly optimized, so this doesn't make a difference for them.

ozbek commented 7 years ago

Has this been applied on the server side already? i.e., new users are getting compressed images already?

ahmedre commented 7 years ago

yes, just replaced all the madani images now.

before:

before

after:

after

also replaced the individual images for people who don't download all the images.

ahmedre commented 7 years ago

jazakumAllah khairan, closing!

tarekeldeeb commented 7 years ago

Jazana wa Eyyakom,

Happy to collaborate with you.

Regards,

Tarek Eldeeb Senior Digital Design Engineer [image: my GPG] http://pgp.mit.edu:11371/pks/lookup?search=0xF1234BFE&op=index&fingerprint=on | [image: my LinkedIn] http://linkedin.com/in/tarekeldeeb | [image: my Tweets] http://twitter.com/tarekeldeeb

2016-11-13 6:58 GMT+02:00 Ahmed El-Helw notifications@github.com:

jazakumAllah khairan, closing!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/quran/quran_android/issues/699#issuecomment-260167223, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFjaVRwpVsX2k0x5xpEKlyu3Jh-VuAMks5q9pjygaJpZM4Kl3HY .

Mr-Ramzan commented 3 years ago

Assalam o Alikum We are a non-profit organization in Pakistan. Recently, we have started working on an app project to teach users Tajweed and Seven/Ten Recitations. We studied publically available apps and came to know about this project from the Quran.com website. I have been looking for ways to incorporate your code into our project and if it is legal to do so. Whilst reading some threads here in the issue section I came to know that your team has stopped committing updates to this repository. We require your permission and assistance once you allow, We can host the code, and files on our servers so it may not add to your costs. Please let me know if there is any way to communicate further. QuraanCollege.com is the name of our organization you may inquire or ask if you have any questions. JazakumAllah khairan