quran / quran_android

a quran reading application for android
http://android.quran.com
GNU General Public License v3.0
1.95k stars 881 forks source link

Abu 'Amr al-Basri #1211

Open saqfish opened 4 years ago

saqfish commented 4 years ago

There are some people who recite/read Abu 'Amr with either Ad-Duri or As-Susi. I would like to contribute my time into adding those two. Is there a detailed process for the mushaf images? I know Saudi has both high quality Ad-Duri and As-Susi images. Thank you for hard work.

ahmedre commented 4 years ago

salam 3alaikum, do you have images in high quality? if so, can you give a sample? i can in sha' Allah give some ideas on how to move forward with this. walsalam 3alaikum.

saqfish commented 4 years ago

How about these:

https://archive.org/details/FP174233c/page/n484

This zip contains large images of the pages:

https://archive.org/download/FP174233c/174233c_jp2.zip

saqfish commented 4 years ago

These are Duri 750x1128:

https://images.qurancomplex.gov.sa/publications/douri/750/jpg_90/0007.jpg

ahmedre commented 4 years ago

wa3laikum alsalam, sorry for the delayed reply - these may work, i am worried though that the quality won't be great on some of today's phones, but we can try it if there aren't higher quality images.

my suggestions:

  1. try out the images on a phone with a good resolution and see how they look like (both in portrait and landscape). if it looks ok, great. but if it looks too grainy, we need better images.

  2. when we find good images, we'll need to cut out the pages themselves (i.e. remove the surrounding borders since otherwise, there'll be a lot of free space on each page). i know there are ways to automate this, but I don't know enough about this.

  3. for detecting the location of every ayah, we can use the ayah-detection scripts.

  4. after that, it's pretty straight forward - updating the pages or the ayah numbers if necessary, and figuring out how we'll ship it - whether in the current app (my preference since i want to move away from solo apps) or as a solo app.

please let me know if i can help with anything. jazakAllah khairan, walsalam 3alaikum.

ducaale commented 4 years ago

I could help automate removing surrounding borders by using opencv if its needed

AbdullahM0hamed commented 3 years ago

Decent images of the mushaf in the riwaayah of susi, can be extracted from https://epub.qurancomplex.gov.sa/issues/qiraat/sousi/offline/download.pdf.

It looks good in the app too, here is a screenshot how of a page I cropped and removed the background from looks in the app: Screenshot_20201022-222055_Quran

Your statement:

i know there are ways to automate this, but I don't know enough about this.

suggests that this process (of removing borders, etc) is/was done manually, is that still the case? Assuming it is, I have noted that pages on the left side, all have their borders in the same, fixed position and likewise the same applies to pages on the right side. The first two pages (Surah Fatihah, and the first page of Surah Baqarah) are the exceptions to this.

AbdullahM0hamed commented 3 years ago

I have cropped the pages and removed the backgrounds and borders (programmatically, except the first two which I did manually), here is a zip of the images that were produced: http://www.mediafire.com/file/f2hk7j4dgoetcne/Mushaf_images_riwaayah_of_susi.zip/file

Some screenshots of how they look in-app: Screenshot_20201023-214234_Quran Screenshot_20201023-214227_Quran

And the script I used (should not be used on the first two pages): http://www.mediafire.com/file/ceo3ee0yrtixip0/make-transparent.py/file - not great in terms of speed (should've probably used C/C++ for this. Although I did it on my phone so it'd probably be much faster on PC)

I will be attempting to use the ayah-detection scripts over the weekend, and will post here if I succeed, In Sha Allah

Edit: Better transparent images (the old ones were less smooth, which was noticeable in landscape mode): http://www.mediafire.com/file/zfjwl3d2goksdfc/Mushaf_images_riwaayah_of_susi_v2.zip/file,

Generated with: http://www.mediafire.com/file/jndixvlx98lrmdg/make-transparent-v2.py/file

saqfish commented 3 years ago

Looks great

AbdullahM0hamed commented 3 years ago

I'm having trouble installing opencv, so I guess I'll have to leave ayah-detection to someone else

Edit: I have managed to get it installed, Alhamdulillah

AbdullahM0hamed commented 3 years ago

So is there any procedure for highlighting ayaat that span multiple pages? It has come to my attention that that the 218th ayah of Al Baqarah goes from the end of page 34 (counting from Surah Fatihah as page 1) to the first 3 lines of page 35

ahmedre commented 3 years ago

jazakumAllah khairan - yes, the ayah-detection scripts should be able to handle that case (since we had that case with Sheremerly before). if you have trouble with opencv, I can upload a Dockerfile that might help.

AbdullahM0hamed commented 3 years ago

So should it be fine if they have separate highlights? If so, Alhamdulillah. I managed to get opencv working, and I am using the ayah-detection scripts. At the moment I'm going through the pages which haven't detected properly so I can do detection on them again later

benomaire commented 3 years ago

جزاكم الله خيرا Good job.

I have 2 comments now:

  1. Consider centering pages 1 and 2. I think this will look better.
  2. I think it is possible to keep some colors. see the attached image. The color of ayah marker may make it easier for you in ayah-detection as well. I prepared the image using gimp and pngquant. page003.zip
AbdullahM0hamed commented 3 years ago

I'm having some issues with ayaat that begin at the end of a line with a two-letter word, such as: Screenshot_20201026-163328_Quran Screenshot_20201026-163316_Quran

But ayat.py detects everything perfectly, and lines.py gets the line count correct (as well as the general area of the line) - is there some value I could try to tweak to correct this, or should I just rectify them manually?

benomaire commented 3 years ago

This may be the same issue reported by @nacer80 here. If it is, then @ahmedre should have fixed it.

BTW, why don't you try find_ayat_v2.py? Also, here are some values you could tweak (BUT these values showed the error I quoted above). However, I think these values miss less Harakat on top.

ahmedre commented 3 years ago

sorry i had forgotten to push this fix - please try again now.

+1 regarding find_ayat_v2.py - i prefer it to be honest, but sometimes it is either too zealous or under zealous and you have to fall back to the v1 script for those.

AbdullahM0hamed commented 3 years ago

Alhamdulillah, I've managed to finish: http://www.mediafire.com/file/axpmy781khabo3d/Mushaf_images_and_database_riwaayah_of_susi.zip/file

BTW, why don't you try find_ayat_v2.py?

find_ayat_v2.py errored out on the first page, but ayat.py didn't, so it seemed like I'd probably have an easier time with ayat.py

benomaire commented 3 years ago

ما شاء الله جزاك الله خيرا

I think the next steps are:

  1. Preparing audio. I suggest starting with Sheikh Abdulrasheed Soufi. I guess @nacer80 can help on that.
  2. Preparing list of number of ayat per page, first ayah of each page, new rob3/thomn values, etc. These should allow the app to build correct list of surah, ayah, and corresponding pages. May be @ahmedre can comment more on this.
  3. Preparing ayah mappings similar to what @Hiyaro is doing for Warsh. This should be helpful to use current tafsir/translations after updating the app. The following resource is useful starting from page 385: أحلى دروسي في رواية السوسي من قراءة أبي عمرو البصري

Other minor comments: a. Is it possible to increase the font of pages 1 and 2? b. Please check these images if the quality is good (Half the size). I just compressed those you uploaded.

AbdullahM0hamed commented 3 years ago

a. Is it possible to increase the font of pages 1 and 2?

Done, and updated the database: http://www.mediafire.com/file/cwe8d5ubfi22o9m/Mushaf_images_and_database_riwaayah_of_susi_v2.zip/file

Please check these images if the quality is good (Half the size). I just compressed those you uploaded

The quality is fine, I ran pngquant on the first two pages to match as well.

I suggest starting with Sheikh Abdulrasheed Soufi.

I was also thinking that it would be a good idea to start with him. I guess I'll take a look at https://github.com/ahmedre/quran_timing_files - no idea how long this will take.

Preparing list of number of ayat per page, first ayah of each page, new rob3/thomn values, etc. These should allow the app to build correct list of surah, ayah, and corresponding pages. May be @ahmedre can comment more on this.

I have been preparing the ayaat changes for the Juz section. I will go back and start adding the starting ayaat of each page too

This should be helpful to use current tafsir/translations after updating the app.

I'm a bit confused about this. Looking at the translation in-app, it appears that actual (hafs) text is used - does this require us to provide the soosi Arabic text?

And also for variations in meaning (such as pronouns changing from 2nd person to 3rd person etc) - do they have to be provided as well?

nacer80 commented 3 years ago

السلام عليكم ورحمة الله وبركاته بارك الله فيكم و وجزاكم الله خيرا. My brothers about timing data base you can use this application is very useful I used it before for Hafs but you can make some change in text file to work with other Rewayet.

AbdullahM0hamed commented 3 years ago

@nacer80 جزاك الله خيرا - As a person who uses their phone to do almost everything, this will be of tremendous help

AbdullahM0hamed commented 3 years ago

I forgot to ask, but how should ayaat spanning 2 pages be handled in the audio database?

AbdullahM0hamed commented 3 years ago

I have this strange issue on Ayah 16, page 4 where if I select the ayah, it plays the correct part of the recording but it doesn't highlight the Ayah (with green), and goes back to page 3 (but doesn't highlight anything here either). I've checked that this Ayah is properly labelled in both the page database and the audio database as 16, any idea what could be causing this issue? (and the ayah doesn't span multiple pages)

nacer80 commented 3 years ago

السلام عليكم ورحمة الله I think the problem with the timing database file , can you please send me the file to check it.

AbdullahM0hamed commented 3 years ago

Here you go: http://www.mediafire.com/file/hfxm732ift6kgc4/abdurrashid_sufi.db/file

You will also need the Mushaf Images and database for this riwaayah, as well as the audio of Surah Baqarah as recited by Sh Abdurrashid Ali Sufi

AbdullahM0hamed commented 3 years ago

OK, so it's definitely not the timing database - checking with other audio also has the issue where highlighting disappears just before that, then reappears after the ayah in question has finished playing.

Logically, the next assumption would be that this is an issue with the database of the ayahinfo for the pages - but I have triple-checked that specific Ayah, and the information associated with it is definitely correct.

ahmedre commented 3 years ago

yeah - we don't support this yet but we'll have to in sha' Allah - probably take the time of the page n + 1 and either keep it in the database or just keep it in a file with the ayah number (guessing there aren't a massive number of these - at least hoping) and we can figure out what to do there.

ducaale commented 3 years ago

I forgot to ask, but how should ayaat spanning 2 pages be handled in the audio database?

I am interested in how this will be resolved since the same solution would benefit Shemerly Mushafs. See https://github.com/quran/quran_android/issues/665#issuecomment-541794496

AbdullahM0hamed commented 3 years ago

guessing there aren't a massive number of these

You appear to be correct, these are the ones I've found:

Surah 2, Ayah 218, pages 34 - 35 Surah 4, Ayah 44, pages 85 - 86 Surah 14, Ayah 27, pages 258 - 259 Surah 24, Ayah 36, pages 354 - 355 Surah 24, Ayah 42, pages 355 - 356

AbdullahM0hamed commented 3 years ago

In other good news, I've found a doc file at qurancomplex which has the actual soosi text - this makes it feasible for this riwaayah to have its own text database like the hafs version, for such things such as sharing text).

I have converted it to a text file and the text is perfectly fine for copying and whatnot. I will attempt to make a quran.ar.db later (although I'm not quite sure what the following tables are for: verses_segdir, verses_segments).

Edit: Updated the text file to remove unnecessary things such as 'center000', which presumably came from formatting. Can confirm there are no letters of English remaining.

AbdullahM0hamed commented 3 years ago

Here is quran.ar.uthmani.v2.db - this copies/shares the correct text, and also makes it so that search finds the correct ayaat with the correct Ayah numbers.

Generated using this script from this file (the other one still had some stuff that needed to be sanitised)

AbdullahM0hamed commented 3 years ago

Ok, so with my current timing database which I made with a modified version of the app nacer sent (soosi text, and Ayah counts) - if you highlight Surah Ma'idah, Ayah 121 and press play, the app (Quran Android) crashes. This is specific to this audio database, as it doesn't occur with other reciters. A glance at the timing database shows that the value for that Ayah is not out of place, and the ayah splitter plays it as it should.

Normally, in this case I'd also check the logs - but the problem is that for the past several months, my phone won't output anything for apps that crash, unfortunately.

Any help is appreciated.

nacer80 commented 3 years ago

  السلام عليكم ورحمة الله وبركاته 

My brother, if database file works with ayah splitter app it means no problem with your database file.

I will send you database editor works with windows is very helpful to edit database files. SQLiteDatabaseBrowserPortable.zip

AbdullahM0hamed commented 3 years ago

@nacer80 you are right, it turns out the issue is withe Ayah-page mapping of the app. I've noticed all the errors have been on pages where the number of the ayah in question in the soosi mushaf are numbers in the Hafs version that would be on a different page

AbdullahM0hamed commented 3 years ago

page_ayaat_start.txt

AbdullahM0hamed commented 3 years ago

Preparing list of number of ayat per page, first ayah of each page, new rob3/thomn values, etc. These should allow the app to build correct list of surah, ayah, and corresponding pages.

I have provided the start ayaat for each page in a previous message. I have now prepared the new values for the Juz section which can be found here is a list of Juz lists with 8 locations (plus Arabic text) for the rub3/thomn etc. It can also be found here which is a text file containing the 8 locations of the Juz the app uses as (Surah Number, Ayah Number) and separates each Juz with a blank between it and the following Juz - it can be used with http://www.mediafire.com/file/g4ba19nzyz5ucs9/quran.ar.uthmani.v2.db/file to get the Arabic text.

Preparing audio. I suggest starting with Sheikh Abdulrasheed Soufi

I have been working on this, and I believe I will be finished tomorrow

AbdullahM0hamed commented 3 years ago

Here is the timing database: http://www.mediafire.com/file/8zrpi0oihbmnad6/abdurrashid_sufi_riwaayah_of_soosi.db/file, for the recitation of Sh Abdurrashid Sufi, which is found here: https://mp3quran.net/eng/soufi

benomaire commented 3 years ago

I'm a bit confused about this. Looking at the translation in-app, it appears that actual (hafs) text is used - does this require us to provide the soosi Arabic text?

Well, all current tafsir/translations were prepared using ayat counting of Hafs. Therefore, they cannot be used directly with other Riwayat (e.g. you will want to see tafsir of ayah 2 and get that of ayah 3). Some ideas of solving this (and other issues) already discussed in several comments on Warsh page.

For soosi arabic text, I think it is good to have it. Thank you.

And also for variations in meaning (such as pronouns changing from 2nd person to 3rd person etc) - do they have to be provided as well?

If you mean to change how Ayat appear (Hafs) in the tafsir, translations. Then, no. I believe they should not be touched. Each Tafsir should keep the original Riwayah used by the author (Mufassir).

Hamza1244 commented 2 years ago

@AbdullahM0hamed ️ السَّلَاْمُ عَلَیْکُم Brother, what program are you using to make the audio database the program for the computer جَزَاكُمُ اللّٰهُ خَيْرََا

mbana commented 2 years ago

Eventually I complete the site I created dedicated to this recitation: https://al-susi.com/. At the moment, I am just hosting the Mushaf via a private GitHub repository. I'm happy to add people to it that want to contribute.

It would nice if someone made a version of the Mushaf with coloured Tajweed.

ghost commented 1 year ago

Assalamualaikum wa rehmatullahi wa barakatuhu. Jazak ALLAHU khairan for making website ❤️. I just downloaded the Mushaf from the website without knowing what you commented here ❤️

AbdullahM0hamed commented 1 year ago

So I've been trying to create a database for the riwaayah of duri lately, and it seems that a lot of ayaat have been skipped - for example in page 40, only the final ayah highlights:

Page 40

But the individual line and ayat scripts work just fine on the pages, example: res

Any idea how this could occur?

AbdullahM0hamed commented 1 year ago

Huh, this seems to be a problem with the soosi database too - I wonder if this was an app problem or an oversight on my part

AbdullahM0hamed commented 1 year ago

I was planning to post all the files at once, once I finished the timing database - but seeing as how I am taking a very long time, I will just post what I have and maybe post an updated audio db if I ever finish it.

• Mushaf images for the riwaayah of doori: https://drive.google.com/file/d/1-eFqH_9VZ05279vg-sXJHH5SCujFVaU2/view?usp=drivesdk

• Ayah info database: https://drive.google.com/file/d/1-dueuSk9UWGyAr-P0GY0dLWF6z32F7JF/view?usp=drivesdk

• Text database (quran.ar.db, not sure how it differs from the other similarly named databases): https://drive.google.com/file/d/1-GWNrmsgNpKJIjinNF5cLkMj8gvVtfR1/view?usp=drivesdk

• Ayah Splitter apk for the riwaayah of Doori (modified app that Nacer sent): https://drive.google.com/file/d/1-BK3xzUbcvEAFGTjMPu3NXNiqZA1vupi/view?usp=drivesdk

• Audio db thus far (up to سورة العنكبوت) for the recitation of Sh Abdurrashid Ali Sufi in the riwaayah of Ad-Doori: https://drive.google.com/file/d/1-PGrZFO9bD0p7uRWP1VLwsxC5mnsZZK3/view?usp=drivesdk

ahmedre commented 1 year ago

jazakumAllah khair - this isn't high on the priority list right now but in sha' Allah when we do, some of this we have good tooling for that can make it easier (ex the audio, some brothers have some automated tools now that would make it easier to cut these).

where did you get the text database?

Hamza1244 commented 1 year ago

@ahmedre They have some automated tools now that would make it easier to cut these brother what is the purpose of this can you tell me how to set the program for me to help you why not do what Ayman Swedish

ahmedre commented 1 year ago

next version will have it in sha' Allah, a brother used these tools to migrate it for us.

Hamza1244 commented 1 year ago

@ahmedre You can't give me those tools brother like the ayah splitter isn't automatic

ahmedre commented 1 year ago

I don't have them either - if the brothers who have them become ready to share with the community they will be opened in sha' Allah.

Hamza1244 commented 1 year ago

@ahmedre It would be nice if he told them to tell me thank you or to tag them