Create dataset ksucca_king_saud_university_corpus_of_classical_arabic

[albertvillanova]

• uid: ksucca_king_saud_university_corpus_of_classical_arabic
• type: processed
• description:
  • name: KSUCCA King Saud University Corpus of Classical Arabic
  • description: King Saud University Corpus of Classical Arabic (KSUCCA) is a pioneering 50 million tokens annotated corpus of Classical Arabic texts from the period of pre-Islamic era until the fourth Hijri century (equivalent to the period from the seventh until early eleventh century CE), which is the period of pure classical Arabic. The main aim of this corpus is to be used for studying the distributional lexical semantics of The Quran words. However, it can be used for other research purposes, such as: • Arabic linguistics, which includes: lexical, morphological, syntactic, semantic and pragmatic research. • Arabic computational linguistics, which includes: lexical, morphological, syntactic, semantic and pragmatic research including their various applications. • Arabic language teaching for both Arabs and non Arabs. • Artificial intelligence. • Natural language processing. • Information retrieval. • Question answering. • Machine translation.
  • homepage: https://sourceforge.net/projects/ksucca-corpus/
  • validated: True
• languages:
  • language_names:
  • Arabic
  • ar-CLS
  • language_comments: Historical Arabic texts from the seventh until early eleventh century CE
  • language_locations:
  • Middle East and North Africa
  • validated: True
• custodian:
  • name: King Saud University
  • type: A university or research institution
  • location: Saudi Arabia
  • contact_name: Maha Alrabiah
  • contact_email: msrabiah@gmail.com
  • contact_submitter: True
  • additional: https://www.ksu.edu.sa/en/
  • validated: True
• availability:
  • procurement:
  • for_download: Yes - it has a direct download link or links
  • download_url: https://sourceforge.net/projects/ksucca-corpus/
  • download_email:
  • licensing:
  • has_licenses: Yes
  • license_text:
  • license_properties:
    • open license
    • non-commercial use
  • license_list:
    • cc-by-nc-2.0: Creative Commons Attribution Non Commercial 2.0 Generic
  • pii:
  • has_pii: No
  • generic_pii_likely:
  • generic_pii_list:
  • numeric_pii_likely:
  • numeric_pii_list:
  • sensitive_pii_likely:
  • sensitive_pii_list:
  • no_pii_justification_class: other
  • no_pii_justification_text: historical texts that are more than 100 years old
  • validated: True
• processed_from_primary:
  • from_primary: Taken from primary source
  • primary_availability: Yes - their documentation/homepage/description is available
  • from_primary_entries:
  • primary_types:
  • books/book publisher
  • primary_license: Yes - the source material has an open license that allows re-use
  • validated: True
• media:
  • category:
  • text
  • text_format:
  • .TXT
  • text_is_transcribed: No
  • database_format:
  • instance_type: book
  • instance_count: 100<n<1K
  • instance_size: n>10,000
  • validated: True
• update_time: 10_10_2021__18_34_26
• fname: ksucca_king_saud_university_corpus_of_classical_arabic-validated-10_10_2021__18_34_26.json

[cakiki]

self-assign

[cakiki]

@albertvillanova done: https://huggingface.co/datasets/bigscience-catalogue-data/ksucca_king_saud_university_corpus_of_classical_arabic

.docx file explains the directory structure

[albertvillanova]

Please note that for LM we are only interested in the non-annotated data.

[albertvillanova]

DONE: https://huggingface.co/datasets/bigscience-catalogue-lm-data/lm_ar_ksucca

Sample:

{
  'text': '\ufeffبسم الله الرحمن الرحيم\nقال أبو عبيدة معمر بن المثنى التيمى، تيم قريش، مولى لهم.\nكان العرب العكاظيون لا يعدون من الشيء إلا ثلاثة ثم يكفون ولا يزيدون عليها شيئا، وإن لحق بعد شيء مثل الثلاثة التي عدوا قبل ذلك لم يعدوه معه...',
  'meta': "{'filename': 'da1.txt'}"
}