MT0032: Word and Definition Format

[jim-gyas]

Description:

The objective is to create scripts that parse dictionary data from different sources and formats, transforming them into a structured JSON format. Each word and its corresponding definition will be organized in a JSON file with "word" and "definition" fields, enabling easy access and processing. The scripts will handle text-based dictionaries and database extractions, ensuring the data is well-formatted for further usage.

Resources:

• Dictionaries Repo: https://github.com/christiansteinert/tibetan-dictionary/tree/master

Completion Criteria:

• Scripts must successfully parse the dictionary data and output them in JSON format with "word" and "definition" fields.
• The scripts will handle specific cases, including text-based dictionaries and database extractions, ensuring accuracy and consistency in the output.
• All relevant dictionaries found within the resources must be parsed and converted to JSON format for future use.

Subtasks :

• [x] Script to Convert Dictionaries into Word-Definition JSON Format:

  • [x] Write a script where everything after "༑" is considered the definition, and everything before it is considered the word. The output should be in JSON format with "word" and "definition" fields.
  • [x] Write a script where the definition is extracted from within the first curly braces {} after "༑", and everything before "༑" is considered the word. The output should also be in JSON format with "word" and "definition" fields.

• [x] Database Parser Script (Output in JSON format):

  • [x] Write a script where the word is taken from the tb_keyor EnWord column and the definition from the def_dic or TbDef column. The output should be a JSON file with "word" and "definition" fields.
  • [x] Write a script where words are stored in the SpCn and TdCn columns, and definitions are taken from the SpCnDf and TdCnDf columns. The output should be a JSON file with "word" and "definition" fields.

• [x] Locate Additional Dictionaries in the Resources:

  • [x] Search through the resources and identify other relevant dictionaries that can be parsed and converted into JSON format.

[jim-gyas]

From Txt File Dictionaries

[
    {
        "word": "ཀ",
        "definition": "བོད་ཡིག་གསལ་བྱེད་དང་པོ་འདིའི་ང་རོ་འདོན་ཚུལ་ལ་སྐྱེ་གནས་མགྲིན་པ་དང,་བྱེད་པ་མགྲིན་པ,་ནང་གི་རྩོལ་བ་མགྲིན་པ་ནང་ཕྲད,་ཕྱིའི་རྩོལ་བ་སྲོག་ཆུང་སྒྲ་མེད, ༡. ༡༽ མིང་གཞིའི་ཕོ་ཡིག་ཅིག ༢༽ རྟགས་མཚུངས་འདྲེན་སྐབས་ག་ད ༡བ་ས་སྟེ་རྗེས་འཇུག་ཕོས་དྲངས་པའི་མིང་གི་ཆ་ཤས་ཤིག་ག་ད་བ་ས་ཐོག་ཀ,་དཔྱིད་ཀ,་རྒྱལ་ཁ་ཐོབ་ཀ་སྔ་མོ,་སྐས་ཀ, ༣༽ གྲངས་ཀྱི་མིང་མཐར་བཅུག་ནས་ཚང་མའམ་ཐམས་ཅད་ཅེས་པའི་དོན་སྟོན་པའི་ཆ་ཤས་ཤིག་སྔ་ཕྱི་གཉིས་ཀ, ༢.་རྩ་བ ༡དང,་ཐོག་མ,་གདོད་མ,་ཆོས་ཐམས་ཅད་ཀ་ནས་དག་པ,་ཀ་ནས་མ་རྫོགས་པའི་བར,་ཀ་ནས་རང་ཆས་སུ་ཡོད་པ་མ་ཡིན་པ,"
    },
    {
        "word": "ཀ་ཀ",
        "definition": "༡༽ སྐྱ་གའི་མིང་དང་དེའི་སྐད། ༢༽ ལུག་གི་ཨ་ཅུག་གམ་ཐེ་གེ། ༣༽ ༼ཡུལ༽ བྱིས་ཆུང་གི་གྱོན་པ།"
    }
    ]

From DB Dictionaries:

[
    {
        "word": "1000",
        "definition": [
            "ཕྲག་སྟོང་།  1000"
        ]
    },
    {
        "word": "01-Mar",
        "definition": [
            "n. ཕྱི་ཟླ་གསུམ་པ། 10 th March གསུམ་བཅུའི་དུས་དྲན།"
        ]
    }
    ]

[jim-gyas]

1) Common Chinese-Tibetan-English Buddhist Terminology (in csv format)

2) BabelStone Tibetan Shorthand Contractions (in html format)

3) 25-tshig-mdzod-chen-mo-Tib

4) 49-LokeshChandraTib

@kaldan007, @TenzinGayche, I found a few dictionaries in the resources repository. Could you please help me identify which ones are valid and which ones might not be?

[TenzinGayche]

@jim-gyas can you extract them all except BabelStone Tibetan Shorthand Contractions (in html format)? note: Some of them are in Wiley so you might have to convert them into unicode , i need them before 5 pm today