tatuylonen / wiktextract

Wiktionary dump file parser and multilingual data extractor
Other
741 stars 82 forks source link

pre-extracted data in .tsv format #140

Open Digital-XxX opened 1 year ago

Digital-XxX commented 1 year ago

Please give me pre-extracted data in .tsv format please. Goldendict mobile cannot read .json dictionaries.

kristian-clausal commented 1 year ago

Unfortunately the data we provide is not suitable to be used straightforwardly in .tsv or .csv. The JSON data is hierarchical, with big and reasonably sprawling word structures that contain smaller structures, dictionaries and lists, and translating that to .tsv needs to be done on a case-by-case basis. It's not a universal data format that is swappable between different programs (at least yet, or in the near future), it's just a bunch of data we've put into an adhoc data structure as need be.

To make what you want possible you need to:

We welcome any contributions to the project to make it more accessible.

Digital-XxX commented 1 year ago
  • program a script that will do that translation by reading the json file object-by-object and then outputting it into .tsv

I think pyglossary supports conversion of .json to .tsv/.tab

kristian-clausal commented 1 year ago
  • program a script that will do that translation by reading the json file object-by-object and then outputting it into .tsv

I think pyglossary supports conversion of .json to .tsv/.tab

We would be happy to have someone implement a conversion utility for our .json to other formats, but someone has to code it first, and our data structure and format can change as time goes by.

Vuizur commented 1 year ago

I created a project that is able to create tsv/stardict/kindle dictionaries from the kaikki dump. It is only not extremely well tested, but possibly it works: https://github.com/Vuizur/ebook_dictionary_creator

Vuizur commented 1 year ago

I also now have a repository with directly downloadable dictionaries for a lot of languages in 3 different formats: https://github.com/Vuizur/Wiktionary-Dictionaries

GrimPixel commented 2 months ago

Here is a new tool: https://codeberg.org/GrimPixel/Text_to_Wordlist You can place your text file in the corresponding directory 0_text, then check the text_setting.yaml and dictionary_setting.yaml, then run extract_text.py and extract_dictionary.py to generate a TSV file with values separated as described in README.adoc.