Added UDHR dataset, a collection of (~200) translations of the Universal Declaration of Human Rights
Motivation and Context
I wanted a relatively lightweight and comprehensive set of source documents for getting language-specific distributions of characters, for use in certain data augmentation transform functions. The current solution relies on ConceptNet, which is 1. very large and 2. less comprehensive, thus 3. unsatisfactory.
How Has This Been Tested?
Wrote the usual tests, and they all pass. Fixed some other datasets' buggy tests in the process!
Types of changes
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation, and I have updated it accordingly.
Description
Motivation and Context
I wanted a relatively lightweight and comprehensive set of source documents for getting language-specific distributions of characters, for use in certain data augmentation transform functions. The current solution relies on ConceptNet, which is 1. very large and 2. less comprehensive, thus 3. unsatisfactory.
How Has This Been Tested?
Wrote the usual tests, and they all pass. Fixed some other datasets' buggy tests in the process!
Types of changes
Checklist: