Metadata length validation: count graphemes

openzim / python-scraperlib

Collection of Python code to re-use across Python-based scrapers

GNU General Public License v3.0

20 stars 18 forks source link

Metadata length validation: count graphemes #176

Closed benoit74 closed 4 months ago

benoit74 commented 4 months ago

Fix #158 Fix #177

Changes

Use regex module to count graphemes, it looks like the most stable and maintained package for our need
Add utility function grapheme_count so that it can be easily reused + fixed in the future if needed
Fix tests to exhibit the problem of graphemes vs codepoints
Fixed type hints for images function calling PIL/pillow, which has updated its type hints in 10.4.0 and is now raising a pyright issue

benoit74 commented 4 months ago

Edit: also had to add commit using Jeena instance just like in https://github.com/openzim/python-scraperlib/pull/175 to let the CI pass

codecov[bot] commented 4 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (7ab3fcd) to head (a2c1892).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #176 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 32 32 Lines 1448 1452 +4 Branches 251 251 ========================================= + Hits 1448 1452 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

benoit74 commented 4 months ago

Requested changes applied, beware I did a fixup! commit which will need to be squashed before merging.