search

clips / pattern

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
https://github.com/clips/pattern/wiki
BSD 3-Clause "New" or "Revised" License
8.72k stars 1.58k forks source link

StopIteration & RuntimeError fix for Python >= 3.7 #294

Open NicolasBizzozzero opened 4 years ago

NicolasBizzozzero commented 4 years ago

Since Python 3.7, all StopIteration exceptions raised inside a generator are transformed into RuntimeError (see PEP-0479 and this answer from StackOverflow). This new behavior made pattern unusable for all Python3.7+ users and all packages depending on it (gensim for instance: https://github.com/RaRe-Technologies/gensim/issues/2438).

This PR fixes the _read generator by removing the StopIteration exception raised in it. It solves the following issues :

coveralls commented 4 years ago

Coverage Status

Coverage decreased (-0.3%) to 68.022% when pulling 66ab34453a3443c06a4ebda092c8d1947c83b17a on NicolasBizzozzero:master into 5b85d998c30ddc6772b56310713530224466083a on clips:master.

zc4242 commented 4 years ago

Thank you for all the work @NicolasBizzozzero !

Seems like the CI build is based on python=3.6 and causing the check failures? Who can help to fix this issue as it seems stupid to not able to fix this for quite a while...

tuky commented 4 years ago

Meanwhile you can monkey patch around this issue by calling this method in your module:

def patch_pattern():
    from pattern import text

    original_read = text._read

    @functools.wraps(original_read)
    def patched_read(*args, **kwargs):
        try:
            for r in original_read(*args, **kwargs):
                yield r
        except RuntimeError:
            pass

    text._read = patched_read