search

explosion / thinc

🔮 A refreshing functional take on deep learning, compatible with your favorite libraries
https://thinc.ai
MIT License
2.82k stars 275 forks source link

Error during pytest #543

Closed lpkoh closed 3 years ago

lpkoh commented 3 years ago

Hi, I am using a virtulenv with Python 3.6 on aarch64 infrastructure.

I have run the instructions as described to build from source. No issues faced. I was on v0.8.0 branch.

When I run: python3 -m pytest

I get: _________________________________________________________________________ ERROR collecting thinc/tests/regression/test_issue208.py _________________________________________________________________________ ImportError while importing test module '/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/tests/regression/test_issue208.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib/python3.6/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) thinc/tests/regression/test_issue208.py:1: in <module> from thinc.api import chain, Linear thinc/api.py:2: in <module> from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init thinc/initializers.py:4: in <module> from .backends import Ops thinc/backends/__init__.py:8: in <module> from .cupy_ops import CupyOps, has_cupy thinc/backends/cupy_ops.py:18: in <module> from .numpy_ops import NumpyOps E ModuleNotFoundError: No module named 'thinc.backends.numpy_ops' ========================================================================================= short test summary info ========================================================================================== ERROR thinc/tests/test_config.py ERROR thinc/tests/test_initializers.py ERROR thinc/tests/test_loss.py ERROR thinc/tests/test_optimizers.py ERROR thinc/tests/test_schedules.py ERROR thinc/tests/test_serialize.py ERROR thinc/tests/test_util.py ERROR thinc/tests/backends/test_mem.py ERROR thinc/tests/backends/test_ops.py ERROR thinc/tests/extra/test_beam_search.py ERROR thinc/tests/layers/test_basic_tagger.py ERROR thinc/tests/layers/test_combinators.py ERROR thinc/tests/layers/test_feed_forward.py ERROR thinc/tests/layers/test_hash_embed.py ERROR thinc/tests/layers/test_layers_api.py ERROR thinc/tests/layers/test_linear.py ERROR thinc/tests/layers/test_lstm.py ERROR thinc/tests/layers/test_mnist.py ERROR thinc/tests/layers/test_mxnet_wrapper.py ERROR thinc/tests/layers/test_pytorch_wrapper.py ERROR thinc/tests/layers/test_reduce_first_last.py ERROR thinc/tests/layers/test_shim.py ERROR thinc/tests/layers/test_sparse_linear.py ERROR thinc/tests/layers/test_tensorflow_wrapper.py ERROR thinc/tests/layers/test_transforms.py ERROR thinc/tests/layers/test_uniqued.py ERROR thinc/tests/layers/test_with_debug.py ERROR thinc/tests/layers/test_with_transforms.py ERROR thinc/tests/model/test_model.py ERROR thinc/tests/model/test_validation.py ERROR thinc/tests/regression/test_issue208.py !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 31 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ============================================================================================ 31 errors in 2.74s ============================================================================================

Full message is longer but its just a repeat of "ERROR collecting" message

polm commented 3 years ago

Sounds like it can't find Thinc. If you run pip freeze can you confirm Thinc is installed in your venv?

lpkoh commented 3 years ago

Hey, thanks for getting back.

So the issue is this. I installed using

pip install --no-build-isolation --editable .

It apparently says it is successful:

  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
    Preparing wheel metadata ... done
Requirement already satisfied: catalogue<2.1.0,>=2.0.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.0.6)
Requirement already satisfied: srsly<3.0.0,>=2.4.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.4.1)
Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4.1 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (3.10.0.2)
Requirement already satisfied: setuptools in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (58.1.0)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.8.1)
Requirement already satisfied: numpy>=1.15.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.19.4)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.7.4)
Requirement already satisfied: contextvars<3,>=2.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.4)
Requirement already satisfied: dataclasses<1.0,>=0.6 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.6)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (3.0.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.0.2)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.0.5)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.7.4)
Requirement already satisfied: zipp>=0.5 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from catalogue<2.1.0,>=2.0.4->thinc==8.0.10) (3.6.0)
Requirement already satisfied: immutables>=0.9 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from contextvars<3,>=2.4->thinc==8.0.10) (0.16)
Building wheels for collected packages: thinc
  Building wheel for thinc (PEP 517) ... done
  Created wheel for thinc: filename=thinc-8.0.10-cp36-cp36m-linux_aarch64.whl size=1629814 sha256=10116ac2b8456575fcbc6d138e66e2242198353a0169078db2109c3ae3f400db
  Stored in directory: /tmp/pip-ephem-wheel-cache-w73yhkd5/wheels/3e/f6/97/1d273c8f4d7d418ac3d1703df387a6ce9dc1b5b4170f8911fc
Successfully built thinc
Installing collected packages: thinc
Successfully installed thinc-8.0.10
WARNING: You are using pip version 21.2.4; however, version 21.3 is available.
You should consider upgrading via the '/home/vap-team-admin/Desktop/balefire_jetson/balefire_env/bin/python -m pip install --upgrade pip' command.

When I try to import in python:

Python 3.6.9 (default, Jan 26 2021, 15:33:00) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import thinc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/__init__.py", line 10, in <module>
    require_cpu()
  File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/util.py", line 162, in require_cpu
    from .backends import set_current_ops, NumpyOps
  File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/__init__.py", line 8, in <module>
    from .cupy_ops import CupyOps, has_cupy
  File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/cupy_ops.py", line 19, in <module>
    from .numpy_ops import NumpyOps
ModuleNotFoundError: No module named 'thinc.backends.numpy_ops'

But thinc does show up with the right version "8.0.10" in pip3 list. In pip3 freeze, it shows this. This is all in a virtualenv btw thinc @ file:///home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc

I also attempted an install from source using 'export PYTHONPATH='pwd'' and 'python setup.py build_ext --inplace' What happens in this case is that pip3 does not list thinc at all. However, the installation is more successful (although I don't thnk it is complte). It fails at 99% for pytest.

Can someone confirm if the install actually works and all the pytests can work? Are the dependencies correct?

lpkoh commented 3 years ago

Specifically, in the 'export PYTHONPATH='pwd'' and 'python setup.py build_ext --inplace' route, running 'python -m pytest thinc' leads to the pytest stalling at

=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/.hypothesis/examples')
rootdir: /home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc, configfile: setup.cfg
plugins: hypothesis-4.0.0, cov-2.7.0
collected 1092 items                                                                                                                                                                                       

thinc/tests/test_config.py ...............................................................................................x...x...x...x                                                              [  9%]
thinc/tests/test_examples.py EEssssssss                                                                                                                                                              [ 10%]
thinc/tests/test_indexing.py .....                                                                                                                                                                   [ 11%]
thinc/tests/test_initializers.py ........                                                                                                                                                            [ 11%]
thinc/tests/test_loss.py ............................                                                                                                                                                [ 14%]
thinc/tests/test_optimizers.py ....................                                                                                                                                                  [ 16%]
thinc/tests/test_schedules.py .......                                                                                                                                                                [ 17%]
thinc/tests/test_serialize.py ...........                                                                                                                                                            [ 18%]
thinc/tests/test_types.py ..........                                                                                                                                                                 [ 18%]
thinc/tests/test_util.py .............                                                                                                                                                               [ 20%]
thinc/tests/backends/test_mem.py .                                                                                                                                                                   [ 20%]
thinc/tests/backends/test_ops.py .....................................................                                                                                                               [ 25%]
thinc/tests/extra/test_beam_search.py .                                                                                                                                                              [ 25%]
thinc/tests/layers/test_basic_tagger.py s                                                                                                                                                            [ 25%]
thinc/tests/layers/test_combinators.py ............................................................................................................................................................. [ 39%]
.................................................................................................................................................................................................... [ 57%]
.                                                                                                                                                                                                    [ 57%]
thinc/tests/layers/test_feed_forward.py ..........................................................................................................................s                                  [ 68%]
thinc/tests/layers/test_hash_embed.py ..                                                                                                                                                             [ 69%]
thinc/tests/layers/test_layers_api.py ...................ss................................s.......................ss.............                                                                   [ 77%]
thinc/tests/layers/test_linear.py ...............                                                                                                                                                    [ 78%]
thinc/tests/layers/test_lstm.py ..........................................s.s                                                                                                                        [ 83%]
thinc/tests/layers/test_mnist.py sss                                                                                                                                                                 [ 83%]
thinc/tests/layers/test_mxnet_wrapper.py ssssssssssssssssss                                                                                                                                          [ 84%]
thinc/tests/layers/test_pytorch_wrapper.py sssssssss                                                                                                                                                 [ 85%]
thinc/tests/layers/test_reduce_first_last.py ....                                                                                                                                                    [ 86%]
thinc/tests/layers/test_shim.py ..                                                                                                                                                                   [ 86%]
thinc/tests/layers/test_sparse_linear.py ..                                                                                                                                                          [ 86%]
thinc/tests/layers/test_tensorflow_wrapper.py sssssssssssssssssssssssss                                                                                                                              [ 88%]
thinc/tests/layers/test_transforms.py .....................                                                                                                                                          [ 90%]
thinc/tests/layers/test_uniqued.py ..                                                                                                                                                                [ 90%]
thinc/tests/layers/test_with_debug.py .                                                                                                                                                              [ 91%]
thinc/tests/layers/test_with_transforms.py .....................................................                                                                                                     [ 95%]
thinc/tests/model/test_model.py ....................................                                                                                                                                 [ 99%]
thinc/tests/model/test_validation.py ..                                                                                                                                                              [ 99%]
thinc/tests/mypy/test_mypy.py

If I terminate process with ctrl-c:

================================================================================================== ERRORS ==================================================================================================
________________________________________________________ ERROR at setup of test_ipython_notebooks[examples/01_intro_model_definition_methods.ipynb] ________________________________________________________

nb_file = 'examples/01_intro_model_definition_methods.ipynb'

    @pytest.fixture
    def test_files(nb_file):
        pytest.importorskip("nbconvert")
        pytest.importorskip("nbformat")
        import nbconvert
        import nbformat
        from nbconvert.preprocessors import ExecutePreprocessor

        if not Path(nb_file).exists():
            return
        kernel_name = os.environ.get("NOTEBOOK_KERNEL", "python3")
        with open(nb_file) as f:
            nb = nbformat.read(f, as_version=4)
        proc = ExecutePreprocessor(timeout=600, kernel_name=kernel_name)
        proc.allow_errors = True
>       proc.preprocess(nb, {"metadata": {"path": "/"}})

thinc/tests/test_examples.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:405: in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/base.py:69: in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:438: in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:578: in run_cell
    exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x7f978e4a20>, msg_id = '2add58b8-8e7f83f44a6406c38e24d7b1_17297_2'
cell = {'cell_type': 'code', 'execution_count': None, 'metadata': {}, 'outputs': [], 'source': '!pip install "thinc>=8.0.0a0"'}, timeout = 1

    def _poll_for_reply(self, msg_id, cell=None, timeout=None):
        try:
            # check with timeout if kernel is still alive
            msg = self.kc.shell_channel.get_msg(timeout=timeout)
>           if msg['parent_header'].get('msg_id') == msg_id:
E           TypeError: 'coroutine' object is not subscriptable

../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:479: TypeError
_____________________________________________________________ ERROR at setup of test_ipython_notebooks[examples/05_benchmarking_layers.ipynb] ______________________________________________________________

nb_file = 'examples/05_benchmarking_layers.ipynb'

    @pytest.fixture
    def test_files(nb_file):
        pytest.importorskip("nbconvert")
        pytest.importorskip("nbformat")
        import nbconvert
        import nbformat
        from nbconvert.preprocessors import ExecutePreprocessor

        if not Path(nb_file).exists():
            return
        kernel_name = os.environ.get("NOTEBOOK_KERNEL", "python3")
        with open(nb_file) as f:
            nb = nbformat.read(f, as_version=4)
        proc = ExecutePreprocessor(timeout=600, kernel_name=kernel_name)
        proc.allow_errors = True
>       proc.preprocess(nb, {"metadata": {"path": "/"}})

thinc/tests/test_examples.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:405: in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/base.py:69: in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:438: in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:578: in run_cell
    exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x7f8d69dbe0>, msg_id = 'b87af2de-028b71bbb45b9d9e191854f1_17297_2'
cell = {'cell_type': 'code', 'execution_count': None, 'metadata': {}, 'outputs': [], 'source': '!pip install "thinc>=8.0.0a0"'}, timeout = 1

    def _poll_for_reply(self, msg_id, cell=None, timeout=None):
        try:
            # check with timeout if kernel is still alive
            msg = self.kc.shell_channel.get_msg(timeout=timeout)
>           if msg['parent_header'].get('msg_id') == msg_id:
E           TypeError: 'coroutine' object is not subscriptable

../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:479: TypeError
============================================================================================= warnings summary =============================================================================================
thinc/tests/backends/test_ops.py::test_lstm_forward_training_fuzz[ops0]
  /home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/ops.py:1146: RuntimeWarning: overflow encountered in exp
    return 1.0 / (1.0 + xp.exp(-X))

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================= short test summary info ==========================================================================================
ERROR thinc/tests/test_examples.py::test_ipython_notebooks[examples/01_intro_model_definition_methods.ipynb] - TypeError: 'coroutine' object is not subscriptable
ERROR thinc/tests/test_examples.py::test_ipython_notebooks[examples/05_benchmarking_layers.ipynb] - TypeError: 'coroutine' object is not subscriptable
=============================================================== 1007 passed, 72 skipped, 4 xfailed, 1 warning, 2 errors in 146.83s (0:02:26) ===============================================================
polm commented 3 years ago

When posting logs please use the "Preview" function to make sure they look OK. You should start and end them with a "fence", or a line, of three backticks, not a single backtick at the start and end.

For the record I am able to run the tests locally without your issues - I get three failures but they're mypy related, so probably due to in-progress changes. Give me some time to look at your logs.

Also, like the README says, insalling with --editable is the normal, correct way to build from source.

polm commented 3 years ago

OK, couple of things. Looking at your output it looks like the only errors you have are related to ipython notebooks. I suspect this isn't a real problem but just something with your environment or version of ipython or something.

Note that this repo uses continuous integration, so every commit has tests run automatically. You can see this by clicking on the checkmark at the top of the repo, next to the most recent commit message. That would be an X if the tests were failing. (That makes me curious why the mypy tests fail for me locally, but maybe something's wrong with my mypy configuration.)

Is there a specific reason you were concerned about test failures, or were you just asking because the README says tests should pass?

Also, one thing is that you're using Python 3.6. I think we currently support it, but the end-of-life for that version is in December. You should move to a more recent version as soon as possible.

lpkoh commented 3 years ago

Thank you so much!

Yes my only concern was that the tests were supposed to pass but it did not. I am currently on a Jetson AGX Xavier (aarch64) device, I don't know if that would be related to any of the issues faced.

If I can follow up with last 2 questions:

  1. I am installing from source for the purpose of using spacy, which requires thinc as a dependency. Does having thinc show up as thinc @ file:///home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc in pip3 freeze allow it to be detected? No version is specified
  2. Will installation with --editable change things? omitting --editable seemed to have caused the installation to be faulty.
polm commented 3 years ago

I am installing from source for the purpose of using spacy, which requires thinc as a dependency. Does having thinc show up as thinc @ file:///home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc in pip3 freeze allow it to be detected? No version is specified

That's normal for editable installs, it's not a problem.

Will installation with --editable change things? omitting --editable seemed to have caused the installation to be faulty.

Editable installs are useful when you're modifying the code regularly, and one side effect is that they help pytest work by getting paths right. Installing Thinc as a dependency from source you can do something like pip path/to/thinc and shouldn't need the --editable flag, though it would also work.

The main reason for the editable flag is that if an item is installed editable, the original source files are used rather than a copy. This means that as you modify your source the updates are automatically reflected in your virtualenv. Since you don't seem to actually be modifying Thinc, just getting it set up, you don't need that flag to use it. (But again, it helps with tests due to path issues or something.)

github-actions[bot] commented 3 years ago

This issue has been automatically closed because it was answered and there was no follow-up discussion.