Closed lpkoh closed 3 years ago
Sounds like it can't find Thinc. If you run pip freeze
can you confirm Thinc is installed in your venv?
Hey, thanks for getting back.
So the issue is this. I installed using
pip install --no-build-isolation --editable .
It apparently says it is successful:
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Preparing wheel metadata ... done
Requirement already satisfied: catalogue<2.1.0,>=2.0.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.0.6)
Requirement already satisfied: srsly<3.0.0,>=2.4.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.4.1)
Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4.1 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (3.10.0.2)
Requirement already satisfied: setuptools in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (58.1.0)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.8.1)
Requirement already satisfied: numpy>=1.15.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.19.4)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.7.4)
Requirement already satisfied: contextvars<3,>=2.4 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.4)
Requirement already satisfied: dataclasses<1.0,>=0.6 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.6)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (3.0.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (2.0.2)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (1.0.5)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from thinc==8.0.10) (0.7.4)
Requirement already satisfied: zipp>=0.5 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from catalogue<2.1.0,>=2.0.4->thinc==8.0.10) (3.6.0)
Requirement already satisfied: immutables>=0.9 in /home/vap-team-admin/Desktop/balefire_jetson/balefire_env/lib/python3.6/site-packages (from contextvars<3,>=2.4->thinc==8.0.10) (0.16)
Building wheels for collected packages: thinc
Building wheel for thinc (PEP 517) ... done
Created wheel for thinc: filename=thinc-8.0.10-cp36-cp36m-linux_aarch64.whl size=1629814 sha256=10116ac2b8456575fcbc6d138e66e2242198353a0169078db2109c3ae3f400db
Stored in directory: /tmp/pip-ephem-wheel-cache-w73yhkd5/wheels/3e/f6/97/1d273c8f4d7d418ac3d1703df387a6ce9dc1b5b4170f8911fc
Successfully built thinc
Installing collected packages: thinc
Successfully installed thinc-8.0.10
WARNING: You are using pip version 21.2.4; however, version 21.3 is available.
You should consider upgrading via the '/home/vap-team-admin/Desktop/balefire_jetson/balefire_env/bin/python -m pip install --upgrade pip' command.
When I try to import in python:
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import thinc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/__init__.py", line 10, in <module>
require_cpu()
File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/util.py", line 162, in require_cpu
from .backends import set_current_ops, NumpyOps
File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/__init__.py", line 8, in <module>
from .cupy_ops import CupyOps, has_cupy
File "/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/cupy_ops.py", line 19, in <module>
from .numpy_ops import NumpyOps
ModuleNotFoundError: No module named 'thinc.backends.numpy_ops'
But thinc does show up with the right version "8.0.10" in pip3 list. In pip3 freeze, it shows this. This is all in a virtualenv btw thinc @ file:///home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc
I also attempted an install from source using 'export PYTHONPATH='pwd'' and 'python setup.py build_ext --inplace' What happens in this case is that pip3 does not list thinc at all. However, the installation is more successful (although I don't thnk it is complte). It fails at 99% for pytest.
Can someone confirm if the install actually works and all the pytests can work? Are the dependencies correct?
Specifically, in the 'export PYTHONPATH='pwd'' and 'python setup.py build_ext --inplace' route, running 'python -m pytest thinc' leads to the pytest stalling at
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/.hypothesis/examples')
rootdir: /home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc, configfile: setup.cfg
plugins: hypothesis-4.0.0, cov-2.7.0
collected 1092 items
thinc/tests/test_config.py ...............................................................................................x...x...x...x [ 9%]
thinc/tests/test_examples.py EEssssssss [ 10%]
thinc/tests/test_indexing.py ..... [ 11%]
thinc/tests/test_initializers.py ........ [ 11%]
thinc/tests/test_loss.py ............................ [ 14%]
thinc/tests/test_optimizers.py .................... [ 16%]
thinc/tests/test_schedules.py ....... [ 17%]
thinc/tests/test_serialize.py ........... [ 18%]
thinc/tests/test_types.py .......... [ 18%]
thinc/tests/test_util.py ............. [ 20%]
thinc/tests/backends/test_mem.py . [ 20%]
thinc/tests/backends/test_ops.py ..................................................... [ 25%]
thinc/tests/extra/test_beam_search.py . [ 25%]
thinc/tests/layers/test_basic_tagger.py s [ 25%]
thinc/tests/layers/test_combinators.py ............................................................................................................................................................. [ 39%]
.................................................................................................................................................................................................... [ 57%]
. [ 57%]
thinc/tests/layers/test_feed_forward.py ..........................................................................................................................s [ 68%]
thinc/tests/layers/test_hash_embed.py .. [ 69%]
thinc/tests/layers/test_layers_api.py ...................ss................................s.......................ss............. [ 77%]
thinc/tests/layers/test_linear.py ............... [ 78%]
thinc/tests/layers/test_lstm.py ..........................................s.s [ 83%]
thinc/tests/layers/test_mnist.py sss [ 83%]
thinc/tests/layers/test_mxnet_wrapper.py ssssssssssssssssss [ 84%]
thinc/tests/layers/test_pytorch_wrapper.py sssssssss [ 85%]
thinc/tests/layers/test_reduce_first_last.py .... [ 86%]
thinc/tests/layers/test_shim.py .. [ 86%]
thinc/tests/layers/test_sparse_linear.py .. [ 86%]
thinc/tests/layers/test_tensorflow_wrapper.py sssssssssssssssssssssssss [ 88%]
thinc/tests/layers/test_transforms.py ..................... [ 90%]
thinc/tests/layers/test_uniqued.py .. [ 90%]
thinc/tests/layers/test_with_debug.py . [ 91%]
thinc/tests/layers/test_with_transforms.py ..................................................... [ 95%]
thinc/tests/model/test_model.py .................................... [ 99%]
thinc/tests/model/test_validation.py .. [ 99%]
thinc/tests/mypy/test_mypy.py
If I terminate process with ctrl-c:
================================================================================================== ERRORS ==================================================================================================
________________________________________________________ ERROR at setup of test_ipython_notebooks[examples/01_intro_model_definition_methods.ipynb] ________________________________________________________
nb_file = 'examples/01_intro_model_definition_methods.ipynb'
@pytest.fixture
def test_files(nb_file):
pytest.importorskip("nbconvert")
pytest.importorskip("nbformat")
import nbconvert
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
if not Path(nb_file).exists():
return
kernel_name = os.environ.get("NOTEBOOK_KERNEL", "python3")
with open(nb_file) as f:
nb = nbformat.read(f, as_version=4)
proc = ExecutePreprocessor(timeout=600, kernel_name=kernel_name)
proc.allow_errors = True
> proc.preprocess(nb, {"metadata": {"path": "/"}})
thinc/tests/test_examples.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:405: in preprocess
nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/base.py:69: in preprocess
nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:438: in preprocess_cell
reply, outputs = self.run_cell(cell, cell_index, store_history)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:578: in run_cell
exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x7f978e4a20>, msg_id = '2add58b8-8e7f83f44a6406c38e24d7b1_17297_2'
cell = {'cell_type': 'code', 'execution_count': None, 'metadata': {}, 'outputs': [], 'source': '!pip install "thinc>=8.0.0a0"'}, timeout = 1
def _poll_for_reply(self, msg_id, cell=None, timeout=None):
try:
# check with timeout if kernel is still alive
msg = self.kc.shell_channel.get_msg(timeout=timeout)
> if msg['parent_header'].get('msg_id') == msg_id:
E TypeError: 'coroutine' object is not subscriptable
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:479: TypeError
_____________________________________________________________ ERROR at setup of test_ipython_notebooks[examples/05_benchmarking_layers.ipynb] ______________________________________________________________
nb_file = 'examples/05_benchmarking_layers.ipynb'
@pytest.fixture
def test_files(nb_file):
pytest.importorskip("nbconvert")
pytest.importorskip("nbformat")
import nbconvert
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
if not Path(nb_file).exists():
return
kernel_name = os.environ.get("NOTEBOOK_KERNEL", "python3")
with open(nb_file) as f:
nb = nbformat.read(f, as_version=4)
proc = ExecutePreprocessor(timeout=600, kernel_name=kernel_name)
proc.allow_errors = True
> proc.preprocess(nb, {"metadata": {"path": "/"}})
thinc/tests/test_examples.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:405: in preprocess
nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/base.py:69: in preprocess
nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:438: in preprocess_cell
reply, outputs = self.run_cell(cell, cell_index, store_history)
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:578: in run_cell
exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x7f8d69dbe0>, msg_id = 'b87af2de-028b71bbb45b9d9e191854f1_17297_2'
cell = {'cell_type': 'code', 'execution_count': None, 'metadata': {}, 'outputs': [], 'source': '!pip install "thinc>=8.0.0a0"'}, timeout = 1
def _poll_for_reply(self, msg_id, cell=None, timeout=None):
try:
# check with timeout if kernel is still alive
msg = self.kc.shell_channel.get_msg(timeout=timeout)
> if msg['parent_header'].get('msg_id') == msg_id:
E TypeError: 'coroutine' object is not subscriptable
../../balefire_env/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py:479: TypeError
============================================================================================= warnings summary =============================================================================================
thinc/tests/backends/test_ops.py::test_lstm_forward_training_fuzz[ops0]
/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/backends/ops.py:1146: RuntimeWarning: overflow encountered in exp
return 1.0 / (1.0 + xp.exp(-X))
-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================= short test summary info ==========================================================================================
ERROR thinc/tests/test_examples.py::test_ipython_notebooks[examples/01_intro_model_definition_methods.ipynb] - TypeError: 'coroutine' object is not subscriptable
ERROR thinc/tests/test_examples.py::test_ipython_notebooks[examples/05_benchmarking_layers.ipynb] - TypeError: 'coroutine' object is not subscriptable
=============================================================== 1007 passed, 72 skipped, 4 xfailed, 1 warning, 2 errors in 146.83s (0:02:26) ===============================================================
When posting logs please use the "Preview" function to make sure they look OK. You should start and end them with a "fence", or a line, of three backticks, not a single backtick at the start and end.
For the record I am able to run the tests locally without your issues - I get three failures but they're mypy related, so probably due to in-progress changes. Give me some time to look at your logs.
Also, like the README says, insalling with --editable
is the normal, correct way to build from source.
OK, couple of things. Looking at your output it looks like the only errors you have are related to ipython notebooks. I suspect this isn't a real problem but just something with your environment or version of ipython or something.
Note that this repo uses continuous integration, so every commit has tests run automatically. You can see this by clicking on the checkmark at the top of the repo, next to the most recent commit message. That would be an X if the tests were failing. (That makes me curious why the mypy tests fail for me locally, but maybe something's wrong with my mypy configuration.)
Is there a specific reason you were concerned about test failures, or were you just asking because the README says tests should pass?
Also, one thing is that you're using Python 3.6. I think we currently support it, but the end-of-life for that version is in December. You should move to a more recent version as soon as possible.
Thank you so much!
Yes my only concern was that the tests were supposed to pass but it did not. I am currently on a Jetson AGX Xavier (aarch64) device, I don't know if that would be related to any of the issues faced.
If I can follow up with last 2 questions:
I am installing from source for the purpose of using spacy, which requires thinc as a dependency. Does having thinc show up as thinc @ file:///home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc in pip3 freeze allow it to be detected? No version is specified
That's normal for editable installs, it's not a problem.
Will installation with --editable change things? omitting --editable seemed to have caused the installation to be faulty.
Editable installs are useful when you're modifying the code regularly, and one side effect is that they help pytest work by getting paths right. Installing Thinc as a dependency from source you can do something like pip path/to/thinc
and shouldn't need the --editable
flag, though it would also work.
The main reason for the editable flag is that if an item is installed editable, the original source files are used rather than a copy. This means that as you modify your source the updates are automatically reflected in your virtualenv. Since you don't seem to actually be modifying Thinc, just getting it set up, you don't need that flag to use it. (But again, it helps with tests due to path issues or something.)
This issue has been automatically closed because it was answered and there was no follow-up discussion.
Hi, I am using a virtulenv with Python 3.6 on aarch64 infrastructure.
I have run the instructions as described to build from source. No issues faced. I was on v0.8.0 branch.
When I run: python3 -m pytest
I get:
_________________________________________________________________________ ERROR collecting thinc/tests/regression/test_issue208.py _________________________________________________________________________ ImportError while importing test module '/home/vap-team-admin/Desktop/balefire_jetson/downloads/thinc/thinc/tests/regression/test_issue208.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib/python3.6/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) thinc/tests/regression/test_issue208.py:1: in <module> from thinc.api import chain, Linear thinc/api.py:2: in <module> from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init thinc/initializers.py:4: in <module> from .backends import Ops thinc/backends/__init__.py:8: in <module> from .cupy_ops import CupyOps, has_cupy thinc/backends/cupy_ops.py:18: in <module> from .numpy_ops import NumpyOps E ModuleNotFoundError: No module named 'thinc.backends.numpy_ops' ========================================================================================= short test summary info ========================================================================================== ERROR thinc/tests/test_config.py ERROR thinc/tests/test_initializers.py ERROR thinc/tests/test_loss.py ERROR thinc/tests/test_optimizers.py ERROR thinc/tests/test_schedules.py ERROR thinc/tests/test_serialize.py ERROR thinc/tests/test_util.py ERROR thinc/tests/backends/test_mem.py ERROR thinc/tests/backends/test_ops.py ERROR thinc/tests/extra/test_beam_search.py ERROR thinc/tests/layers/test_basic_tagger.py ERROR thinc/tests/layers/test_combinators.py ERROR thinc/tests/layers/test_feed_forward.py ERROR thinc/tests/layers/test_hash_embed.py ERROR thinc/tests/layers/test_layers_api.py ERROR thinc/tests/layers/test_linear.py ERROR thinc/tests/layers/test_lstm.py ERROR thinc/tests/layers/test_mnist.py ERROR thinc/tests/layers/test_mxnet_wrapper.py ERROR thinc/tests/layers/test_pytorch_wrapper.py ERROR thinc/tests/layers/test_reduce_first_last.py ERROR thinc/tests/layers/test_shim.py ERROR thinc/tests/layers/test_sparse_linear.py ERROR thinc/tests/layers/test_tensorflow_wrapper.py ERROR thinc/tests/layers/test_transforms.py ERROR thinc/tests/layers/test_uniqued.py ERROR thinc/tests/layers/test_with_debug.py ERROR thinc/tests/layers/test_with_transforms.py ERROR thinc/tests/model/test_model.py ERROR thinc/tests/model/test_validation.py ERROR thinc/tests/regression/test_issue208.py !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 31 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ============================================================================================ 31 errors in 2.74s ============================================================================================
Full message is longer but its just a repeat of "ERROR collecting" message