search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.33k stars 1.61k forks source link

Pytorch example internal error #15486

Closed tbress closed 3 years ago

tbress commented 3 years ago

Description

What steps will reproduce the problem?

In the Coursera course on machine learning lab (https://www.coursera.org/learn/machine-learning-duke/ungradedLab/IP4b8/logistic-regression/lab) the following code is given. When I run it in Spyder I get an internal error:

import numpy as np

import matplotlib.pyplot as plt

import torch import torch.nn.functional as F

from tqdm.notebook import tqdm

from torchvision import datasets, transforms

Load the data

mnist_train = datasets.MNIST(root="./datasets", train=True, transform=transforms.ToTensor(), download=True) mnist_test = datasets.MNIST(root="./datasets", train=False, transform=transforms.ToTensor(), download=True) train_loader = torch.utils.data.DataLoader(mnist_train, batch_size=100, shuffle=True) test_loader = torch.utils.data.DataLoader(mnist_test, batch_size=100, shuffle=False)

Training

Initialize parameters

W = torch.randn(784, 10)/np.sqrt(784) W.requiresgrad() b = torch.zeros(10, requires_grad=True)

Optimizer (lr is the learning rate, or step size)

optimizer = torch.optim.SGD([W,b], lr=0.1)

Iterate through the training set minibatches (one epoch)

for images, labels in train_loader:

Zero out the gradient

optimizer.zero_grad()
# Forward pass
x = images.view(-1, 28*28)
y = torch.matmul(x, W) + b
cross_entropy = F.cross_entropy(y, labels)
# Backward pass
cross_entropy.backward()
optimizer.step()

Testing

correct = 0 total = len(mnist_test)

with torch.no_grad():

Iterate through the test set minibatches

for images, labels in test_loader:
    # Forward pass
    x = images.view(-1, 28*28)
    y = torch.matmul(x, W) + b

    predictions = torch.argmax(y, dim=1)
    correct += torch.sum((predictions == labels).float())

print('Test accuracy: {}'.format(correct/total))

Traceback

Exception in comms call get_namespace_view:
  File "C:\Users\615237\AppData\Local\Programs\Spyder\pkgs\spyder_kernels\comms\commbase.py", line 315, in _comm_message
    encoding='latin-1')
ModuleNotFoundError: No module named 'torch'

Versions

Dependencies


# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  1.6.0 (OK)
cookiecutter >=1.6.0          :  1.7.2 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree                  :  None (OK)
IPython >=7.6.0               :  7.22.0 (OK)
jedi =0.17.2                  :  0.17.2 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.0.1 (OK)
nbconvert >=4.0               :  6.0.7 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
paramiko >=2.4.0              :  2.7.2 (OK)
parso =0.7.0                  :  0.7.0 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.8.1 (OK)
pylint >=1.0                  :  2.7.4 (OK)
pyls >=0.36.2;<1.0.0          :  0.36.2 (OK)
pyls_black >=0.4.6            :  0.4.6 (OK)
pyls_spyder >=0.3.2           :  0.3.2 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.1.10 (OK)
qtawesome >=1.0.2             :  1.0.2 (OK)
qtconsole >=5.0.3             :  5.0.3 (OK)
qtpy >=1.5.0                  :  1.9.0 (OK)
rtree >=0.8.3                 :  0.9.4 (OK)
setuptools >=39.0.0           :  56.0.0 (OK)
sphinx >=0.6.6                :  3.5.4 (OK)
spyder_kernels >=2.0.1;<2.1.0 :  2.0.1 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog                      :  1.0.2 (OK)
zmq >=17                      :  22.0.3 (OK)

# Optional:
cython >=0.21                 :  0.29.23 (OK)
matplotlib >=2.0.0            :  None (NOK)
numpy >=1.7                   :  1.19.3 (OK)
pandas >=1.1.1                :  1.2.4 (OK)
scipy >=0.17.0                :  1.6.2 (OK)
sympy >=0.7.3                 :  1.8 (OK)
ccordoba12 commented 3 years ago

Hey @tbress, thanks for reporting. If you want to use to other packages that don't come with our Windows installer (like Pytorch), you need to install Miniconda, create a conda environment after that with the packages you want to use and spyder-kernels, and finally connect Spyder to that environment.

Please try that and let us know if it works for you.

tbress commented 3 years ago

Thanks, Carlos. I'm trying to follow your advice using this website: Working with packages and environments in Spyder · spyder-ide/spyder Wiki (github.com) https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder#the-modular-approach. I installed miniconda and have a base environment activated. I installed spyder-kernels in that environment. When I list the packages in the environment I see that I have spyder-kernels v1.10.2. When I point Spyder at the miniconda directory I get an error message saying "Your Python environment or installation doesn't have the spyder‑kernels module or the right version of it installed (>= 2.0.1 and < 2.1.0). Without this module is not possible for Spyder to create a console for you."

I then tried updating spyder-kernels using "conda install -c conda-forge spyder-kernels" in my base environment, but when I list the packages I still see the v1.10.2 and I still can't open a kernel in Spyder. I'm new to setting up environments, is there a step I'm missing? Thanks.

On Sun, May 2, 2021 at 7:12 PM Carlos Cordoba @.***> wrote:

Hey @tbress https://github.com/tbress, thanks for reporting. If you want to use to other packages that don't come with our Windows installer (like Pytorch), you need to install Miniconda https://docs.conda.io/en/latest/miniconda.html, create a conda environment https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-with-commands after that with the packages you want to use and spyder-kernels, and finally connect Spyder http://docs.spyder-ide.org/current/faq.html#using-existing-environment to that environment.

Please try that and let us know if it works for you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/15486#issuecomment-830923498, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATT2ZUNCWZZTAA5L3654HW3TLXL6LANCNFSM437ZJGKA .

ccordoba12 commented 3 years ago

I'm trying to follow your advice using this website

Great! That page has a more detailed explanation than the one I provided above.

I'm new to setting up environments, is there a step I'm missing? Thanks

Ok, please then run

conda install -c conda-forge spyder-kernels=2.0.1

to force the installation of that version instead of 1.10.2

tbress commented 3 years ago

Thanks, that works. I appreciate your help, I've used Spyder for engineering and science work in the past and now I going to take a data science class and want to use Spyder as my IDE. I now have miniconda installed, and pytorch installed in the miniconda environment as well as the correct version of spyder-kernels. I can start Spyder and connect to a kernel. But when I try running the code I sent you before I still have an issue. The code completes successfully and gives me the right answer, but I still get the Spyder internal problem pop-up window. When I show the details of the internal error pop-up I get:

Exception in comms call get_namespace_view:

File "C:\Users\615237\AppData\Local\Programs\Spyder\pkgs\spyder_kernels\comms\commbase.py", line 315, in _comm_message

encoding='latin-1')

ModuleNotFoundError: No module named 'torch'

On Mon, May 3, 2021 at 12:17 PM Carlos Cordoba @.***> wrote:

I'm trying to follow your advice using this website

Great! That page has a more detailed explanation than the one I provided above.

I'm new to setting up environments, is there a step I'm missing? Thanks

Ok, please then run

conda install -c conda-forge spyder-kernels=2.0.1

to force the installation of that version instead of 1.10.2

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/15486#issuecomment-831370915, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATT2ZUI4VCTMOCLIVIWGTQLTL3EDLANCNFSM437ZJGKA .

ccordoba12 commented 3 years ago

I think you're missing the last step I mentioned above:

and finally connect Spyder to that environment.

Please follow the instructions in the link to connect your Miniconda environment to Spyder.

tbress commented 3 years ago

Thanks, Carlos. I took your advice again and came across something strange. First I rebooted my computer, then started Spyder. I had previously set my Python preference for the miniconda environment, so it started in that environment. I went through the steps of reselecting the miniconda Python environment again, there was no issue with that. But then I tried to restart the kernel as you suggested in your link. I got the following error:

Traceback (most recent call last): File "C:\Users\615237\Miniconda3\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\615237\Miniconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\615237\Miniconda3\lib\site‑packages\spyder_kernels\console__main__.py", line 23, in start.main() File "C:\Users\615237\Miniconda3\lib\site‑packages\spyder_kernels\console\start.py", line 284, in main kernel.initialize() File "C:\Users\615237\Miniconda3\lib\site‑packages\traitlets\config\application.py", line 87, in inner return method(app, *args, **kwargs) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 567, in initialize self.init_sockets() File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 271, in init_sockets self.shell_port = self._bind_socket(self.shell_socket, self.shell_port) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 218, in _bind_socket return self._try_bind_socket(s, port) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 194, in _try_bind_socket s.bind("tcp://%s:%i" % (self.ip, port)) File "C:\Users\615237\Miniconda3\lib\site‑packages\zmq\sugar\socket.py", line 173, in bind super().bind(addr) File "zmq/backend/cython/socket.pyx", line 542, in zmq.backend.cython.socket.Socket.bind File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc zmq.error.ZMQError: Address in use

On Mon, May 3, 2021 at 8:09 PM Carlos Cordoba @.***> wrote:

I think you're missing the last step I mentioned above:

and finally connect Spyder http://docs.spyder-ide.org/current/faq.html#using-existing-environment to that environment.

Please follow the instructions in the link to connect your Miniconda environment to Spyder.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/15486#issuecomment-831610484, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATT2ZUNCJCIBPWBJCI3K2YTTL43LBANCNFSM437ZJGKA .

tbress commented 3 years ago

Hello, Carlos. I believe I've solved the problem. Originally I had installed Spyder before having installed miniconda. I then installed miniconda later and installed the other packages. I uninstalled Spyder and reinstalled using conda and everything works now. Thanks again for your help.

On Tue, May 4, 2021 at 12:02 PM Thomas Bress @.***> wrote:

Thanks, Carlos. I took your advice again and came across something strange. First I rebooted my computer, then started Spyder. I had previously set my Python preference for the miniconda environment, so it started in that environment. I went through the steps of reselecting the miniconda Python environment again, there was no issue with that. But then I tried to restart the kernel as you suggested in your link. I got the following error:

Traceback (most recent call last): File "C:\Users\615237\Miniconda3\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\615237\Miniconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\615237\Miniconda3\lib\site‑packages\spyder_kernels\console__main__.py", line 23, in start.main() File "C:\Users\615237\Miniconda3\lib\site‑packages\spyder_kernels\console\start.py", line 284, in main kernel.initialize() File "C:\Users\615237\Miniconda3\lib\site‑packages\traitlets\config\application.py", line 87, in inner return method(app, *args, **kwargs) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 567, in initialize self.init_sockets() File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 271, in init_sockets self.shell_port = self._bind_socket(self.shell_socket, self.shell_port) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 218, in _bind_socket return self._try_bind_socket(s, port) File "C:\Users\615237\Miniconda3\lib\site‑packages\ipykernel\kernelapp.py", line 194, in _try_bind_socket s.bind("tcp://%s:%i" % (self.ip, port)) File "C:\Users\615237\Miniconda3\lib\site‑packages\zmq\sugar\socket.py", line 173, in bind super().bind(addr) File "zmq/backend/cython/socket.pyx", line 542, in zmq.backend.cython.socket.Socket.bind File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc zmq.error.ZMQError: Address in use

On Mon, May 3, 2021 at 8:09 PM Carlos Cordoba @.***> wrote:

I think you're missing the last step I mentioned above:

and finally connect Spyder http://docs.spyder-ide.org/current/faq.html#using-existing-environment to that environment.

Please follow the instructions in the link to connect your Miniconda environment to Spyder.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spyder-ide/spyder/issues/15486#issuecomment-831610484, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATT2ZUNCJCIBPWBJCI3K2YTTL43LBANCNFSM437ZJGKA .

ccordoba12 commented 3 years ago

Great! I'm really glad you were able to finally solve it!

And just so you know, we're planning to improve installing environments and packages for the future.