Closed brando90 closed 4 years ago
brando90
commented
4 years ago I also noticed that in the bottom right I have a notification:
There is already a debug configuration "Python: Current File (Integrated Terminal)" running.but I'm unsure what that means.
brando90
commented
4 years ago I also noticed that in the bottom right I have a notification:
There is already a debug configuration "Python: Current File (Integrated Terminal)" running.but I'm unsure what that means.
Ok, I shut down VS code and re-opened it and that notification message (I've quoted) went away...so probably wasn't important?
but the main error remains.
brando90
commented
4 years ago It's very interesting but I think I found something.
To my very shocking surprise, it seems that my own code interferes with the debugger for some very mysterious reason. This should not be happening. I would have never guessed the debugger would interfere with my code...
The line of code that seems to be doing this is the standard way to loop through data sets in pytorch:
for epoch in range(nb_epochs):
for i, data in enumerate(trainloader, 0):
print(f'i={i}')it fails whenever it reaches the second for loop and thus never executes the print statement.
brando90
commented
4 years ago I tried changing the number of workers to 1 (since I think thats something to do with multiprocesses) but the debugger still freaked out with the same error as above...
brando90
commented
4 years ago I've tried different version of the python extension down to 2019.11.49689 but no luck...error still happens.
brando90
commented
4 years ago I went to version 2018.12.1 and now the debugger seems to freeze once it reaches the famous line:
for i, data in enumerate(trainloader, 0):which gives me a feel it must be something wrong with the extension itself if changing the version changed how it got stuck with the debugger...
brando90
commented
4 years ago seems that version 0.9.1 makes the debugger work...so this confirms that there is something wrong with the python extension.
brando90
commented
4 years ago I'm doing bindary search to find the version where it stops working:
2018.1.0 debugger works 2018.7.0 debugger works 2018.7.1 debugger works 2018.8.0 debugger works 2018.9.0 debugger works 2018.9.1 debugger works 2018.9.2 debugger works
brando90
commented
4 years ago @karthiknadig Well running the cifar10 tutorial on version 2020.1.58038 gives me a multithreaded/processing error (can't tell if its the same, the error messages are to large). But at least I found a script that reproduces the error:
# To add a new cell, type '# %%'
# To add a new markdown cell, type '# %% [markdown]'
# %%
#from IPython import get_ipython
# %%
#get_ipython().run_line_magic('matplotlib', 'inline')
# %% [markdown]
#
# Training a Classifier
# =====================
#
# This is it. You have seen how to define neural networks, compute loss and make
# updates to the weights of the network.
#
# Now you might be thinking,
#
# What about data?
# ----------------
#
# Generally, when you have to deal with image, text, audio or video data,
# you can use standard python packages that load data into a numpy array.
# Then you can convert this array into a ``torch.*Tensor``.
#
# - For images, packages such as Pillow, OpenCV are useful
# - For audio, packages such as scipy and librosa
# - For text, either raw Python or Cython based loading, or NLTK and
# SpaCy are useful
#
# Specifically for vision, we have created a package called
# ``torchvision``, that has data loaders for common datasets such as
# Imagenet, CIFAR10, MNIST, etc. and data transformers for images, viz.,
# ``torchvision.datasets`` and ``torch.utils.data.DataLoader``.
#
# This provides a huge convenience and avoids writing boilerplate code.
#
# For this tutorial, we will use the CIFAR10 dataset.
# It has the classes: ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,
# ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’. The images in CIFAR-10 are of
# size 3x32x32, i.e. 3-channel color images of 32x32 pixels in size.
#
# .. figure:: /_static/img/cifar10.png
# :alt: cifar10
#
# cifar10
#
#
# Training an image classifier
# ----------------------------
#
# We will do the following steps in order:
#
# 1. Load and normalizing the CIFAR10 training and test datasets using
# ``torchvision``
# 2. Define a Convolutional Neural Network
# 3. Define a loss function
# 4. Train the network on the training data
# 5. Test the network on the test data
#
# 1. Loading and normalizing CIFAR10
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#
# Using ``torchvision``, it’s extremely easy to load CIFAR10.
#
#
# %%
import torch
import torchvision
import torchvision.transforms as transforms
# %% [markdown]
# The output of torchvision datasets are PILImage images of range [0, 1].
# We transform them to Tensors of normalized range [-1, 1].
#
#
# %%
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
shuffle=True, num_workers=2)
testset = torchvision.datasets.CIFAR10(root='./data', train=False,
download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
shuffle=False, num_workers=2)
classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
x,y = trainset[0]
x.shape
# %% [markdown]
# Let us show some of the training images, for fun.
#
#
# %%
import matplotlib.pyplot as plt
import numpy as np
# functions to show an image
def imshow(img):
img = img / 2 + 0.5 # unnormalize
npimg = img.numpy()
plt.imshow(np.transpose(npimg, (1, 2, 0)))
plt.show()
# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()
# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))
# %% [markdown]
# 2. Define a Convolutional Neural Network
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Copy the neural network from the Neural Networks section before and modify it to
# take 3-channel images (instead of 1-channel images as it was defined).
#
#
# %%
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 5 * 5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16 * 5 * 5)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
net = Net()
# %% [markdown]
# 3. Define a Loss function and optimizer
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Let's use a Classification Cross-Entropy loss and SGD with momentum.
#
#
# %%
import torch.optim as optim
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
# %% [markdown]
# 4. Train the network
# ^^^^^^^^^^^^^^^^^^^^
#
# This is when things start to get interesting.
# We simply have to loop over our data iterator, and feed the inputs to the
# network and optimize.
#
#
# %%
for epoch in range(2): # loop over the dataset multiple times
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
# get the inputs; data is a list of [inputs, labels]
inputs, labels = data
# zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# print statistics
running_loss += loss.item()
if i % 2000 == 1999: # print every 2000 mini-batches
print('[%d, %5d] loss: %.3f' %
(epoch + 1, i + 1, running_loss / 2000))
running_loss = 0.0
print('Finished Training')
# %% [markdown]
# 5. Test the network on the test data
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#
# We have trained the network for 2 passes over the training dataset.
# But we need to check if the network has learnt anything at all.
#
# We will check this by predicting the class label that the neural network
# outputs, and checking it against the ground-truth. If the prediction is
# correct, we add the sample to the list of correct predictions.
#
# Okay, first step. Let us display an image from the test set to get familiar.
#
#
# %%
dataiter = iter(testloader)
images, labels = dataiter.next()
# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))
# %% [markdown]
# Okay, now let us see what the neural network thinks these examples above are:
#
#
# %%
outputs = net(images)
# %% [markdown]
# The outputs are energies for the 10 classes.
# The higher the energy for a class, the more the network
# thinks that the image is of the particular class.
# So, let's get the index of the highest energy:
#
#
# %%
_, predicted = torch.max(outputs, 1)
print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]
for j in range(4)))
# %% [markdown]
# The results seem pretty good.
#
# Let us look at how the network performs on the whole dataset.
#
#
# %%
correct = 0
total = 0
with torch.no_grad():
for data in testloader:
images, labels = data
outputs = net(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test images: %d %%' % (
100 * correct / total))
# %% [markdown]
# That looks way better than chance, which is 10% accuracy (randomly picking
# a class out of 10 classes).
# Seems like the network learnt something.
#
# Hmmm, what are the classes that performed well, and the classes that did
# not perform well:
#
#
# %%
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
for data in testloader:
images, labels = data
outputs = net(images)
_, predicted = torch.max(outputs, 1)
c = (predicted == labels).squeeze()
for i in range(4):
label = labels[i]
class_correct[label] += c[i].item()
class_total[label] += 1
for i in range(10):
print('Accuracy of %5s : %2d %%' % (
classes[i], 100 * class_correct[i] / class_total[i]))
# %% [markdown]
# Okay, so what next?
#
# How do we run these neural networks on the GPU?
#
# Training on GPU
# ----------------
# Just like how you transfer a Tensor onto the GPU, you transfer the neural
# net onto the GPU.
#
# Let's first define our device as the first visible cuda device if we have
# CUDA available:
#
#
# %%
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# Assuming that we are on a CUDA machine, this should print a CUDA device:
print(device)
# %% [markdown]
# The rest of this section assumes that ``device`` is a CUDA device.
#
# Then these methods will recursively go over all modules and convert their
# parameters and buffers to CUDA tensors:
#
# .. code:: python
#
# net.to(device)
#
#
# Remember that you will have to send the inputs and targets at every step
# to the GPU too:
#
# .. code:: python
#
# inputs, labels = data[0].to(device), data[1].to(device)
#
# Why dont I notice MASSIVE speedup compared to CPU? Because your network
# is really small.
#
# **Exercise:** Try increasing the width of your network (argument 2 of
# the first ``nn.Conv2d``, and argument 1 of the second ``nn.Conv2d`` –
# they need to be the same number), see what kind of speedup you get.
#
# **Goals achieved**:
#
# - Understanding PyTorch's Tensor library and neural networks at a high level.
# - Train a small neural network to classify images
#
# Training on multiple GPUs
# -------------------------
# If you want to see even more MASSIVE speedup using all of your GPUs,
# please check out :doc:`data_parallel_tutorial`.
#
# Where do I go next?
# -------------------
#
# - :doc:`Train neural nets to play video games </intermediate/reinforcement_q_learning>`
# - `Train a state-of-the-art ResNet network on imagenet`_
# - `Train a face generator using Generative Adversarial Networks`_
# - `Train a word-level language model using Recurrent LSTM networks`_
# - `More examples`_
# - `More tutorials`_
# - `Discuss PyTorch on the Forums`_
# - `Chat with other users on Slack`_
#
#
#
# %%
karthiknadig
commented
4 years ago @brando90 Looking at the first stack trace. The extension is using the old debugger, which had limited support for multiprocess (the path has old_ptvsd on it). Here are the instructions to turn on the newer debugger. https://github.com/microsoft/ptvsd/issues/1706#issuecomment-572287267
If you run into similar issue with the new one, let us know.
brando90
commented
4 years ago @brando90 Looking at the first stack trace. The extension is using the old debugger, which had limited support for
multiprocess(the path hasold_ptvsdon it). Here are the instructions to turn on the newer debugger. microsoft/ptvsd#1706 (comment)If you run into similar issue with the new one, let us know.
Why is it that having the most recent version of the python vs code extension not automatically activate all the most up to date features?
karthiknadig
commented
4 years ago Why is it that having the most recent version of the python vs code extension not automatically activate all the most up to date features?
We roll out features in waves. The new debugger is available to 20% of the users. We added to ability to Opt-into new features this month. To use the new debugger you have to opt into the experiments in the referenced comment. You need the insiders version of the python extension which is the VSIX file that you download. You can install that file in VS Code using Extension > Install from VSIX :
brando90
commented
4 years ago @karthiknadig
I am getting a new bug despite following the instructions of installing the internal debugger you sent me. Err msg:
(automl-meta-learning) brandomiranda~/automl-meta-learning ❯ cd /Users/brandomiranda/automl-meta-learning ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /Users/brandomiranda/miniconda3/envs/automl-meta-learning/bin/python /Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 63618 /Users/brandomiranda/automl-meta-learning/automl/automl/meta_optimizers/differentiable_SGD.py
--> main in differentiable SGD
hello world
Files already downloaded and verified
Files already downloaded and verified
Files already downloaded and verified
-> Decoder ran successfully!
-> Sampler ran successfully!
t = 0
E00007.831: Exception escaped from start_client
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/ptvsd_launcher.py", line 48, in <module>
main(ptvsdArgs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
run()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/brandomiranda/automl-meta-learning/automl/automl/meta_optimizers/differentiable_SGD.py", line 155, in <module>
learned_scheduler_with_NN()
File "/Users/brandomiranda/automl-meta-learning/automl/automl/meta_optimizers/differentiable_SGD.py", line 144, in learned_scheduler_with_NN
out_eta0=a, h0=h, c0=c, trainloader=trainloader, criterion=criterion
File "/Users/brandomiranda/automl-meta-learning/automl/automl/meta_optimizers/differentiable_SGD.py", line 116, in meta_optimize
grads = self.grad1(mdl0, mdl, trainloader, criterion, nb_iterations=1, nb_epochs=1)
File "/Users/brandomiranda/automl-meta-learning/automl/automl/meta_optimizers/differentiable_SGD.py", line 79, in grad1
for i, data in enumerate(trainloader, 0):
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
return _MultiProcessingDataLoaderIter(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 719, in __init__
w.start()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 112, in start
self._popen = self._Popen(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
self._launch(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
self.pid = os.fork()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 528, in new_fork
_on_forked_process()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 50, in _on_forked_process
pydevd.settrace_forked()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2427, in settrace_forked
patch_multiprocessing=True,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
wait_for_ready_to_run,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2230, in _locked_settrace
debugger.connect(host, port) # Note: connect can raise error.
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 1060, in connect
s = start_client(host, port)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 136, in _start_client
return start_client(daemon, h, p)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.2.60336-dev/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
zsh: terminated env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 --default --client --hostit's at the same line where I try to loop over a pytorch data loader.
fabioz
commented
4 years ago It's still using the old_ptvsd, so, something is missing from the steps (https://github.com/microsoft/ptvsd/issues/1706#issuecomment-572287267) to opt into using the new version.
fabioz
commented
4 years ago Closing as this is a known issue with the old version of ptvsd.
brando90
commented
4 years ago @fabioz @karthiknadig now I have this issue:
Failed to launch debugger for child process 30617why? How do I debug this?
Version: 1.42.1 Commit: c47d83b293181d9be64f27ff093689e8e7aed054 Date: 2020-02-11T14:44:27.652Z Electron: 6.1.6 Chrome: 76.0.3809.146 Node.js: 12.4.0 V8: 7.6.303.31-electron.0 OS: Darwin x64 19.3.0
martimors
commented
4 years ago ^^^^^ I have the same issue!
int19h
commented
4 years ago Can you please file a separate issue for this, and include the logs produced by "logToFile":true in your launch.json?
martimors
commented
4 years ago Can you please file a separate issue for this, and include the logs produced by
"logToFile":truein your launch.json?
Hi,
This has no effect in my client (at least I cannot find any log file anywhere).
However, the issue seems to be resolved in 1.45.0.
karthiknadig
commented
4 years ago @dingobar For next time the logs are stored in ~/.vscode/extensions/ms-python.python<version>/ directory.
brando90
commented
4 years ago Closing as this is a known issue with the old version of ptvsd.
I am having this issue again:
(automl-meta-learning) brandomiranda~/automl-meta-learning ❯ cd /Users/brandomiranda/automl-meta-learning ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /Users/brandomiranda/miniconda3/envs/automl-meta-learning/bin/python /Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 60347 /Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py
-------> Inside Experiment Code <--------
---> hostname: Beatrizs-iMac.local, current_time: Jun21_15-58-48, githash: b'cee887edd44ae7ad8dcb093c3e44fef76b718cec\n'
root:INFO:logger.py:loginfo:lineno 95:-> githash = b'cee887edd44ae7ad8dcb093c3e44fef76b718cec\n' hostname = Beatrizs-iMac.local current_time: Jun21_15-58-48
root:INFO:logger.py:loginfo:lineno 95:-> os.environ = environ({'SHELL': '/bin/zsh', 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.rxxKFF8G3T/Listeners', 'XPC_FLAGS': '0x0', 'XPC_SERVICE_NAME': '0', 'HOME': '/Users/brandomiranda', 'LOGNAME': 'brandomiranda', 'TMPDIR': '/var/folders/1q/vzn82jt941n11cw3jgpzcgxm0000gp/T/', 'TERM_PROGRAM': 'vscode', 'TERM_PROGRAM_VERSION': '1.46.1', 'LANG': 'en_US.UTF-8', 'COLORTERM': 'truecolor', 'VSCODE_GIT_IPC_HANDLE': '/var/folders/1q/vzn82jt941n11cw3jgpzcgxm0000gp/T/vscode-git-dbce52237c.sock', 'GIT_ASKPASS': '/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh', 'VSCODE_GIT_ASKPASS_NODE': '/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer)', 'VSCODE_GIT_ASKPASS_MAIN': '/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js', 'PWD': '/Users/brandomiranda/automl-meta-learning', 'TERM': 'xterm-256color', 'SHLVL': '1', 'OLDPWD': '/Users/brandomiranda/automl-meta-learning', 'CONDA_EXE': '/Users/brandomiranda/miniconda3/bin/conda', '_CE_M': '', '_CE_CONDA': '', 'CONDA_PYTHON_EXE': '/Users/brandomiranda/miniconda3/bin/python', 'CONDA_SHLVL': '2', 'PATH': '/Users/brandomiranda/miniconda3/envs/automl-meta-learning/bin:/Users/brandomiranda/miniconda3/bin:/Users/brandomiranda/miniconda3/condabin:/bin:/usr/bin:/usr/ucb:/usr/local/bin', 'CONDA_PREFIX': '/Users/brandomiranda/miniconda3/envs/automl-meta-learning', 'CONDA_DEFAULT_ENV': 'automl-meta-learning', 'CONDA_PROMPT_MODIFIER': '(automl-meta-learning) ', 'CONDA_PREFIX_1': '/Users/brandomiranda/miniconda3', '_': '/usr/bin/env', 'PYTHONIOENCODING': 'UTF-8', 'PYTHONUNBUFFERED': '1'})
root:INFO:logger.py:loginfo:lineno 95:-> >>> Selected random seed: 3062923817
len(self.episode_loader) = 64
len(self.episode_loader) = 16
len(self.episode_loader) = 20
root:INFO:logger.py:loginfo:lineno 95:-> Is tensorboard being used? False
root:INFO:logger.py:loginfo:lineno 95:-> About to start training with args: namespace(ExceptionType=UncatachableException(), base_model='TO BE SET', base_model_mode='child_mdl_from_opt_as_a_mdl_for_few_shot_learning_paper', bn_eps=0.001, bn_momentum=0.95, comment='_episodes_60000_nb_inner_train_steps_5_base_model_mode_child_mdl_from_opt_as_a_mdl_for_few_shot_learning_paper', condor_jobid=-1, copy_initial_weights=False, criterion=CrossEntropyLoss(), current_logs_path=PosixPath('/Users/brandomiranda/automl-meta-learning/automl-proj/experiments/logs/logs_Jun21_15-58-48_jobid_-1'), data_root=PosixPath('/Users/brandomiranda/automl-meta-learning/data/miniImagenet'), debug=False, debug_test=False, device=device(type='cpu'), episodes=60000, episodes_test=100, episodes_val=25, experiment_name='exp', githash=b'cee887edd44ae7ad8dcb093c3e44fef76b718cec\n', grad_clip_mode=None, hostname='Beatrizs-iMac.local', image_size=84, inner_debug_eval=False, inner_debug_train=False, inner_lr=0.1, jobid=-1, k_eval=15, k_shot=5, log_root=PosixPath('/Users/brandomiranda/automl-meta-learning/automl-proj/experiments/logs'), log_train_freq=10, log_val_freq=1000, logger=<uutils.logger.Logger object at 0x7fb9ae79db50>, logging=True, mail_user='brando.science@gmail.com', meta_learner='maml_fixed_inner_lr', mode='meta-train', my_stdout_filepath=PosixPath('/Users/brandomiranda/automl-meta-learning/automl-proj/experiments/logs/logs_Jun21_15-58-48_jobid_-1/my_stdout.log'), n_classes=5, n_workers=4, nb_inner_train_steps=5, outer_debug=True, outer_i=1, outer_lr=0.001, pin_mem=True, pw_path=PosixPath('/Users/brandomiranda/pw_app.config.json'), resume_ckpt_path=None, save_ckpt=True, seed=3062923817, slurm_array_task_id=-1, slurm_jobid=-1, tb=False, track_higher_grads=True)
E00034.313: Exception escaped from start_client
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
E00034.313: Exception escaped from start_client
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
E00034.313: Exception escaped from start_client
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
E00034.313: Exception escaped from start_client
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py", line 48, in <module>
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py", line 48, in <module>
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py", line 48, in <module>
main(ptvsdArgs)
File "/Users/brandomi main(ptvsdArgs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py", line 48, in <module>
main(ptvsdArgs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
run()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
main(ptvsdArgs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
run()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
randa/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
run()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
run()mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 85, in _run_code
pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
exec(code, run_globals)
mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 288, in <module>
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 85, in _run_code
mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 288, in <module>
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
args = main(args)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 239, in main
exec(code, run_globals)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 288, in <module>
mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/li args = main(args)meta_train(args, meta_learner, outer_opt, meta_train_set, meta_val_set)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 239, in main
File "/Users/brandomiranda/automl-meta-learning/automl-proj/meta_learning/training/meta_training.py", line 49, in meta_train
args = main(args)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 239, in main
for (SQ_x, SQ_y) in meta_train_set: # sample data set split episode_x = D = (D^{train},D^{test})
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
meta_train(args, meta_learner, outer_opt, meta_train_set, meta_val_set)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/meta_learning/training/meta_training.py", line 49, in meta_train
meta_train(args, meta_learner, outer_opt, meta_train_set, meta_val_set)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/meta_learning/training/meta_training.py", line 49, in meta_train
for (SQ_x, SQ_y) in meta_train_set: # sample data set split episode_x = D = (D^{train},D^{test})
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
b/python3.7/runpy.py", line 85, in _run_code
for (SQ_x, SQ_y) in meta_train_set: # sample data set split episode_x = D = (D^{train},D^{test})
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
exec(code, run_globals)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 288, in <module>
args = main(args)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 239, in main
meta_train(args, meta_learner, outer_opt, meta_train_set, meta_val_set)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/meta_learning/training/meta_training.py", line 49, in meta_train
for (SQ_x, SQ_y) in meta_train_set: # sample data set split episode_x = D = (D^{train},D^{test})
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
return _MultiProcessingDataLoaderIter(self)return _MultiProcessingDataLoaderIter(self)return _MultiProcessingDataLoaderIter(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 719, in __init__
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 719, in __init__
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 719, in __init__
return _MultiProcessingDataLoaderIter(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 719, in __init__
w.start()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 112, in start
w.start()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 112, in start
w.start()w.start()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 112, in start
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 112, in start
self._popen = self._Popen(self)self._popen = self._Popen(self)self._popen = self._Popen(self)self._popen = self._Popen(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)return _default_context.get_context().Process._Popen(process_obj)return _default_context.get_context().Process._Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
return Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
return Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
return Popen(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
self._launch(process_obj)self._launch(process_obj)self._launch(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
self._launch(process_obj)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/popen_fork.py", line 70, in _launch
self.pid = os.fork()
self.pid = os.fork()
self.pid = os.fork()
self.pid = os.fork() File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 528, in new_fork
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.891 File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 528, in new_fork
48/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 528, in new_fork
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 528, in new_fork
_on_forked_process()_on_forked_process()_on_forked_process()
_on_forked_process()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 50, in _on_forked_process
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 50, in _on_forked_process
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 50, in _on_forked_process
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 50, in _on_forked_process
pydevd.settrace_forked()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/pydevd.settrace_forked()python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2427, in settrace_forked
pydevd.settrace_forked() File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2427, in settrace_forked
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2427, in settrace_forked
pydevd.settrace_forked()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2427, in settrace_forked
patch_multiprocessing=True,patch_multiprocessing=True,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
patch_multiprocessing=True,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
patch_multiprocessing=True,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
wait_for_ready_to_run,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2230, in _locked_settrace
wait_for_ready_to_run,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2230, in _locked_settrace
wait_for_ready_to_run,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2230, in _locked_settrace
wait_for_ready_to_run,
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2230, in _locked_settrace
debugger.connect(host, port) # Note: connect can raise error.
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydev debugger.connect(host, port) # Note: connect can raise error.
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 1060, in connect
debugger.connect(host, port) # Note: connect can raise error.
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 1060, in connect
s = start_client(host, port)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 136, in _start_client
debugger.connect(host, port) # Note: connect can raise error.
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 1060, in connect
return start_client(daemon, h, p)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
s = start_client(host, port)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 136, in _start_client
d/pydevd.py", line 1060, in connect
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
return start_client(daemon, h, p)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
s = start_client(host, port)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 136, in _start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
return start_client(daemon, h, p)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
s = start_client(host, port)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 136, in _start_client
return next(self.gen)
sock, start_session = daemon.start_client((host, port)) File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
File return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
"/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
return start_client(daemon, h, p)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/log.py", line 110, in g
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
return f(*args, **kwargs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/pydevd_hooks.py", line 74, in start_client
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
raise RuntimeError('already started')
RuntimeError: already started
sock, start_session = daemon.start_client((host, port))
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 214, in start_client
return next(self.gen)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 110, in started
self.start()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
with self.started():
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/contextlib.py", line 112, in __enter__
raise RuntimeErrorError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/util.py", line 334, in _exit_function
p.join()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/multiprocessing/process.py", line 138, in join
assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
Traceback (most recent call last):
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/ptvsd_launcher.py", line 48, in <module>
main(ptvsdArgs)
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
run()
File "/Users/brandomiranda/.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
runpy.run_path(target, run_name='__main__')
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 288, in <module>
args = main(args)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py", line 239, in main
meta_train(args, meta_learner, outer_opt, meta_train_set, meta_val_set)
File "/Users/brandomiranda/automl-meta-learning/automl-proj/meta_learning/training/meta_training.py", line 49, in meta_train
for (SQ_x, SQ_y) in meta_train_set: # sample data set split episode_x = D = (D^{train},D^{test})
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
return _MultiProcessingDataLoaderIter(self)
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 746, in __init__
self._try_put_index()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 861, in _try_put_index
index = self._next_index()
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 339, in _next_index
return next(self._sampler_iter) # may raise StopIteration
File "/Users/brandomiranda/ultimate-utils/ultimate-utils-project/torch_uutils/dataloaders/episodic_dataloader.py", line 282, in __iter__
random_for_indices_all_tasks = torch.randperm(self.total_classes) # Return a random permutation of integers in a range e.g. tensor([28, 36, 53, 6, 2, 38, 9, 42, 46, 58, 44, 25, 41, 20, 26, 62, 57, 63, 16, 27, 32, 61, 29, 21, 45, 48, 60, 7, 56, 0, 47, 4, 50, 39, 49, 35, 43, 15, 33, 17, 13, 24, 59, 14, 22, 37, 34, 1, 8, 11, 10, 54, 3, 51, 19, 52, 12, 5, 31, 23, 55, 18, 30, 40])
File "/Users/brandomiranda/miniconda3/envs/automl-meta-learning/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 42495) is killed by signal: Unknown signal: 0.
zsh: terminated env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 --default --client --host Was this not resolved?
brando90
commented
4 years ago brando90
commented
4 years ago 1706 (comment)
I want to always have a version where the multithreading is working. How do I do that?
brando90
commented
4 years ago For now this works but I do not want to have to change settings of vscode. I want vscode to work and not have me think about the editor at all.
Current solution:
"DebugAdapterFactory - experiment",
"PtvsdWheels37 - experiment"
]
teichert
commented
4 years ago Might be related: A fresh install of the stable python extension a few minutes ago still resulted in the following error (when using pytorch with anaconda on an ubuntu machine); switching to the weekly channel resolved it:
...
File "/home/adam/.vscode/extensions/ms-python.python-2020.6.91350/pythonFiles/lib/python/old_ptvsd/ptvsd/daemon.py", line 145, in start
raise RuntimeError('already started')
RuntimeError: already started
patch_multiprocessing=True,
File "/home/adam/.vscode/extensions/ms-python.python-2020.6.91350/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/pydevd.py", line 2179, in settrace
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/home/adam/anaconda3/lib/python3.7/multiprocessing/util.py", line 322, in _exit_function
p.join()
File "/home/adam/anaconda3/lib/python3.7/multiprocessing/process.py", line 138, in join
assert self._parent_pid == os.getpid(), 'can only join a child process'
AssertionError: can only join a child process
TerminatedThe error message is also indicative of not using the new debugger. Following the suggestion in https://github.com/microsoft/ptvsd/issues/2062#issuecomment-647625098 should fix this.
For the next stable version of the extension (or for the current insiders builds), you won't be needing that anymore, because the old debugger is gone entirely.
Environment data
"python.jediEnabled"set to; more info microsoft/vscode-python#3977): N/AExpected behaviour
When I run the debugger I expect it to stop at my break point but it does not. It throws me an error instead. The run without debugger seems to work though.
Actual behaviour
Following giant error is thrown in the integrated vs code terminal:
Steps to reproduce:
[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]
Try to loop through a trainloader in pytorch:
the second line freaks out my debugger and throws the above error.
Logs (Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython))Empty logs
Output from
Consoleunder theDeveloper Toolspanel (toggle Developer Tools on underHelp; turn on source maps to make any tracebacks be useful by runningEnable source map support for extension debugging)I tried this but nothing interesting appeared...
SO: https://stackoverflow.com/questions/59955601/how-does-one-fix-the-vs-code-debugger-if-it-stopped-working-and-throws-a-multith developer community post: https://developercommunity.visualstudio.com/content/problem/899746/how-does-one-fix-the-vs-code-debugger-if-it-stoppe.html VS: https://social.msdn.microsoft.com/Forums/vstudio/en-US/eed257c5-e48e-4762-ae7b-3a9a54e4b8bb/how-does-one-fix-the-vs-code-debugger-if-it-stopped-working-and-throws-a-multithreadedmultiprocess?forum=vsdebug