search

inferno-pytorch / inferno

A utility library around PyTorch
Other
244 stars 41 forks source link

Can't run the basic example #182

Open LucaMarconato opened 5 years ago

LucaMarconato commented 5 years ago

Description

I have tried to run the script in the Readme, after having set the three directories that must be set and disabling CUDA. When running the script with python3 hello_world.py I got two errors, I made the first disappear (see below), but the second is still present. The expected behavior is to get no error.

What I Did

The full code is reported below, in a file called hello_world.py.

import torch.nn as nn
from inferno.io.box.cifar import get_cifar10_loaders
from inferno.trainers.basic import Trainer
from inferno.trainers.callbacks.logging.tensorboard import TensorboardLogger
from inferno.extensions.layers.convolutional import ConvELU2D
from inferno.extensions.layers.reshape import Flatten

# Fill these in:
LOG_DIRECTORY = 'log'
SAVE_DIRECTORY = 'save'
DATASET_DIRECTORY = 'data'
DOWNLOAD_CIFAR = True
USE_CUDA = False

# Build torch model
model = nn.Sequential(
    ConvELU2D(in_channels=3, out_channels=256, kernel_size=3),
    nn.MaxPool2d(kernel_size=2, stride=2),
    ConvELU2D(in_channels=256, out_channels=256, kernel_size=3),
    nn.MaxPool2d(kernel_size=2, stride=2),
    ConvELU2D(in_channels=256, out_channels=256, kernel_size=3),
    nn.MaxPool2d(kernel_size=2, stride=2),
    Flatten(),
    nn.Linear(in_features=(256 * 4 * 4), out_features=10),
    nn.LogSoftmax(dim=1)
)

# Load loaders
train_loader, validate_loader = get_cifar10_loaders(DATASET_DIRECTORY,
                                                    download=DOWNLOAD_CIFAR)

# Build trainer
trainer = Trainer(model) \
  .build_criterion('NLLLoss') \
  .build_metric('CategoricalError') \
  .build_optimizer('Adam') \
  .validate_every((2, 'epochs')) \
  .save_every((5, 'epochs')) \
  .save_to_directory(SAVE_DIRECTORY) \
  .set_max_num_epochs(10) \
  .build_logger(TensorboardLogger(log_scalars_every=(1, 'iteration'),
                                  log_images_every='never'),
                log_directory=LOG_DIRECTORY)

# Bind loaders
trainer \
    .bind_loader('train', train_loader) \
    .bind_loader('validate', validate_loader)

if USE_CUDA:
  trainer.cuda()

# Go!
trainer.fit()

I first created the three folders specified in the script with mkdir log, mkdir save, mkdir data. I then ran the script with python3 hello_world.py. I first got the error:

  File "hello_world.py", line 2, in <module>
    from inferno.io.box.cifar import get_cifar10_loaders
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/__init__.py", line 6, in <module>
    from . import io
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/io/__init__.py", line 4, in <module>
    from . import volumetric
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/io/volumetric/__init__.py", line 1, in <module>
    from .volume import VolumeLoader, HDF5VolumeLoader, TIFVolumeLoader
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/io/volumetric/volume.py", line 8, in <module>
    from ...utils import io_utils as iou
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/utils/io_utils.py", line 5, in <module>
    from scipy.misc import imsave
ImportError: cannot import name 'imsave'

which I could solve by running conda install -c anaconda scipy. I was not expecting this error because, since I installed inferno with conda, I expected all the dependencies to be already installed.

The second error that now I get is the following:

Traceback (most recent call last):
  File "hello_world.py", line 2, in <module>
    from inferno.io.box.cifar import get_cifar10_loaders
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/__init__.py", line 7, in <module>
    from . import trainers
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/trainers/__init__.py", line 1, in <module>
    from . import basic
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/trainers/basic.py", line 20, in <module>
    from .callbacks.logging.base import Logger
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/trainers/callbacks/logging/__init__.py", line 4, in <module>
    from .tensorboard import TensorboardLogger
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/inferno/trainers/callbacks/logging/tensorboard.py", line 1, in <module>
    import tensorboardX as tX
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/writer.py", line 15, in <module>
    from .event_file_writer import EventFileWriter
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/event_file_writer.py", line 28, in <module>
    from .proto import event_pb2
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/proto/event_pb2.py", line 15, in <module>
    from tensorboardX.proto import summary_pb2 as tensorboardX_dot_proto_dot_summary__pb2
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/proto/summary_pb2.py", line 15, in <module>
    from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/proto/tensor_pb2.py", line 15, in <module>
    from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
  File "/miniconda3/envs/my_test/lib/python3.6/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 22, in <module>
    serialized_pb=_b('\n(tensorboardX/proto/resource_handle.proto\x12\x0ctensorboardX\"r\n\x13ResourceHandleProto\x12\x0e\n\x06\x64\x65vice\x18\x01 \x01(\t\x12\x11\n\tcontainer\x18\x02 \x01(\t\x12\x0c\n\x04name\x18\x03 \x01(\t\x12\x11\n\thash_code\x18\x04 \x01(\x04\x12\x17\n\x0fmaybe_type_name\x18\x05 \x01(\tB/\n\x18org.tensorflow.frameworkB\x0eResourceHandleP\x01\xf8\x01\x01\x62\x06proto3')

How to fix it?

constantinpape commented 5 years ago

Regarding the first error: scipy.imsave is deprecated and not included in more recent scipy versions anymore. We fixed this in #177, unfortunately this is not reflected in the conda-forge version yet. Will try to fix that soon.

The second error seems to be related to tensorboardX, however the traceback you have posted is missing the actual error message. If you post this and the version of tensorboardX in your environment (you can use conda list to find the version) I can try to tell you more.

LucaMarconato commented 5 years ago

This is the last missing line of the error, sorry:

TypeError: __new__() got an unexpected keyword argument 'serialized_options'

And this is from conda list:

tensorboard               1.9.0                    py36_0    conda-forge
tensorboardx              1.8                        py_0    conda-forge
tensorflow                1.9.0                    py36_0    conda-forge
constantinpape commented 5 years ago

I could not reproduce this on current master (also using tensorboardx 1.8). I have updated the conda-forge version, but the new version will only be online in a few hours, I will try with that at some point next week.

DerThorsten commented 5 years ago

@LucaMarconato can we close this or is this still an issue?

LucaMarconato commented 5 years ago

I will in the next days and give you feedback

LucaMarconato commented 5 years ago

I just tried again running the basic example on a fresh conda evnironment. The above error disappeared, but now I get a different error:

[+][2019-10-14 11:42:16.118985] [PROGRESS] Training iteration 0 (batch 0 of epoch 0).
Traceback (most recent call last):
  File "main.py", line 54, in <module>
    trainer.fit()
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/basic.py", line 1336, in fit
    self.train_for(break_callback=lambda *args: self.stop_fitting(max_num_iterations,
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/basic.py", line 1440, in train_for
    iteration_num=iteration_num)
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/callbacks/base.py", line 107, in call
    callback(**kwargs)
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/callbacks/base.py", line 163, in __call__
    getattr(self, kwargs.get('trigger'))(**kwargs)
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/callbacks/logging/tensorboard.py", line 246, in end_of_training_iteration
    allow_image_logging=log_images_now)
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/callbacks/logging/tensorboard.py", line 225, in log_object
    self.log_histogram(tag, values, self.trainer.iteration_count)
  File "/miniconda3/envs/to_del/lib/python3.7/site-packages/inferno/trainers/callbacks/logging/tensorboard.py", line 440, in log_histogram
    raise NotImplementedError
NotImplementedError

The versions number are different now:

output of conda list:

# packages in environment at /miniconda3/envs/to_del:
#
# Name                    Version                   Build  Channel
blas                      1.0                         mkl  
ca-certificates           2019.9.11            hecc5488_0    conda-forge
certifi                   2019.9.11                py37_0    conda-forge
cffi                      1.12.3           py37hccf1714_0    conda-forge
cloudpickle               1.2.2                      py_0    conda-forge
cycler                    0.10.0                     py_1    conda-forge
cytoolz                   0.10.0           py37h01d97ff_0    conda-forge
dask-core                 2.5.2                      py_0    conda-forge
decorator                 4.4.0                      py_0    conda-forge
dill                      0.3.1.1                  py37_0    conda-forge
freetype                  2.10.0               h24853df_1    conda-forge
h5py                      2.10.0          nompi_py37h6248fd5_100    conda-forge
hdf5                      1.10.5          nompi_h0cbb7df_1103    conda-forge
imageio                   2.6.1                    py37_0    conda-forge
inferno                   v0.4.0                     py_0    conda-forge
intel-openmp              2019.4                      233  
jpeg                      9c                h1de35cc_1001    conda-forge
kiwisolver                1.1.0            py37h770b8ee_0    conda-forge
libcxx                    4.0.1                hcfea43d_1  
libcxxabi                 4.0.1                hcfea43d_1  
libedit                   3.1.20181209         hb402a30_0  
libffi                    3.2.1                         1    bioconda
libgfortran               3.0.1                h93005f0_2  
libpng                    1.6.37               h2573ce8_0    conda-forge
libprotobuf               3.9.2                hd9629dc_0  
libtiff                   4.0.10            hd08fb8f_1003    conda-forge
lz4-c                     1.8.3             h6de7cb9_1001    conda-forge
matplotlib-base           3.1.1            py37h3a684a6_1    conda-forge
mkl                       2019.4                      233  
mkl-service               2.3.0            py37hfbe908c_0  
mkl_fft                   1.0.14           py37h5e564d8_0  
mkl_random                1.1.0            py37ha771720_0  
ncurses                   6.1                  h0a44026_1  
networkx                  2.3                        py_0    conda-forge
ninja                     1.9.0                h04f5b5a_0    conda-forge
numpy                     1.17.2           py37h99e6662_0  
numpy-base                1.17.2           py37h6575580_0  
olefile                   0.46                       py_0    conda-forge
openssl                   1.1.1c               h01d97ff_0    conda-forge
pillow                    6.2.0            py37hb68e598_0  
pip                       19.2.3                   py37_0  
protobuf                  3.9.2            py37h0a44026_0  
pycparser                 2.19                     py37_1    conda-forge
pyparsing                 2.4.2                      py_0    conda-forge
python                    3.7.4                h359304d_1  
python-dateutil           2.8.0                      py_0    conda-forge
pytorch                   1.1.0                   py3.7_0    pytorch
pywavelets                1.0.3            py37h1e5eb4f_1    conda-forge
pyyaml                    5.1.2            py37h01d97ff_0    conda-forge
readline                  7.0                  h1de35cc_5  
scikit-image              0.15.0           py37h86efe34_2    conda-forge
scipy                     1.3.1            py37h1410ff5_0  
setuptools                41.4.0                   py37_0  
six                       1.12.0                   py37_0  
sqlite                    3.30.0               ha441bb4_0  
tensorboardx              1.9                        py_0    conda-forge
tk                        8.6.8                ha441bb4_0  
toolz                     0.10.0                     py_0    conda-forge
torchvision               0.2.2                      py_3    pytorch
tornado                   6.0.3            py37h01d97ff_0    conda-forge
wheel                     0.33.6                   py37_0  
xz                        5.2.4                h1de35cc_4  
yaml                      0.1.7             h1de35cc_1001    conda-forge
zlib                      1.2.11               h1de35cc_3  
zstd                      1.4.0                ha9f0a20_0    conda-forge