Closed haritsahm closed 2 years ago
It turned out the return data from dataset.py was wrong. dataset.py
import os
import cv2
import pandas as pd
import torch.utils.data
class SearchDataset(torch.utils.data.Dataset):
def __init__(self, csv_path, image_dir, transform=None):
df = pd.read_csv(csv_path)
self.image_id = df['image'].values
self.labels = df['labels'].values
self.transform = transform
self.image_dir = image_dir
def __len__(self):
return len(self.labels)
def __getitem__(self, idx):
image_id = self.image_id[idx]
label = self.labels[idx]
image_path = os.path.join(self.image_dir, image_id)
image = cv2.imread(image_path)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
if self.transform is not None:
transformed = self.transform(image=image)
image = transformed["image"]
return image, label
I fixed it but another error popped up
env: HYDRA_FULL_ERROR=1
_version: 2
task: classification
policy_model:
task_factor: 0.1
gp_factor: 10
temperature: 0.05
num_sub_policies: 40
num_chunks: 4
operation_count: 4
operations:
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_r: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_g: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_b: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.RandomBrightness
- _target_: autoalbument.faster_autoaugment.models.policy_operations.RandomContrast
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Solarize
- _target_: autoalbument.faster_autoaugment.models.policy_operations.HorizontalFlip
- _target_: autoalbument.faster_autoaugment.models.policy_operations.VerticalFlip
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Rotate
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftX
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftY
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Scale
- _target_: autoalbument.faster_autoaugment.models.policy_operations.CutoutFixedNumberOfHoles
- _target_: autoalbument.faster_autoaugment.models.policy_operations.CutoutFixedSize
classification_model:
_target_: models.DenseNetClassificationModel
num_classes: _MISSING_
architecture: resnet18
pretrained: false
data:
dataset:
_target_: dataset.SearchDataset
csv_path: /content/Kaggle-Plant-Pathology-2021/Kaggle-Plant-Pathology-2021/train_encoded.csv
image_dir: /content/Kaggle-Plant-Pathology-2021/Kaggle-Plant-Pathology-2021/train_images/img_sz_512/
input_dtype: uint8
preprocessing:
- Resize:
height: 512
width: 512
normalization:
mean:
- 0.485
- 0.456
- 0.406
std:
- 0.229
- 0.224
- 0.225
dataloader:
_target_: torch.utils.data.DataLoader
batch_size: 16
shuffle: true
num_workers: 8
pin_memory: true
drop_last: true
searcher:
_target_: autoalbument.faster_autoaugment.search.FasterAutoAugmentSearcher
trainer:
_target_: pytorch_lightning.Trainer
gpus: 1
benchmark: false
max_epochs: 20
resume_from_checkpoint: null
optim:
main:
_target_: torch.optim.Adam
lr: 0.001
betas:
- 0
- 0.999
policy:
_target_: torch.optim.Adam
lr: 0.001
betas:
- 0
- 0.999
callbacks:
- _target_: autoalbument.callbacks.MonitorAverageParameterChange
- _target_: autoalbument.callbacks.SavePolicy
- _target_: pytorch_lightning.callbacks.ModelCheckpoint
save_last: true
dirpath: checkpoints
logger:
_target_: pytorch_lightning.loggers.TensorBoardLogger
save_dir: /content/Kaggle-Plant-Pathology-2021/Kaggle-Plant-Pathology-2021/outputs/2021-03-28/12-37-33/tensorboard_logs
seed: 42
Working directory: /content/Kaggle-Plant-Pathology-2021/Kaggle-Plant-Pathology-2021/outputs/2021-03-28/12-37-33
[2021-03-28 12:37:36,242][pytorch_lightning.utilities.seed][INFO] - Global seed set to 42
[2021-03-28 12:37:37,456][autoalbument.faster_autoaugment.datamodule][INFO] - Preprocessing transform:
Compose([
Resize(always_apply=False, p=1, height=512, width=512, interpolation=1),
Normalize(always_apply=False, p=1.0, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255),
ToTensorV2(always_apply=True, p=1.0, transpose_mask=True),
], p=1.0, bbox_params=None, keypoint_params=None, additional_targets={})
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/utilities/distributed.py:52: UserWarning: ModelCheckpoint(save_last=True, monitor=None) is a redundant configuration. You can save the last checkpoint with ModelCheckpoint(save_top_k=None, monitor=None).
warnings.warn(*args, **kwargs)
[2021-03-28 12:37:37,678][pytorch_lightning.utilities.distributed][INFO] - GPU available: True, used: True
[2021-03-28 12:37:37,679][pytorch_lightning.utilities.distributed][INFO] - TPU available: None, using: 0 TPU cores
[2021-03-28 12:37:38,028][pytorch_lightning.accelerators.gpu][INFO] - LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
2021-03-28 12:37:41.636444: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
[2021-03-28 12:37:42,830][pytorch_lightning.core.lightning][INFO] -
| Name | Type | Params
-------------------------------------------------------------
0 | main_model | DenseNetClassificationModel | 15.0 M
1 | policy_model | Policy | 6.4 K
-------------------------------------------------------------
15.0 M Trainable params
0 Non-trainable params
15.0 M Total params
59.841 Total estimated model params size (MB)
/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
cpuset_checked))
Epoch 0: 0% 0/1164 [00:04<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/bin/autoalbument-search", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/hydra/main.py", line 37, in decorated_main
strict=strict,
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 347, in _run_hydra
lambda: hydra.run(
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 350, in <lambda>
overrides=args.overrides,
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 112, in run
configure_logging=with_log_configuration,
File "/usr/local/lib/python3.7/dist-packages/hydra/core/utils.py", line 127, in run_job
ret.return_value = task_function(task_cfg)
File "/usr/local/lib/python3.7/dist-packages/autoalbument/cli/search.py", line 55, in main
searcher.search()
File "/usr/local/lib/python3.7/dist-packages/autoalbument/faster_autoaugment/search.py", line 65, in search
self.trainer.fit(self.model, datamodule=self.datamodule)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 499, in fit
self.dispatch()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 546, in dispatch
self.accelerator.start_training(self)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/accelerators/accelerator.py", line 73, in start_training
self.training_type_plugin.start_training(trainer)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 114, in start_training
self._results = trainer.run_train()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 637, in run_train
self.train_loop.run_training_epoch()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/training_loop.py", line 493, in run_training_epoch
batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/training_loop.py", line 659, in run_training_batch
split_batch, batch_idx, opt_idx, self.trainer.hiddens
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/training_loop.py", line 293, in training_step
training_step_output = self.trainer.accelerator.training_step(args)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/accelerators/accelerator.py", line 156, in training_step
return self.training_type_plugin.training_step(*args)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 125, in training_step
return self.lightning_module.training_step(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/autoalbument/faster_autoaugment/models/faa_model.py", line 86, in training_step
self.manual_backward(d_n_loss.unsqueeze(0), main_optimizer, -ones.unsqueeze(0))
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/core/lightning.py", line 1225, in manual_backward
self.trainer.train_loop.backward(loss, optimizer=None, opt_idx=None, *args, **kwargs)
TypeError: backward() got multiple values for argument 'optimizer'
Hey @haritsahm
Could you please provide the output of the pip freeze
command in your Python environment?
Hey @creafz,
Here is the output of pip freeze
. Oh i forgot to mention that I use this in Google Colab
/content
/content/Kaggle-Plant-Pathology-2021
absl-py==0.12.0
aiohttp==3.7.4.post0
alabaster==0.7.12
albumentations==0.5.2
altair==4.1.0
antlr4-python3-runtime==4.8
appdirs==1.4.4
argon2-cffi==20.1.0
astor==0.8.1
astropy==4.2
astunparse==1.6.3
async-generator==1.10
async-timeout==3.0.1
atari-py==0.2.6
atomicwrites==1.4.0
attrs==20.3.0
audioread==2.1.9
autoalbument==0.4.0
autograd==1.3
Babel==2.9.0
backcall==0.2.0
beautifulsoup4==4.6.3
bleach==3.3.0
blis==0.4.1
bokeh==2.3.0
Bottleneck==1.3.2
branca==0.4.2
bs4==0.0.1
CacheControl==0.12.6
cachetools==4.2.1
catalogue==1.0.0
certifi==2020.12.5
cffi==1.14.5
chainer==7.4.0
chardet==3.0.4
click==7.1.2
cloudpickle==1.3.0
cmake==3.12.0
cmdstanpy==0.9.5
colorama==0.4.4
colorcet==2.0.6
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.3.2
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.3
cupy-cuda101==7.4.0
cvxopt==1.2.6
cvxpy==1.0.31
cycler==0.10.0
cymem==2.0.5
Cython==0.29.22
daft==0.0.4
dask==2.12.0
datascience==0.10.6
debugpy==1.0.0
decorator==4.4.2
defusedxml==0.7.1
descartes==1.1.0
dill==0.3.3
distributed==1.25.3
dlib==19.18.0
dm-tree==0.1.5
docopt==0.6.2
docutils==0.16
dopamine-rl==1.0.5
earthengine-api==0.1.258
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
efficientnet-pytorch==0.6.3
en-core-web-sm==2.2.5
entrypoints==0.3
ephem==3.7.7.1
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==1.0.61
fastdtw==0.3.4
fastprogress==1.0.0
fastrlock==0.6
fbprophet==0.7.1
feather-format==0.4.1
filelock==3.0.12
firebase-admin==4.4.0
fix-yahoo-finance==0.0.22
Flask==1.1.2
flatbuffers==1.12
folium==0.8.3
fsspec==0.8.7
future==0.18.2
gast==0.3.3
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.50
geopy==1.17.0
gin-config==0.4.0
glob2==0.7
google==2.0.3
google-api-core==1.26.2
google-api-python-client==1.12.8
google-auth==1.28.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.3
google-cloud-bigquery==1.21.0
google-cloud-bigquery-storage==1.1.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.7.0
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.53.0
googledrivedownloader==0.4
graphviz==0.10.1
greenlet==1.0.0
grpcio==1.32.0
gspread==3.0.1
gspread-dataframe==3.0.8
gym==0.17.3
h5py==2.10.0
HeapDict==1.0.1
hijri-converter==2.1.1
holidays==0.10.5.2
holoviews==1.14.2
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.4
httplib2shim==0.0.3
humanize==0.5.1
hydra-core==1.0.6
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.10
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.4.0
importlib-metadata==3.8.1
importlib-resources==5.1.2
imutils==0.5.4
inflect==2.1.0
iniconfig==1.1.1
intel-openmp==2021.2.0
intervaltree==2.1.0
ipykernel==4.10.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.6.3
itsdangerous==1.1.0
jax==0.2.11
jaxlib==0.1.64+cuda110
jdcal==1.4.1
jedi==0.18.0
jieba==0.42.1
Jinja2==2.11.3
joblib==1.0.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.5
jupyter-console==5.2.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kaggle==1.5.12
kapre==0.1.3.1
Keras==2.4.3
Keras-Preprocessing==1.1.2
keras-vis==0.4.1
kiwisolver==1.3.1
knnimpute==0.1.0
korean-lunar-calendar==0.2.1
librosa==0.8.0
lightgbm==2.2.3
llvmlite==0.34.0
lmdb==0.99
LunarCalendar==0.0.9
lxml==4.2.6
Markdown==3.3.4
MarkupSafe==1.1.1
matplotlib==3.2.2
matplotlib-venn==0.11.6
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.7.0
moviepy==0.2.3.5
mpmath==1.2.1
msgpack==1.0.2
multidict==5.1.0
multiprocess==0.70.11.1
multitasking==0.0.9
munch==2.5.0
murmurhash==1.0.5
music21==5.5.0
natsort==5.5.0
nbclient==0.5.3
nbconvert==5.6.1
nbformat==5.1.2
nest-asyncio==1.5.1
networkx==2.5
nibabel==3.0.2
nltk==3.2.5
notebook==5.3.1
np-utils==0.5.12.1
numba==0.51.2
numexpr==2.7.3
numpy==1.19.5
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.0
okgrade==0.4.3
omegaconf==2.0.6
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
opencv-python-headless==4.5.1.48
openpyxl==2.5.9
opt-einsum==3.3.0
osqp==0.6.2.post0
packaging==20.9
palettable==3.3.0
pandas==1.1.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
pandocfilters==1.4.3
panel==0.11.1
param==1.10.1
parso==0.8.1
pathlib==1.0.1
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.1.2
pip-tools==4.5.1
plac==1.1.3
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
pooch==1.3.0
portpicker==1.3.1
prefetch-generator==1.0.1
preshed==3.0.5
pretrainedmodels==0.7.4
prettytable==2.1.0
progressbar2==3.38.0
prometheus-client==0.9.0
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.12.4
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.7.0
py==1.10.0
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.2
pycparser==2.20
pyct==0.4.8
pydata-google-auth==1.1.0
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyerfa==1.7.2
pyglet==1.5.0
Pygments==2.6.1
pygobject==3.26.1
pymc3==3.7
PyMeeus==0.5.11
pymongo==3.11.3
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrsistent==0.17.3
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==0.0.0
python-chess==0.23.11
python-dateutil==2.8.1
python-louvain==0.15
python-slugify==4.0.1
python-utils==2.5.6
pytorch-lightning==1.2.6
pytz==2018.9
pyviz-comms==2.0.1
PyWavelets==1.1.1
PyYAML==5.1.2
pyzmq==22.0.3
qdldl==0.1.5.post0
qtconsole==5.0.3
QtPy==1.9.0
regex==2019.12.20
requests==2.23.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==3.4.3
rsa==4.7.2
ruamel.yaml==0.17.2
ruamel.yaml.clib==0.2.2
scikit-image==0.16.2
scikit-learn==0.22.2.post1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.2
seaborn==0.11.1
segmentation-models-pytorch==0.1.3
Send2Trash==1.5.0
setuptools-git==1.2
Shapely==1.7.1
simplegeneric==0.8.1
six==1.15.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==4.2.0
snowballstemmer==2.1.0
sortedcontainers==2.3.0
SoundFile==0.10.3.post1
spacy==2.2.4
Sphinx==1.8.5
sphinxcontrib-serializinghtml==1.1.4
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.4.3
sqlparse==0.4.1
srsly==1.0.5
statsmodels==0.10.2
sympy==1.7.1
tables==3.4.4
tabulate==0.8.9
tblib==1.7.0
tensorboard==2.4.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.4.1
tensorflow-datasets==4.0.1
tensorflow-estimator==2.4.0
tensorflow-gcs-config==2.4.0
tensorflow-hub==0.11.0
tensorflow-metadata==0.29.0
tensorflow-probability==0.12.1
termcolor==1.1.0
terminado==0.9.3
testpath==0.4.4
text-unidecode==1.3
textblob==0.15.3
textgenrnn==1.4.1
Theano==1.0.5
thinc==7.4.0
tifffile==2021.3.17
timm==0.3.2
toml==0.10.2
toolz==0.11.1
torch==1.8.1+cu101
torch-summary==1.4.5
torchmetrics==0.2.0
torchsummary==1.5.1
torchtext==0.9.1
torchvision==0.9.1+cu101
tornado==5.1.1
tqdm==4.41.1
traitlets==5.0.5
tweepy==3.10.0
typeguard==2.7.1
typing-extensions==3.7.4.3
tzlocal==1.5.1
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.9.0
wasabi==0.8.2
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.12.1
xarray==0.15.1
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yarl==1.6.3
yellowbrick==0.9.1
zict==2.0.0
zipp==3.4.1
Could you please try to downgrade PyTorch-Lightning to version 1.1.8 and rerun the code? The current version of AutoAlbument requires pytorch-lightning>=1.1.8,<1.2
.
Okay so I was able to run the autoalbument-search but then the execution is finished without training or other processes. No errors, no warnings, just stop at the end CUDA Device.
env: HYDRA_FULL_ERROR=1
_version: 2
task: classification
policy_model:
task_factor: 0.1
gp_factor: 10
temperature: 0.05
num_sub_policies: 40
num_chunks: 8
operation_count: 6
operations:
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_r: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_g: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftRGB
shift_b: true
- _target_: autoalbument.faster_autoaugment.models.policy_operations.RandomBrightness
- _target_: autoalbument.faster_autoaugment.models.policy_operations.RandomContrast
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Solarize
- _target_: autoalbument.faster_autoaugment.models.policy_operations.HorizontalFlip
- _target_: autoalbument.faster_autoaugment.models.policy_operations.VerticalFlip
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Rotate
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftX
- _target_: autoalbument.faster_autoaugment.models.policy_operations.ShiftY
- _target_: autoalbument.faster_autoaugment.models.policy_operations.Scale
- _target_: autoalbument.faster_autoaugment.models.policy_operations.CutoutFixedNumberOfHoles
- _target_: autoalbument.faster_autoaugment.models.policy_operations.CutoutFixedSize
classification_model:
_target_: DenseNet.DenseNetClassificationModel
num_classes: _MISSING_
architecture: resnet18
pretrained: false
data:
dataset:
_target_: dataset.SearchDataset
csv_path: /content/Kaggle-Plant-Pathology-2021/train_simplified.csv
image_dir: /content/train_images/img_sz_256
input_dtype: uint8
preprocessing:
- Resize:
height: 224
width: 224
normalization:
mean:
- 0.485
- 0.456
- 0.406
std:
- 0.229
- 0.224
- 0.225
dataloader:
_target_: torch.utils.data.DataLoader
batch_size: 24
shuffle: true
num_workers: 4
pin_memory: true
drop_last: true
searcher:
_target_: autoalbument.faster_autoaugment.search.FasterAutoAugmentSearcher
trainer:
_target_: pytorch_lightning.Trainer
gpus: 1
benchmark: true
max_epochs: 40
resume_from_checkpoint: null
optim:
main:
_target_: torch.optim.Adam
lr: 0.001
betas:
- 0
- 0.999
policy:
_target_: torch.optim.Adam
lr: 0.001
betas:
- 0
- 0.999
callbacks:
- _target_: autoalbument.callbacks.MonitorAverageParameterChange
- _target_: autoalbument.callbacks.SavePolicy
- _target_: pytorch_lightning.callbacks.ModelCheckpoint
save_last: true
dirpath: checkpoints
logger:
_target_: pytorch_lightning.loggers.TensorBoardLogger
save_dir: /content/Kaggle-Plant-Pathology-2021/outputs/2021-04-09/09-25-42/tensorboard_logs
seed: 42
Working directory: /content/Kaggle-Plant-Pathology-2021/outputs/2021-04-09/09-25-42
[2021-04-09 09:25:45,004][pytorch_lightning.utilities.seed][INFO] - Global seed set to 42
Deleting features.conv0.weight
Deleting features.norm0.weight
Deleting features.norm0.bias
Deleting features.norm0.running_mean
Deleting features.norm0.running_var
Deleting features.transition1.norm.weight
Deleting features.transition1.norm.bias
Deleting features.transition1.norm.running_mean
Deleting features.transition1.norm.running_var
Deleting features.transition1.conv.weight
Deleting features.transition2.norm.weight
Deleting features.transition2.norm.bias
Deleting features.transition2.norm.running_mean
Deleting features.transition2.norm.running_var
Deleting features.transition2.conv.weight
Deleting features.transition3.norm.weight
Deleting features.transition3.norm.bias
Deleting features.transition3.norm.running_mean
Deleting features.transition3.norm.running_var
Deleting features.transition3.conv.weight
Deleting features.norm5.weight
Deleting features.norm5.bias
Deleting features.norm5.running_mean
Deleting features.norm5.running_var
Deleting classifier.weight
Deleting classifier.bias
[2021-04-09 09:25:46,412][autoalbument.faster_autoaugment.datamodule][INFO] - Preprocessing transform:
Compose([
Resize(always_apply=False, p=1, height=224, width=224, interpolation=1),
Normalize(always_apply=False, p=1.0, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255),
ToTensorV2(always_apply=True, p=1.0, transpose_mask=True),
], p=1.0, bbox_params=None, keypoint_params=None, additional_targets={})
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/utilities/distributed.py:50: UserWarning: ModelCheckpoint(save_last=True, monitor=None) is a redundant configuration. You can save the last checkpoint with ModelCheckpoint(save_top_k=None, monitor=None).
warnings.warn(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/utilities/distributed.py:50: UserWarning: You have set progress_bar_refresh_rate < 20 on Google Colab. This may crash. Consider using progress_bar_refresh_rate >= 20 in Trainer.
warnings.warn(*args, **kwargs)
[2021-04-09 09:25:46,626][pytorch_lightning.utilities.distributed][INFO] - GPU available: True, used: True
[2021-04-09 09:25:46,626][pytorch_lightning.utilities.distributed][INFO] - TPU available: None, using: 0 TPU cores
[2021-04-09 09:25:46,626][pytorch_lightning.accelerators.accelerator_connector][INFO] - LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Here is the output of my pip freeze
absl-py==0.12.0
aiohttp==3.7.4.post0
alabaster==0.7.12
albumentations==0.5.2
altair==4.1.0
antlr4-python3-runtime==4.8
appdirs==1.4.4
argon2-cffi==20.1.0
astor==0.8.1
astropy==4.2
astunparse==1.6.3
async-generator==1.10
async-timeout==3.0.1
atari-py==0.2.6
atomicwrites==1.4.0
attrs==20.3.0
audioread==2.1.9
autoalbument==0.4.0
autograd==1.3
Babel==2.9.0
backcall==0.2.0
beautifulsoup4==4.6.3
bleach==3.3.0
blis==0.4.1
bokeh==2.3.0
Bottleneck==1.3.2
branca==0.4.2
bs4==0.0.1
CacheControl==0.12.6
cachetools==4.2.1
catalogue==1.0.0
certifi==2020.12.5
cffi==1.14.5
chainer==7.4.0
chardet==3.0.4
click==7.1.2
cloudpickle==1.3.0
cmake==3.12.0
cmdstanpy==0.9.5
colorama==0.4.4
colorcet==2.0.6
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.3.2
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.3
cupy-cuda101==7.4.0
cvxopt==1.2.6
cvxpy==1.0.31
cycler==0.10.0
cymem==2.0.5
Cython==0.29.22
daft==0.0.4
dask==2.12.0
datascience==0.10.6
debugpy==1.0.0
decorator==4.4.2
defusedxml==0.7.1
descartes==1.1.0
dill==0.3.3
distributed==1.25.3
dlib==19.18.0
dm-tree==0.1.5
docopt==0.6.2
docutils==0.16
dopamine-rl==1.0.5
earthengine-api==0.1.258
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
efficientnet-pytorch==0.6.3
en-core-web-sm==2.2.5
entrypoints==0.3
ephem==3.7.7.1
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==1.0.61
fastdtw==0.3.4
fastprogress==1.0.0
fastrlock==0.6
fbprophet==0.7.1
feather-format==0.4.1
filelock==3.0.12
firebase-admin==4.4.0
fix-yahoo-finance==0.0.22
Flask==1.1.2
flatbuffers==1.12
folium==0.8.3
fsspec==0.9.0
future==0.18.2
gast==0.3.3
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.50
geopy==1.17.0
gin-config==0.4.0
glob2==0.7
google==2.0.3
google-api-core==1.26.2
google-api-python-client==1.12.8
google-auth==1.28.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.3
google-cloud-bigquery==1.21.0
google-cloud-bigquery-storage==1.1.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.7.0
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.53.0
googledrivedownloader==0.4
graphviz==0.10.1
greenlet==1.0.0
grpcio==1.32.0
gspread==3.0.1
gspread-dataframe==3.0.8
gym==0.17.3
h5py==2.10.0
HeapDict==1.0.1
hijri-converter==2.1.1
holidays==0.10.5.2
holoviews==1.14.2
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.4
httplib2shim==0.0.3
humanize==0.5.1
hydra-core==1.0.6
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.10
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.4.0
importlib-metadata==3.8.1
importlib-resources==5.1.2
imutils==0.5.4
inflect==2.1.0
iniconfig==1.1.1
intel-openmp==2021.2.0
intervaltree==2.1.0
ipykernel==4.10.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.6.3
itsdangerous==1.1.0
jax==0.2.11
jaxlib==0.1.64+cuda110
jdcal==1.4.1
jedi==0.18.0
jieba==0.42.1
Jinja2==2.11.3
joblib==1.0.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.5
jupyter-console==5.2.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kaggle==1.5.12
kapre==0.1.3.1
Keras==2.4.3
Keras-Preprocessing==1.1.2
keras-vis==0.4.1
kiwisolver==1.3.1
knnimpute==0.1.0
korean-lunar-calendar==0.2.1
librosa==0.8.0
lightgbm==2.2.3
llvmlite==0.34.0
lmdb==0.99
LunarCalendar==0.0.9
lxml==4.2.6
Markdown==3.3.4
MarkupSafe==1.1.1
matplotlib==3.2.2
matplotlib-venn==0.11.6
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.7.0
moviepy==0.2.3.5
mpmath==1.2.1
msgpack==1.0.2
multidict==5.1.0
multiprocess==0.70.11.1
multitasking==0.0.9
munch==2.5.0
murmurhash==1.0.5
music21==5.5.0
natsort==5.5.0
nbclient==0.5.3
nbconvert==5.6.1
nbformat==5.1.2
nest-asyncio==1.5.1
networkx==2.5
nibabel==3.0.2
nltk==3.2.5
notebook==5.3.1
np-utils==0.5.12.1
numba==0.51.2
numexpr==2.7.3
numpy==1.19.5
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.0
okgrade==0.4.3
omegaconf==2.0.6
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
opencv-python-headless==4.5.1.48
openpyxl==2.5.9
opt-einsum==3.3.0
osqp==0.6.2.post0
packaging==20.9
palettable==3.3.0
pandas==1.1.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
pandocfilters==1.4.3
panel==0.11.1
param==1.10.1
parso==0.8.2
pathlib==1.0.1
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.1.2
pip-tools==4.5.1
plac==1.1.3
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
pooch==1.3.0
portpicker==1.3.1
prefetch-generator==1.0.1
preshed==3.0.5
pretrainedmodels==0.7.4
prettytable==2.1.0
progressbar2==3.38.0
prometheus-client==0.10.0
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.12.4
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.7.0
py==1.10.0
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.2
pycparser==2.20
pyct==0.4.8
pydata-google-auth==1.1.0
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyerfa==1.7.2
pyglet==1.5.0
Pygments==2.6.1
pygobject==3.26.1
pymc3==3.7
PyMeeus==0.5.11
pymongo==3.11.3
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrsistent==0.17.3
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==0.0.0
python-chess==0.23.11
python-dateutil==2.8.1
python-louvain==0.15
python-slugify==4.0.1
python-utils==2.5.6
pytorch-lightning==1.1.8
pytz==2018.9
pyviz-comms==2.0.1
PyWavelets==1.1.1
PyYAML==5.1.2
pyzmq==22.0.3
qdldl==0.1.5.post0
qtconsole==5.0.3
QtPy==1.9.0
regex==2019.12.20
requests==2.23.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==3.4.3
rsa==4.7.2
ruamel.yaml==0.17.4
ruamel.yaml.clib==0.2.2
scikit-image==0.16.2
scikit-learn==0.22.2.post1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.2
seaborn==0.11.1
segmentation-models-pytorch==0.1.3
Send2Trash==1.5.0
setuptools-git==1.2
Shapely==1.7.1
simplegeneric==0.8.1
six==1.15.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==4.2.0
snowballstemmer==2.1.0
sortedcontainers==2.3.0
SoundFile==0.10.3.post1
spacy==2.2.4
Sphinx==1.8.5
sphinxcontrib-serializinghtml==1.1.4
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.4.3
sqlparse==0.4.1
srsly==1.0.5
statsmodels==0.10.2
sympy==1.7.1
tables==3.4.4
tabulate==0.8.9
tblib==1.7.0
tensorboard==2.4.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.4.1
tensorflow-datasets==4.0.1
tensorflow-estimator==2.4.0
tensorflow-gcs-config==2.4.0
tensorflow-hub==0.11.0
tensorflow-metadata==0.29.0
tensorflow-probability==0.12.1
termcolor==1.1.0
terminado==0.9.3
testpath==0.4.4
text-unidecode==1.3
textblob==0.15.3
textgenrnn==1.4.1
Theano==1.0.5
thinc==7.4.0
tifffile==2021.3.31
timm==0.3.2
toml==0.10.2
toolz==0.11.1
torch==1.8.1+cu101
torchsummary==1.5.1
torchtext==0.8.0
torchvision==0.9.1+cu101
tornado==5.1.1
tqdm==4.41.1
traitlets==5.0.5
tweepy==3.10.0
typeguard==2.7.1
typing-extensions==3.7.4.3
tzlocal==1.5.1
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.9.0
wasabi==0.8.2
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.12.1
xarray==0.15.1
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yarl==1.6.3
yellowbrick==0.9.1
zict==2.0.0
zipp==3.4.1
Hey guys! I got this error pop up after run
!autoalbument-search --config-dir .
with my custom model and dataset When i tried the CIFAR10 example, everything went well.Can you guys show me what's wrong? Thank you
densenet.py
dataset.py
Error output