ValueError on Training the Leaf Counter tutorial

[Mahi-Mai]

Hello! I'm excited to start using this tool, and I'm in the middle of setting up an environment for it on a cloud based platform. To make sure everything is working as intended, I'm working through the leaf counting tutorial here.

I would say the biggest difference between my code and the tutorial is this line:

model.load_ippn_leaf_count_dataset_from_directory('/repos/deepplantphenomics/deepplantphenomics/test_data/test_Ara2013_Canon')

I'm wondering if the dataset here is somehow different from when the tutorial was first written?

Everything executes fine, but once I get to model.begin_training() I get a value error. Here's my code:

import sys
sys.path.append('/repos/plantcv')
sys.path.append('/repos/deepplantphenomics')

from plantcv import plantcv as pcv
import cv2

import deepplantphenomics as dpp

model = dpp.DPPModel(debug=True, save_checkpoints=False, tensorboard_dir='/mnt/Setup and Troubleshooting/tensorlogs', report_rate=20)
# 3 channels for colour, 1 channel for greyscale
channels = 3

# Setup and hyperparameters
model.set_batch_size(4)
model.set_number_of_threads(8)
model.set_image_dimensions(128, 128, channels)
model.set_resize_images(True)

model.set_problem_type('regression')
model.set_num_regression_outputs(1)
model.set_train_test_split(0.8)
model.set_learning_rate(0.0001)
model.set_weight_initializer('xavier')
model.set_maximum_training_epochs(500)

# Augmentation options
model.set_augmentation_brightness_and_contrast(True)
model.set_augmentation_flip_horizontal(True)
model.set_augmentation_flip_vertical(True)
model.set_augmentation_crop(True)

# Load all data for IPPN leaf counting dataset
model.load_ippn_leaf_count_dataset_from_directory('/repos/deepplantphenomics/deepplantphenomics/test_data/test_Ara2013_Canon')

# Define a model architecture
model.add_input_layer()

model.add_convolutional_layer(filter_dimension=[5, 5, channels, 32], stride_length=1, activation_function='tanh')
model.add_pooling_layer(kernel_size=3, stride_length=2)

model.add_convolutional_layer(filter_dimension=[5, 5, 32, 64], stride_length=1, activation_function='tanh')
model.add_pooling_layer(kernel_size=3, stride_length=2)

model.add_convolutional_layer(filter_dimension=[3, 3, 64, 64], stride_length=1, activation_function='tanh')
model.add_pooling_layer(kernel_size=3, stride_length=2)

model.add_convolutional_layer(filter_dimension=[3, 3, 64, 64], stride_length=1, activation_function='tanh')
model.add_pooling_layer(kernel_size=3, stride_length=2)

model.add_output_layer()

# Begin training the regression model
model.begin_training()

I'm executing it in a Jupyter Notebook. Everything executes fine until I get to that final line, which throws me this error:

03:27PM: Parsing dataset...

InvalidArgumentError                      Traceback (most recent call last)
/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1575   try:
-> 1576     c_op = c_api.TF_FinishOperation(op_desc)
   1577   except errors.InvalidArgumentError as e:

InvalidArgumentError: Dimensions must be equal, but are 8 and 0 for 'DynamicPartition' (op: 'DynamicPartition') with input shapes: [8], [0].

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-9-7fb97535fd52> in <module>()
      1 # Begin training the regression model
----> 2 model.begin_training()

/repos/deepplantphenomics/deepplantphenomics/deepplantpheno.py in begin_training(self, return_test_loss)
    896 
    897         with self.__graph.as_default():
--> 898             self.__assemble_graph()
    899 
    900             # Either load the network parameters from a checkpoint file or start training

/repos/deepplantphenomics/deepplantphenomics/deepplantpheno.py in __assemble_graph(self)
    557                                            self.__validation_split, self.__all_moderation_features,
    558                                            self.__training_augmentation_images, self.__training_augmentation_labels,
--> 559                                            self.__split_labels)
    560 
    561                 # parse the images and set the appropriate environment variables

/repos/deepplantphenomics/deepplantphenomics/loaders.py in split_raw_data(images, labels, test_ratio, validation_ratio, moderation_features, augmentation_images, augmentation_labels, split_labels)
     41     # create partitions, we set train/validation to None if they're not being used
     42     if test_ratio != 0 and validation_ratio != 0:
---> 43         train_images, test_images, val_images = tf.dynamic_partition(images, mask, 3)
     44         train_labels, test_labels, val_labels = tf.dynamic_partition(labels, mask, 3)
     45     elif test_ratio != 0 and validation_ratio == 0:

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/gen_data_flow_ops.py in dynamic_partition(data, partitions, num_partitions, name)
    607     _, _, _op = _op_def_lib._apply_op_helper(
    608         "DynamicPartition", data=data, partitions=partitions,
--> 609         num_partitions=num_partitions, name=name)
    610     _result = _op.outputs[:]
    611     _inputs_flat = _op.inputs

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    785         op = g.create_op(op_type_name, inputs, output_types, name=scope,
    786                          input_types=input_types, attrs=attr_protos,
--> 787                          op_def=op_def)
    788       return output_structure, op_def.is_stateful, op
    789 

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    452                 'in a future version' if date is None else ('after %s' % date),
    453                 instructions)
--> 454       return func(*args, **kwargs)
    455     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    456                                        _add_deprecated_arg_notice_to_docstring(

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in create_op(***failed resolving arguments***)
   3153           input_types=input_types,
   3154           original_op=self._default_original_op,
-> 3155           op_def=op_def)
   3156       self._create_op_helper(ret, compute_device=compute_device)
   3157     return ret

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in __init__(self, node_def, g, inputs, output_types, control_inputs, input_types, original_op, op_def)
   1729           op_def, inputs, node_def.attr)
   1730       self._c_op = _create_c_op(self._graph, node_def, grouped_inputs,
-> 1731                                 control_input_ops)
   1732 
   1733     # Initialize self._outputs.

/usr/local/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1577   except errors.InvalidArgumentError as e:
   1578     # Convert to ValueError for backwards compatibility.
-> 1579     raise ValueError(str(e))
   1580 
   1581   return c_op

ValueError: Dimensions must be equal, but are 8 and 0 for 'DynamicPartition' (op: 'DynamicPartition') with input shapes: [8], [0].

Any help would be appreciated. Thanks for putting all this together!

[jubbens]

@Mahi-Mai thanks for bringing this up! I think it looks like the labels are possibly not being loaded. If you use the ippn leaf count dataset loader, it is by default looking for a file called Leaf_counts.csv in the dataset folder which should have one image name, ground truth leaf count pair per line. Can you confirm the labels file is there?

[Mahi-Mai]

Thanks!

It's the file seen here: https://github.com/p2irc/deepplantphenomics/blob/master/deepplantphenomics/test_data/test_Ara2013_Canon/Leaf_counts.csv

My notebook is referencing the repo directly. I was able to import the library fine, so I don't see why I shouldn't be able to read this from my Notebook...

[Mahi-Mai]

Here are the results of pip freeze, if you're curious:

absl-py==0.4.0
aiohttp==1.0.5
alabaster==0.7.9
anaconda-clean==1.0
anaconda-client==1.5.1
anaconda-navigator==1.3.1
argcomplete==1.0.0
astor==0.7.1
astroid==1.4.7
astropy==1.2.1
async-timeout==1.0.0
Babel==2.3.4
backcall==0.1.0
backports.shutil-get-terminal-size==1.0.0
beautifulsoup4==4.5.1
bitarray==0.8.1
blaze==0.10.1
bleach==2.1.3
bokeh==0.12.2
boto==2.42.0
Bottleneck==1.1.0
cairocffi==0.7.2
cffi==1.7.0
chardet==2.3.0
chest==0.2.3
click==6.6
cloudpickle==0.2.1
clyent==1.2.2
colorama==0.3.7
conda==4.2.9
conda-build==2.0.4
configobj==5.0.6
contextlib2==0.5.3
cryptography==1.5
cycler==0.10.0
Cython==0.24.1
cytoolz==0.8.0
dask==0.11.0
datashape==0.5.2
decorator==4.3.0
dill==0.2.5
docutils==0.12
dynd==0.7.3.dev1
entrypoints==0.2.3
et-xmlfile==1.0.1
fastcache==1.0.2
filelock==2.0.6
Flask==0.11.1
Flask-Cors==2.1.2
gast==0.2.0
gevent==1.1.2
greenlet==0.4.10
grpcio==1.14.1
h5py==2.6.0
HeapDict==1.0.0
html5lib==1.0.1
idna==2.1
imagesize==0.7.1
ipykernel==4.8.2
ipython==6.3.1
ipython-genutils==0.2.0
ipywidgets==5.2.2
itsdangerous==0.24
jdcal==1.2
jedi==0.12.0
Jinja2==2.10
joblib==0.12.2
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.3
jupyter-console==5.0.0
jupyter-core==4.4.0
jupyterlab==0.32.0
jupyterlab-launcher==0.10.5
lazy-object-proxy==1.2.1
llvmlite==0.13.0
locket==0.2.0
lxml==3.6.4
Markdown==2.6.11
MarkupSafe==1.0
matplotlib==1.5.3
mistune==0.8.3
mpmath==0.19
multidict==2.1.2
multipledispatch==0.4.8
nb-anacondacloud==1.2.0
nb-conda==2.0.0
nb-conda-kernels==2.0.0
nbconvert==5.3.1
nbformat==4.4.0
nbpresent==3.0.2
networkx==1.11
nltk==3.2.1
nose==1.3.7
notebook==5.4.1
numba==0.28.1
numexpr==2.6.1
numpy==1.14.5
odo==0.5.0
opencv-python==3.2.0.6
openpyxl==2.3.2
pandas==0.23.4
pandocfilters==1.4.2
parso==0.2.0
partd==0.3.6
path.py==0.0.0
pathlib2==2.1.0
patsy==0.4.1
pep8==1.7.0
pexpect==4.5.0
pickleshare==0.7.4
Pillow==3.3.1
pkginfo==1.3.2
ply==3.9
prompt-toolkit==1.0.15
protobuf==3.6.1
psutil==4.3.1
psycopg2==2.6.2
ptyprocess==0.5.2
py==1.4.31
pyasn1==0.1.9
pycosat==0.6.1
pycparser==2.14
pycrypto==2.6.1
pycurl==7.43.0
pyflakes==1.3.0
Pygments==2.2.0
pylint==1.5.4
pymssql==2.1.3
pyOpenSSL==16.0.0
pyparsing==2.1.4
pytest==2.9.2
python-dateutil==2.7.2
pytz==2016.6.1
PyYAML==3.12
pyzmq==17.0.0
QtAwesome==0.3.3
qtconsole==4.2.1
QtPy==1.1.2
redis==2.10.5
requests==2.11.1
rope-py3k==0.9.4.post1
rpy2==2.8.3
ruamel-yaml===-VERSION
scikit-image==0.12.3
scikit-learn==0.18
scipy==0.19.1
seaborn==0.9.0
Send2Trash==1.5.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
snowballstemmer==1.2.1
sockjs-tornado==1.0.3
Sphinx==1.4.6
sphinx-rtd-theme==0.1.9
spyder==3.0.0
SQLAlchemy==1.0.13
statsmodels==0.8.0
sympy==1.0
tables==3.2.3.1
tensorboard==1.10.0
tensorflow==1.10.0
termcolor==1.1.0
terminado==0.8.1
testpath==0.3.1
toolz==0.8.0
tornado==5.0.2
traitlets==4.3.2
unicodecsv==0.14.1
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.11.11
widgetsnbextension==1.2.6
wrapt==1.10.6
xlrd==1.0.0
XlsxWriter==0.9.3
xlwt==1.1.2

[Mahi-Mai]

I copied the test_data directory directly into mine and then edited the labels to match the filenames. (They didn't match.)

I redirected the model.load: model.load_ippn_leaf_count_dataset_from_directory('./test_data/test_Ara2013_Canon')

When that didn't work I tried this:

model.load_multiple_labels_from_csv('./test_data/test_Ara2013_Canon/Leaf_counts.csv', id_column=0)
model.load_images_with_ids_from_directory('./test_data/test_Ara2013_Canon')

Each results in:

08:24PM: Total raw examples is 8 08:24PM: Parsing dataset...

But in the end model.begin_training() still fails with the same error as before.

[jubbens]

cc @nhiggs

[nicohiggs]

@Mahi-Mai thanks for pointing this out to us. The latest version of master should be working correctly for you now.

Let us know of any further issues.