meysam-madadi
commented
7 years ago Hi,
For anybody who has the same problem. I modified ArrayIterator init function as follows and it is able to receive a list of lshapes.
def __init__(self, X, y=None, nclass=None, lshape=None, make_onehot=True, name=None):
"""
During initialization, the input data will be converted to backend tensor objects
(e.g. CPUTensor or GPUTensor). If the backend uses the GPU, the data is copied over to the
device.
Args:
X (ndarray, shape: [# examples, feature size]): Input features of the
dataset.
y (ndarray, shape:[# examples, 1 or feature size], optional): Labels corresponding to
the input features. If absent, the input features themselves will be returned as
target values (e.g. autoencoder)
nclass (int, optional): The number of classes in labels. Not necessary if
labels are not provided or where the labels are non-categorical.
lshape (tuple, optional): Local shape for the input features
(e.g. # channels, height, width)
make_onehot (bool, optional): True if y is a categorical label that has to be converted
to a one hot representation.
"""
# Treat singletons like list so that iteration follows same syntax
super(ArrayIterator, self).__init__(name=name)
X = X if isinstance(X, list) else [X]
self.ndata = len(X[0])
assert self.ndata >= self.be.bsz
self.start = 0
self.nclass = nclass
self.ybuf = None
if make_onehot and nclass is None and y is not None:
raise AttributeError('Must provide number of classes when creating onehot labels')
# if labels provided, they must have same # examples as the features
if y is not None:
assert all([y.shape[0] == x.shape[0] for x in X]), \
"Input features and labels must have equal number of examples."
# for classifiction, the labels must be from 0 .. K-1, where K=nclass
if make_onehot:
assert y.max() <= nclass - 1 and y.min() >= 0, \
"Labels must range from 0 to {} (nclass-1).".format(nclass - 1)
assert (np.floor(y) == y).all(), \
"Labels must only contain integers."
# if local shape is provided, then the product of lshape should match the
# number of features
if lshape is not None:
lshape = lshape if isinstance(lshape, list) else [lshape]
assert len(lshape)==len(X)
assert all([x.shape[1] == np.prod(s) for (x,s) in zip(X,lshape)]), \
"product of lshape {} does not match input feature size".format(lshape)
# store shape of the input data
self.shape = [x.shape[1] if lshape is None else s for s in lshape]
if len(self.shape) == 1:
self.shape = self.shape[0]
self.lshape = lshape
Hi, I have a network with multiple channels and thus multiple inputs. Shapes of inputs are different. I know that input data can be a list of numpy arrays, but lshape attribute must be a tuple. Is it possible to introduce different lshapes to ArrayIterator? What about HDF5Iterator? Is it possible multiple inputs with different shapes using HDF5Iterator? Thx a lot.