search

MLBazaar / MLPrimitives

Primitives for machine learning and data science.
https://mlbazaar.github.io/MLPrimitives
MIT License
69 stars 38 forks source link

Inferring data shapes with single dimension for `keras` adapter #265

Closed sarahmish closed 2 years ago

sarahmish commented 2 years ago

Description

When having numpy arrays of single dimension, for example

X = np.array([0, 2, 1, 3, 5]) # shape (5, )

the keras adapter fails due to the _augment_hyperparameters function where we try to infer the shapes of the input and target variables. More specifically, the following lines can be edited

def _augment_hyperparameters(self, X, mode, kwargs):
    shape = np.asarray(X)[0].shape
    length = shape[0]
    self._setdefault(kwargs, '{}_shape'.format(mode), shape)
    self._setdefault(kwargs, '{}_dim'.format(mode), length)
    self._setdefault(kwargs, '{}_length'.format(mode), length)

    return kwargs

rather than assuming the data is at least contains two dimensions, i.e. shape = (m, n, ..) we can consume a shape of any size and cast it as uni-dimensional if we infer an empty tuple, more concretely the edited function would look like

def _augment_hyperparameters(self, X, mode, kwargs):
    shape = np.asarray(X)[0].shape
    if shape:
        length = shape[0]
    else:
        length = 1 # supporting shape (l, )

    self._setdefault(kwargs, '{}_shape'.format(mode), shape)
    self._setdefault(kwargs, '{}_dim'.format(mode), length)
    self._setdefault(kwargs, '{}_length'.format(mode), length)

    return kwargs

This issue was initially discovered in https://github.com/sintel-dev/GreenGuard/issues/59