Using capturemock with keras

[arseniymerkulov]

I want to capturemock keras fit function to receive fitted model results without keras installed. Following code causes error:

from capturemock import capturemock, RECORD, REPLAY

@capturemock("keras.models.Sequential.fit", mode=RECORD)
def test_training():
    from tensorflow.keras.models import Sequential
    from tensorflow.keras.layers import Dense
    from tensorflow.keras.datasets import mnist
    from tensorflow.keras import utils
    import tensorflow as tf

    num_classes = 10
    (x_train, y_train), (x_test, y_test) = mnist.load_data()

    x_train = x_train.reshape(x_train.shape[0], -1)
    x_train = x_train.astype('float32') / 255.
    y_train = utils.to_categorical(y_train, num_classes)

    model = Sequential()
    model.add(Dense(800, input_dim=x_train.shape[1], activation='relu'))
    model.add(Dense(100, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

    res = model.fit(x_train,
                    y_train,
                    batch_size=128,
                    epochs=15,
                    verbose=1)

    assert ['loss', 'accuracy'] == list(res.history.keys())
    assert [1, 15, 469] == list(res.params.values())

if __name__ == '__main__':
    test_training()

Traceback (most recent call last):
  File "E:\wok\capturemocking\test_keras.py", line 44, in <module>
    test_training()
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\__init__.py", line 618, in wrapped_func
    result = func(*funcargs, **funckw)
  File "E:\wok\capturemocking\test_keras.py", line 36, in test_training
    res = model_fit()
  File "E:\wok\capturemocking\test_keras.py", line 24, in model_fit
    res = model.fit(x_train,
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\pythonclient.py", line 160, in __call__
    return self.captureMockTrafficHandler.callFunction(self.captureMockProxyName,
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\pythontraffic.py", line 653, in callFunction
    return self.callRealFunction(traffic, captureMockFunction, captureMockProxy)
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\pythontraffic.py", line 658, in callRealFunction
    realRet = self.callStackChecker.callNoInterception(captureMockProxy.captureMockCallback, 
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\capturepython.py", line 26, in callNoInterception
    return method(*args, **kw)
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\capturemock\pythontraffic.py", line 432, in callRealFunction
    return captureMockFunction(*realArgs, **realKw)
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\Arsen\anaconda3\envs\capturemocking\lib\site-packages\keras\engine\training.py", line 1466, in fit
    self._assert_compile_was_called()
AttributeError: 'numpy.ndarray' object has no attribute '_assert_compile_was_called'

For keras it seems like model.compile function didnt executed. I think its because capturemock wrapping entity for model.fit didnt have compiled properties from keras. Can i fix this? For example, can i record only result of all model preparation methods jointly with creating only one mocking entity?

[gjb1002]

I think you may be pushing the boundaries of what Capturemock is capable of here :) I have not encountered compiled properties before. Potentially support for them could be added, but I doubt I'll have time to look at that any time soon...

Your analysis of the problem seems correct, in any case.

I'm a little unsure what you're trying to achieve by capturemocking "model.fit" though. Isn't that the main function you're trying to test here? The point of capturemock is normally to capture and fix something outside of your control, in order to have stable input to something you do control.

[arseniymerkulov]

No, i am trying to test model behavior without model fitting and keras installed. Thank you for your answer