[Question] Can we export code with autokeras that does not depend on the library?

[neel04]

After training the model using AutoKeras, can we somehow export the structure of the model to be recreated in native TensorFlow and Keras (no dependencies on Autokeras lib)?

Can this process be automated, or can we just print out all the layers in the trained model and recreate that in TensorFlow/Keras?

I think this is a pretty vital feature that is useful in a lot of use-cases. It also helps in re-creating the best-found model in Pytorch and customize to our needs.

[haifeng-jin]

I am not sure how to do it. It is the same as saving some customized layers with the Keras model and load it without the customized layer code. If you can find a solution to that it can be applied.

[neel04]

yeah, so Autokeras uses some customized layers. So If I train the model, can I export all the minimal, basic code needed to construct that layer and use it independently of AutoKeras?

[shun-lin]

I think you can with the Model class that is stored in self.tuner right?

Does the below work?

clf = ak.ModelClass()
model = clf.export_model()
model.save(...)

And I think you can used the saved_model elsewhere without importing autokeras.

[neel04]

@shun-lin I mean that there be a way where I can export the code only in pure Keras and TensorFlow rather than depending on AutoKeras modules. @haifeng-jin

[shun-lin]

I think this is exactly what save model in Tensorflow does right? https://www.tensorflow.org/guide/saved_model

Save model does not depends on the module but it produces the raw graph and weights.

[neel04]

@shun-lin ye, I know that man. but as I said before, I want to export the model code where it just uses Tensorflow and Keras; i.e recreate the model with only tf.keras and not using Autokeras

[shun-lin]

oh ops sorry for mis-understanding your question.

[eschibli]

It should be possible to at least ameliorate the issue by moving the custom layers to another repo with fewer dependencies. I don't think it's possible to easily cache a model that uses custom layers without requiring those layers to be defined before loading.

[neel04]

I agree with @eschibli proposal - even if we can have the standalone code for the custom layers used in AutoKeras, then we can still re-create the exact keras code. @haifeng-jin would you happen to know if this is possible OR if the custom layers currently can be re-created without any dependencies to AutoKeras?

Could you try making a standalone colab notebook for showing how we can use the AutoKeras custom layers seperately?

[haifeng-jin]

As AutoKeras is still not stable enough, it is not time for us to separate another repo for it. Meanwhile, I will be consulting others from Keras team to see if there is a solution to pack the custom layers without importing the original code.

[neel04]

cool, thanks a lot @haifeng-jin :+1:

[neel04]

Meanwhile, as I understand, the custom layers are actually blocks. would you mind explaining what layers/tensor-ops each block does which may be replicated in Keras with pre-existing layers?

[haifeng-jin]

All the layers we implemented as customized layers are in this file. https://github.com/keras-team/autokeras/blob/master/autokeras/keras_layers.py

[neel04]

I meant the "blocks" - like TextClassification block - what keras layers is that block made up of? and can I use it outside AutoKeras, say standalone colab?

[haifeng-jin]

For the blocks, they cannot be used stand-alone. The blocks contains the hyperparameters, which is defined in keras-tuner. To find out what layers are used in the block, you have to check the source code of the block.

[neel04]

I got this guy who apparently did it here:- https://stackoverflow.com/questions/66944411/issue-replicating-autokeras-structureddataclassifier/67084576#67084576 without any isolation of code, only of the model. His answer is the accepted one, the other one is mine.

This seems to be impossible since you can't reconstruct layers that aren't defined in TF/Keras from the weights alone and his layers certainly weren't. Any idea what he did? His answer seems very unclear and vague to me.

[neel04]

Hey @haifeng-jin A pretty simple request:- There is this Custom AutoKeras layer <autokeras.keras_layers.ExpandLastDim at 0x7f8bfcbed250>, which is basically just the tf.expand_dims op.

Could we re-create the layer in Keras (like AK did) and could you summarize it's importance/function? it's code just seems to be this

@tf.keras.utils.register_keras_serializable()
class ExpandLastDim(preprocessing.PreprocessingLayer):
    def call(self, inputs):
        return tf.expand_dims(inputs, axis=-1)

Thanks in advance!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.