Open kirillgroshkov opened 2 years ago
Try making a fat jar and running it through proguard (for the shrinker). I don't know what rules you would need, it's probably going to be trial and error. If/once you get a working set we should document it here, and there might be a way to distribute them in the jars.
If your models can be exported to TF Lite, you might want to consider using that: https://github.com/bytedeco/javacpp-presets/tree/master/tensorflow-lite That's less than 3 MB per platform.
If your models can be exported to TF Lite, you might want to consider using that: https://github.com/bytedeco/javacpp-presets/tree/master/tensorflow-lite That's less than 3 MB per platform.
Oh that's a cool idea! I'll explore that! 👍
Our uncompressed TensorFlow library alone is 375M, which is slightly bigger than the one used for Python (312M), I think the difference is because we also export the C++ client API is ours. Still, I have no clue how we can distribute a library that big in a smaller jar...
I only need to run Inference of SavedModelBundle, nothing else
Even if you only do inference, you need all the TensorFlow ops and kernels to be present in your library in case your graph is using one of them. I haven't checked but I suspect that most of the bytes of this library are taken by the ops (over 1K).
Another option to consider is XLA AOT. I'm not entirely sure how you would use that from Java but you should be able to export the function and work from there.
I was trying to run ML Inference with tensorflow-java deployed to AppEngine, but quickly failed with this error:
Which means that AppEngine forbids to deploy any single file that's bigger than 32Mb. And the jar for
linux-x86_64
is ~62Mb right now. It is confirmed there: https://cloud.google.com/appengine/quotas#CodeJust checking if someone has any advice/workaround in mind?
Like, is it possible to maybe do a custom-build (I only need to run Inference of
SavedModelBundle
, nothing else) of tensorflow-java for Linux, to fit in under 32Mb? Or, split the jar into multiple sub-32Mb files?Not looking for AppEngine advice (as this repo is not a right place for it), but for something that can be done with this library itself. 🙏