Questions about TF hub

[KineticCookie]

Hello! I read docs and source code of TF hub and have questions:

1. https://github.com/tensorflow/hub/blob/master/docs/hosting.md protocol is very small. Is it intended, or do you plan to extend it? I would like to see API to get module specs without downloading the tarball, and some module discovery mechanism.
2. Do you have plans about versioning mechanism? Looks like version is just part of a module URL and it could be missing if module is deployed differently.
3. Is there a docker image or maybe an instruction on how to do a local deployment of TF hub?

Thank you in advance.

[andresusanopinto]

1) The protocol was created as a way to host both the documentation and module from the same URL. Although there are no planned features that required it, it seem better to leave the extensible option on the table rather than closed. Please file specific feature requests and :+1: accordingly.

2) At the moment, it is unclear if the version will be part of the protocol, or is left as something that exists at a layer above. For now it is a "recommended best practice".

3) Do you mean a local deployment of "tfhub.dev"? No. The hosting.md is all there is at the moment.

[KineticCookie]

Thank you for the answer.

Do you mean a local deployment of "tfhub.dev"? No. The hosting.md is all there is at the moment.

I want to deploy a TF module registry for playground and was wondering if there is a solution to quickly create a TF hub compatible server, so I could access modules from internal network with x.x.x.x/modules/ssd/1 URL.

Seems like I need to develop a custom module registry server.

[akhorlin]

One alternative to hosting your own registry server is to store module archive file (.tar.gz archived module) in a GCS or S3 bucket. Then this file can be made public which would generate a URL to it. Then, this URL can be used in the constructor of hub.Module.

On Wed, Apr 18, 2018 at 10:15 AM Bulat notifications@github.com wrote:

Thank you for the answer.

Do you mean a local deployment of "tfhub.dev"? No. The hosting.md is all there is at the moment.

I want to deploy a TF module registry for playground and was wondering if there is a solution to quickly create a TF hub compatible server, so I could access modules from internal network with x.x.x.x/modules/ssd/1 URL.

Seems like I need to develop a custom module registry server.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/hub/issues/26#issuecomment-382303915, or mute the thread https://github.com/notifications/unsubscribe-auth/AbunTEt4C-XKrNku_Lt91JSY8rzXqKM-ks5tpvYvgaJpZM4TU8HY .

[KineticCookie]

@akhorlin thanks for the solution. But I don't want to use cloud storages (all the work is done in LAN, and isolated) and the solution doesn't provide standardised docs for modules. (like https://www.tensorflow.org/hub/modules/google/imagenet/inception_v1/feature_vector/1)

[letsbuild]

Is there a docker image or maybe an instruction on how to do a local deployment of TF hub?

This is not exactly what you might have asked, but is related to TF-Hub and Docker. Namely, if you want to run TF Hub inside a docker container and run some of Colabs in that environment you can easily build a docker image that extents the Jupyter TensorFlow docker stack as follows:

1. Create a Dockerfile with:

   # Start from a TensorFlow-compatible Docker image
   FROM jupyter/tensorflow-notebook
   # Add the extra dependencies for TF-hub
   RUN pip install --upgrade pip
   RUN pip install --upgrade "tensorflow>=1.7.0"
   RUN pip install --upgrade tensorflow-hub
   RUN pip install --upgrade seaborn

2. Build the image:

   docker build --rm -t jupyter/tensorflow-with-tfhub-notebook .

3. Start the image:

   docker run -p 8888:8888 jupyter/tensorflow-with-tfhub-notebook

4. Log into the Jupyter notebook in http://localhost:8888
5. Download the ipynp files and upload them to the local notebook --- they should run verbatim (although you might need to move any pip install from the ipynp cells to the Dockerfile config).

Any module you run will be locally cached in the TF-hub cache dir and will be part of your image. So, in theory, you could run all modules and they will be locally cached and part of the image.