Integrated Gradio Demo App

[Ankur-singh]

Describe the feature The Jina project is an incredible tool that simplifies the process of deploying machine learning models by just writing an executor. It enables users to get a complete working service with ease. However, during the development and testing stages, it would be extremely beneficial to have an integrated Gradio demo app. This will provide an interactive user interface that allows users to easily interact and test the model without the need for using CURL, Postman, or programming languages.

Your proposal

• The integrated Gradio demo app can be accessible at the /demo endpoint.
• Users can interact with the model using the Gradio interface, providing input and viewing the model's output in real-time.
• The Gradio demo app should automatically generate the appropriate input components based on the model's input types (e.g., string, integer, float, boolean, file, path) and utilize the docArray model's additional information for description, default values, minimum, maximum, and choices, when available.
• The Gradio integration should be optional, allowing users to enable or disable it based on their use-case. This will enable developers to use the interactive demo during development and prototyping and easily disable it for production deployment.

Benefits:

• Eases the testing and interaction with the model during development and prototyping stages.
• Provides a user-friendly interface for non-technical stakeholders to interact with the model and understand its capabilities.
• Improves user experience by eliminating the need for external tools (e.g., CURL, Postman) or custom code for testing the model.

Implementation Approach:

• Utilize Gradio's Python library to create the interactive demo app.
• Mount the Gradio demo app using FastAPI Gateway.
• Leverage the existing docArray documents information to generate appropriate input and output components with optional descriptions.

This feature will further enhance the usability and user experience of the Jina project, making it an even more valuable tool for deploying machine learning models.

I am willing to take on this feature request as it presents an excellent opportunity to learn more about Jina internals.

[JoanFM]

This looks like an amazing feature.

[Ankur-singh]

Perfect, I will start working on it this weekend

[alaeddine-13]

Amazing feature @Ankur-singh indeed. Here is how I imagine the feature being added to jina:

• for simplicity, suppose for now the feature is only added to Deployments that use the HTTP protocol
• at the gateway level, the gradio app needs to be mounted to the fastapi server under /demo. You can create 1 page per executor endpoint. Note that the gateway already fetches the schema definitions of the executor server and you can find it in the gateway class in the field self.streamer._endpoints_models_map.
• at the executor level, if the executor uses http, the gradio app can be mounted to the executor server. Executor endpoint definitions are also already parsed and you can find them in the Executor field self.requests.

[Ankur-singh]

Thank you so much @alaeddine-13 for all the insights.

[Clarasia]

I found this issue on https://ovio.org/projects and would like to contribute! Please guide on how to contribute and work on the project. Thanks

[Clarasia]

I have a minimal knowledge on python and would like to contribute to this open-source project.

[Ankur-singh]

Hey @Clarasia, I have already halfway through it. But still feel free to go ahead and give it a try. It will be a good learning experience.

[Clarasia]

Hi Ankur,

I was wondering how I can contribute to this project and when I should start.

All the best Clarasia

---

From: Ankur Singh @.> Sent: 16 August 2023 18:09 To: jina-ai/jina @.> Cc: Clarasia @.>; Mention @.> Subject: Re: [jina-ai/jina] Integrated Gradio Demo App (Issue #6026)

Hey @Clarasiahttps://github.com/Clarasia, I have already halfway through it. But still feel free to go ahead and give it a try. It will be a good learning experience.

— Reply to this email directly, view it on GitHubhttps://github.com/jina-ai/jina/issues/6026#issuecomment-1680974948, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BBSOXFNASYSWNZMSHH3B6WLXVT5EVANCNFSM6AAAAAA3MNCPY4. You are receiving this because you were mentioned.Message ID: @.***>

[JoanFM]

Hey @Ankur-singh ,

I suggest that if you already have some work in progress, you can open a draft PR so that we can perhaps start giving early feedback to the development, design, etc ...

[Ankur-singh]

I have create a Github gist (link) which demonstrates how to add a Gradio gateway to a Jina Flow.

I wrote a function that takes Pydantic model as input and returns Gradio interface generated using the input model. The Gradio app is build dynamically and is mounted the FastAPI app object at /demo endpoint.

I have all the pieces but I will need to some help to add it to the flow/base.py file. It would be great if the Gradio app (/demo) can be places in the "HTTP extension" section when flow is executed.

[JoanFM]

You need to apply this in 2 places:

One in https://github.com/jina-ai/jina/blob/master/jina/serve/runtimes/worker/http_fastapi_app.py to be used when Deployment is used.

The other;

https://github.com/jina-ai/jina/blob/master/jina/serve/runtimes/gateway/http_fastapi_app_docarrayv2.py

to be used when Flow is used.

Just one thing I think it shlould not be under /demo perhaps but under some private reserved name like _jina_gradio_demo to avoid affecting when people may have a /demo endpoint defined

[Ankur-singh]

Thanks @JoanFM, these resources are really helpful. I agree with the private reserved name _jina_gradio_demo for the endpoint.

[jina-bot]

@jina-ai/product This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 14 days