Build Executor (model) UI in Jina

[Nick17t]

Project idea 1: Build Executor (model) UI in jina

info	details
Skills needed	Python
Project size	175 hours
Difficulty level	Easy
Mentors	@Alaeddine Abdessalem, @Philip Vollet

Project Brief Description

• Jina Executors are components that perform certain tasks and expose them as services using gRPC. Executors accept DocumentArrays as input and output. However, with DocArray v2 focusing on type annotations and enabling annotation of executor endpoints, it becomes possible for executors to describe their services and input/output in the same way as OpenAPI schemas. This allows us to offer built-in UIs for executors, enabling people to easily use their services with multi-modal data. The goal is to build this feature in Jina using Gradio.

Detailed Description

A jina model UI should work like a swagger UI in an HTTP framework. A swagger UI typically relies on an OpenAPI specifications to know about the services and input output schema, and then will generate the UI based on that. However, jina executors rely on gRPC and gRPC do not expose OpenAPI. This means we either have to implement an OpenAPI equivalent for gRPC services that Executors expose or adopt a certain standard (I like the kserve). Building such a service specification depends on knowing exactly the needed input and output schemas. Jina can now support defining input and output schemas of the endpoints using the docarray v2 in beta support. The API looks like this actually: https://docs.jina.ai/concepts/executor/docarray-v2/. This means that, assuming an executor is built using DocArray v2 in type annotations, we can generate service specifications in the same way FastAPI generates open API specifications based on pydantic type hints.

Then, the next step would be using this service specification endpoint to build a model UI. This can be done using gradio. However, since we cannot serve a UI on the executor's gRPC service, we can only host the UI on the gateway level. jina gateways recently added support for customization. The new support allows basically building any server based on any protocol. This means, a gateway can target an Executor within a jina flow and expose the UI for it.

I think the following API can make sense:

from jina import Flow

flow = Flow(protocol='ui').add(MyExecutorModel)

or

from jina import Flow, ModelUIGateway

flow = Flow().config_gateway(uses=ModelUIGateway).add(MyExecutorModel)

In jina, this would request the Executor's service specifications over grpc and then will dynamically build the gradio interface.

To sum it up, the proposed technologies are not necessarily what we need to adopt, as long as we implement the following:

• Executor service specifications (like OpenAPI specifications in http) based on DocArray v2 type hints
• Leverage the service specifications to build a model UI gateway. The UI Gateway is a custom Jina gateway and can use any framework (gradio, streamlit or even any web framework).
• Integrate the model ui gateway into jina (offer simple API to use it)

Expected outcomes

• Submit one or more Pull Requests (PRs) to the Jina repository that enables providing a built-in Executor UI for Executors. The UI can be built using Gradio and should be able to infer information about the Executor service using type annotations.

[Asrst]

Hi @Nick17t,

I would like to contribute to this as part of GSoC 2023. Just trying to understand the plan here - are we looking to automatically build some thing similar to swagger ui for each jina executor (but using gradio)?

Hi @alaeddine-13 , Any inputs here will help me to come up with a better proposal. pls let me know

[ranjan2829]

Project idea 1: Build Executor (model) UI in jina

info | details -- | -- Skills needed | Python Project size | 175 hours Difficulty level | Easy Mentors | @Alaeddine Abdessalem, @Philip Vollet

Project Description

• Jina Executors are components that perform certain tasks and expose them as services using gRPC. Executors accept DocumentArrays as input and output. However, with DocArray v2 focusing on type annotations and enabling annotation of executor endpoints, it becomes possible for executors to describe their services and input/output in the same way as OpenAPI schemas. This allows us to offer built-in UIs for executors, enabling people to easily use their services with multi-modal data. The goal is to build this feature in Jina using Gradio.

Expected outcomes

• Submit one or more Pull Requests (PRs) to the Jina repository that enables providing a built-in Executor UI for Executors. The UI can be built using Gradio and should be able to infer information about the Executor service using type annotations.

Project idea 1: Build Executor (model) UI in jina info details Skills needed Python Project size 175 hours Difficulty level Easy Mentors @[Alaeddine Abdessalem](https://github.com/alaeddine-13), @[Philip Vollet](https://www.linkedin.com/in/philipvollet) Project Description Jina Executors are components that perform certain tasks and expose them as services using gRPC. Executors accept DocumentArrays as input and output. However, with DocArray v2 focusing on type annotations and enabling annotation of executor endpoints, it becomes possible for executors to describe their services and input/output in the same way as OpenAPI schemas. This allows us to offer built-in UIs for executors, enabling people to easily use their services with multi-modal data. The goal is to build this feature in Jina using Gradio. Expected outcomes Submit one or more Pull Requests (PRs) to the Jina repository that enables providing a built-in Executor UI for Executors. The UI can be built using Gradio and should be able to infer information about the Executor service using type annotations. Install Jina: First, you need to install Jina. You can use pip to install it: Copy code pip install jina Define your Executor: Next, you need to define your Executor. For example, you can define a simple Executor that takes a list of strings as input and returns the length of each string: from jina.executors import BaseExecutor class MyExecutor(BaseExecutor): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) def exec_fn(self, input): return [len(s) for s in input] This code defines a MyExecutor class that inherits from Jina's BaseExecutor class. The MyExecutor class has an exec_fn method that takes a list of strings as input and returns the length of each string as a list. Define a Flow: Once you have defined your Executor, you need to define a Flow that uses your Executor. For example, you can define a Flow that takes input from a REST API endpoint and passes it to your Executor: from jina.flow import Flow flow = Flow().add( name='my_executor', uses=MyExecutor, parallel=2 ) This code defines a Flow object that uses your MyExecutor class. The parallel parameter specifies the number of parallel instances of the Executor to run. Define a REST API endpoint: Next, you need to define a REST API endpoint that accepts input data and passes it to your Flow. For example, you can define a Flask app with a route that accepts POST requests: python _**from flask import Flask, request, jsonify app = Flask(__name__)**_ @app.route('/api/execute', methods=['POST']) **def execute(): input_data = request.json['data'] with flow: result = flow.index(input_data) return jsonify({'result': result})** This code defines a Flask app with a /api/execute route that accepts POST requests. The route extracts the input data from the request, passes it to the Flow, and returns the result as a JSON response. Run the Flask app: Finally, you need to run the Flask app to start the REST API endpoint: if __name__ == '__main__': app.run(debug=True, port=5000) This code runs the Flask app with debugging enabled on port 5000. That's it! With these steps, you should be able to build an Executor UI in Jina that uses a REST API endpoint to pass input data to your Executor and return the result. Of course, you will need to add more code to handle things like input validation, error handling, and authentication, but this should give you a good starting point.

[alaeddine-13]

Hey @Asrst @ranjan2829 , Thanks for your interest in contributing !

I believe your understanding is correct. A jina model UI should work like a swagger UI in an HTTP framework. A swagger UI typically relies on an OpenAPI specifications to know about the services and input output schema, and then will generate the UI based on that. However, jina executors rely on gRPC and gRPC do not expose OpenAPI. This means we either have to implement an OpenAPI equivalent for gRPC services that Executors expose or adopt a certain standard (I like the kserve). Building such a service specification depends on knowing exactly the needed input and output schemas. Jina can now support defining input and output schemas of the endpoints using the docarray v2 in beta support. The API looks like this actually: https://docs.jina.ai/concepts/executor/docarray-v2/. This means that, assuming an executor is built using DocArray v2 in type annotations, we can generate service specifications in the same way FastAPI generates open API specifications based on pydantic type hints.

Then, the next step would be using this service specification endpoint to build a model UI. This can be done using gradio. However, since we cannot serve a UI on the executor's gRPC service, we can only host the UI on the gateway level. jina gateways recently added support for customization. The new support allows basically building any server based on any protocol. This means, a gateway can target an Executor within a jina flow and expose the UI for it.

I think the following API can make sense:

from jina import Flow

flow = Flow(protocol='ui').add(MyExecutorModel)

or

from jina import Flow, ModelUIGateway

flow = Flow().config_gateway(uses=ModelUIGateway).add(MyExecutorModel)

In jina, this would request the Executor's service specifications over grpc and then will dynamically build the gradio interface.

To sum it up, the proposed technologies are not necessarily what we need to adopt, as long as we implement the following:

• Executor service specifications (like OpenAPI specifications in http) based on DocArray v2 type hints
• Leverage the service specifications to build a model UI gateway. The UI Gateway is a custom Jina gateway and can use any framework (gradio, streamlit or even any web framework).
• Integrate the model ui gateway into jina (offer simple API to use it)

[ranjan2829]

@alaeddine-13 Thank you for providing more information about your feature request for a Jina model UI. Based on your explanation, it sounds like a very useful and valuable addition to the Jina platform.

I appreciate the level of detail you have provided about the technical implementation of the feature, as well as the proposed API. It's great to see that you have already thought through many of the technical challenges and possible solutions.

We will certainly consider your request and explore the feasibility of implementing it in a future release of Jina. As you mentioned, we may need to evaluate different technologies and frameworks to determine the best approach for implementing this feature, but we will definitely keep your proposed solution in mind.

Thank you again for your interest in contributing to the development of Jina. If you have any further suggestions or feedback, please do not hesitate to let us know.

Best regards, Ranjan

[anurag2315]

from jina import Flow, Executor from jina.types.document import DocumentArray from jina.types.arrays.memmap import DocumentArrayMemmap from jina.types.document.generators import from_ndarray from jina import Document from jina.types.ndarray.generic import NdArray import gradio as gr

class MyExecutorModel(Executor): def init(self, *args, *kwargs): super().init(args, **kwargs)

def foo(self, inputs: DocumentArray, *args, **kwargs) -> DocumentArray:
    for doc in inputs:
        # some processing logic
        doc.embedding = NdArray(np.random.rand(10,))
    return inputs

flow = Flow().add(MyExecutorModel)

Define the input and output schema using DocArray v2 type hints

input_schema = DocumentArrayMemmap( field_resolver={ 'text': 'string', 'image': 'blob' } ) output_schema = DocumentArrayMemmap( field_resolver={ 'embedding': 'ndarray' } )

Define the Gradio interface for the MyExecutorModel's "foo" method

def gradio_interface(inputs): inputs = from_ndarray(inputs, input_schema) outputs = flow.post(on='/foo', inputs=inputs, return_results=True)[0].data.docs outputs = outputs.to_ndarray(output_schema) return outputs

Define the Model UI Gateway and configure it to use the Gradio interface

from jina.types.request import Response from jina import Gateway from jina.proto.jina_pb2_grpc import GatewayServicer, add_GatewayServicer_to_server from concurrent import futures import grpc

class ModelUIGateway(Gateway): class Service(GatewayServicer): def init(self, gradio_interface): self.gradio_interface = gradio_interface

    def Call(self, request, context):
        inputs = request.request
        outputs = self.gradio_interface(inputs)
        return Response(query=inputs, data=outputs)

def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)
    self.gradio_interface = None

def post_init(self):
    self.grpc_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    add_GatewayServicer_to_server(self.Service(self.gradio_interface), self.grpc_server)
    self.grpc_server.add_insecure_port('[::]:50051')
    self.grpc_server.start()

def __call__(self, *args, **kwargs):
    self.gradio_interface = kwargs.pop('gradio_interface')
    super().__call__(*args, **kwargs)

Use the Model UI Gateway to host the Gradio interface for the MyExecutorModel's "foo" method

gradio_interface = gr.Interface(fn=gradio_interface, inputs=gr.inputs.Group([gr.inputs.Textbox(label="Text"), gr.inputs.Image(label="Image")]), outputs="numpy") ui_gateway = ModelUIGateway().config(service_port=50051, uses='grpc://localhost:50051', uses_after='gradio_interface') with flow, ui_gateway: flow.protocol = 'ui' flow.protocol_gateway = 'grpc://localhost:50051' flow.protocol_gateway_args = {'gradio_interface': gradio_interface} flow.block()

[AkbarHabeeb]

Dear Jina Team, I'm interested in this idea. currently, I'm doing the feasibility analysis and designing various aspects of building an executor UI. Will share more details once ready.

[anurag2315]

@alaeddine-13 @philipvollet

[wanaaqy99]

i want to contribute in this project looking forward to work with you all.

[ranjan2829]

Project idea 1: Build Executor (model) UI in jina

info details Skills needed Python Project size 175 hours Difficulty level Easy Mentors @alaeddine Abdessalem, @philip Vollet

Project Description

• Jina Executors are components that perform certain tasks and expose them as services using gRPC. Executors accept DocumentArrays as input and output. However, with DocArray v2 focusing on type annotations and enabling annotation of executor endpoints, it becomes possible for executors to describe their services and input/output in the same way as OpenAPI schemas. This allows us to offer built-in UIs for executors, enabling people to easily use their services with multi-modal data. The goal is to build this feature in Jina using Gradio.

Expected outcomes

• Submit one or more Pull Requests (PRs) to the Jina repository that enables providing a built-in Executor UI for Executors. The UI can be built using Gradio and should be able to infer information about the Executor service using type annotations.

Project idea 1: Build Executor (model) UI in jina info details Skills needed Python Project size 175 hours Difficulty level Easy Mentors @Alaeddine Abdessalem, @Philip Vollet Project Description Jina Executors are components that perform certain tasks and expose them as services using gRPC. Executors accept DocumentArrays as input and output. However, with DocArray v2 focusing on type annotations and enabling annotation of executor endpoints, it becomes possible for executors to describe their services and input/output in the same way as OpenAPI schemas. This allows us to offer built-in UIs for executors, enabling people to easily use their services with multi-modal data. The goal is to build this feature in Jina using Gradio. Expected outcomes Submit one or more Pull Requests (PRs) to the Jina repository that enables providing a built-in Executor UI for Executors. The UI can be built using Gradio and should be able to infer information about the Executor service using type annotations. Install Jina: First, you need to install Jina. You can use pip to install it:

Copy code pip install jina

Define your Executor: Next, you need to define your Executor. For example, you can define a simple Executor that takes a list of strings as input and returns the length of each string:

from jina.executors import BaseExecutor

class MyExecutor(BaseExecutor):

def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)

def exec_fn(self, input):
    return [len(s) for s in input]

This code defines a MyExecutor class that inherits from Jina's BaseExecutor class. The MyExecutor class has an exec_fn method that takes a list of strings as input and returns the length of each string as a list.

Define a Flow: Once you have defined your Executor, you need to define a Flow that uses your Executor. For example, you can define a Flow that takes input from a REST API endpoint and passes it to your Executor:

from jina.flow import Flow

flow = Flow().add( name='my_executor', uses=MyExecutor, parallel=2 ) This code defines a Flow object that uses your MyExecutor class. The parallel parameter specifies the number of parallel instances of the Executor to run.

Define a REST API endpoint: Next, you need to define a REST API endpoint that accepts input data and passes it to your Flow. For example, you can define a Flask app with a route that accepts POST requests:

python

_**from flask import Flask, request, jsonify

app = Flask(name)**_

@app.route('/api/execute', methods=['POST']) def execute(): input_data = request.json['data'] with flow: result = flow.index(input_data) return jsonify({'result': result}) This code defines a Flask app with a /api/execute route that accepts POST requests. The route extracts the input data from the request, passes it to the Flow, and returns the result as a JSON response.

Run the Flask app: Finally, you need to run the Flask app to start the REST API endpoint:

if name == 'main': app.run(debug=True, port=5000) This code runs the Flask app with debugging enabled on port 5000.

That's it! With these steps, you should be able to build an Executor UI in Jina that uses a REST API endpoint to pass input data to your Executor and return the result. Of course, you will need to add more code to handle things like input validation, error handling, and authentication, but this should give you a good starting point.

@alaeddine-13

[jamesbright]

I would love to be part of this project, looking forward to contribute and learn a lot.

[asuzukosi]

Here is a proof of concept for the implementation of the project. @alaeddine-13 @Nick17t what do you think? Also I would like to ask:

• if we implement a flow withe the UI gateway will that flow be used only of the UI of the executors?
• How do we account for the issue of flows having multiple executors?
• Will all the executors be listed on the same page? or will we have multiple urls for the same executor?
• If the UI protocol is used along with other protocols how will that affect the information displayed on the terminal

  
  from pydantic import BaseModel
  import gradio as gr

class InputModel(BaseModel): name: str id: int

class OutputModel(BaseModel): name: str othername: str

mappings = { str: gr.Textbox, int: gr.Number, }

def get_gradiointerfaces(model): fields = model.fields outputs = [] for k,v in fields.items(): intereface = mappings[v.type] output = intereface(label=k) outputs.append(output) return outputs

print(get_gradio_interfaces(OutputModel))

def greet(*args, **kwargs): pass

def lauch_interfaces(inputModel, outputModel): demo = gr.Interface( fn=greet, inputs=get_gradio_interfaces(inputModel), outputs=get_gradio_interfaces(outputModel)) demo.launch()

lauch_interfaces(inputModel=InputModel, outputModel=OutputModel)

[giuliocn]

Hello @ranjan2829, I am considering this project for my application to GSoC 2023. It seems like I have the required Python skills, but I am new to open source contributions and I am wondering how many people (contributors) you will need to complete the project.

Thanks for the details and starting code you have shared above.

[iamharshvardhan]

Dear @ranjan2829, I would like to contribute to the project for GSOC 2023. Although I have intermediate skills in python, I am eager to learn and contribute. I believe that this project will enable me to enhance my python skills and motivate me to do more open source contributions.

Thank you for considering.

[robinokwanma]

Nice project, I look forward to making contributions and leaving an impact.

[MarcDavila1022]

Hello @ranjan2829,

I would like to contribute to this project for GSOC 2023. I am new to open source contributions and I am excited to get into it. I want to learn more about Jina and hopefully make contributions to this project. Thank your for the wonderful idea.

[Nick17t]

Hi @jamesbright @AkbarHabeeb @Asrst @asuzukosi @krishnabhaktanurag01 @MarcDavila1022 @robinokwanma @iamharshvardhan @giuliocn @ranjan2829 @wanaaqy99

I am delighted to hear that you are interested in contributing to the Jina AI community! 🎉

To get started, please take a moment to fill out our survey so that we can learn more about you and your skills.

Also, don't forget to mark your calendars for the GSoC x Jina AI webinar on March 23rd at 2 pm (CET). This is an excellent opportunity to learn more about the projects and ask any questions you have about the requirements and expectations.

Our mentors will provide an in-depth overview of the projects and answer any questions you may have. So please don't hesitate to ask any questions or seek clarification on any aspect of the project.

Is there anything specific you would like to learn from the webinar? Do you have any questions about the Build Executor (model) UI in Jina project that you would like to see clarified during the Q&A session? Let me know, and I'll be happy to help!

Looking forward to seeing you at the webinar, and thank you for your interest in the Jina AI community! 😊

[Vanshikagoel0012]

Hello @ranjan2829,

I would like to contribute to this project for GSOC 2023. I am new to open source contributions and I am excited to get into it. I want to learn more about Jina and hopefully make contributions to this project.

[giuliocn]

Could we subclass from FastAPIBaseGateway to define our custom gateway? https://docs.jina.ai/concepts/gateway/customization/#subclass-from-fastapibasegateway Furthermore, if we used a custom gateway it would expose a custom REST API over HTTP protocol, correct? Then, couldn't we take direct advantage of Swagger UI to show the executor's specification exposed by our gateway? @alaeddine-13 @ranjan2829

[KenaMuigai]

Hello @philipvollet @alaeddine-13. I would like to contribute to this project for the GSoC. I think this is a great idea and an interesting project.

[Sanvi-Sundarrajan]

Hey @philipvollet @alaeddine-13, I would like to contribute for this project in GSoC 2023 as I find it interesting and I am familiar with the pre-requisite technologies.

[shantanavKapse]

Hello @Nick17t @alaeddine-13! I am interested to contribute to this project as a part of GSoC 2023. In this project we want a web UI where we can create and manage executors. Is this correct? Let me know if I am wrong understanding the project.

[Sanvi-Sundarrajan]

Hello @alaeddine-13 @philipvollet @Nick17t The code The code sample I’ve tried with references and understanding from the above conversation is as follows: • I installed jina locally on my system pip install jina

• Create a base executer class class MyExecutor(BaseExecutor): def init(self, *args, *kwargs): super().init(args, **kwargs)

_def execfn(self, input): return [len(s) for s in input] • Defining a flow from jina import Flow flow = Flow(protocol='ui').add(MyExecutor)

I have to work with the input /output schema for DocArray v2 and define Gradio interface for the executor created . After this we have to define the Model UI Gateway and integrate to use it with Gradio interface .

[sohit24]

Hello @Nick17t I really love to contribute to this project for GSOC 2023. I am new to open source contributions and I don't know anything. I have tried googling and saw a bunch of you tube videos but it made me confuse more and ended up understanding nothing about the open source contributions for gsoc.....

If you don't mind can you help me with the open source contributions, like how can we contribute to them

Thank you in advance

[Piyushoberoy]

Hello, @Nick17t I really love to contribute to this project, since I had experience with this type of work earlier in my last semester. It will be very helpful if you give me a chance to work with you. I have done 2 research also in the field of machine learning and computer vision, along with this I have made multiple technologies that are very useful for society such as Seasonal Sale Prediction, University Admission Prediction, and many more. I sure giving me a chance to work under your guidance will never let your efforts go in vain. Looking forward to a positive response from your side.

Regards Piyush

[koushik2890]

Hello, @Nick17t

I have an intermediate knowledge in Python and I am a beginner in open source projects. I would really love to learn and contribute in this project. Thanks for considering.

Regards Koushik

[AviGawande]

Hello @alaeddine-13 could you please tell me that whether this project is open for GSOC 2024. If yes then it is very exciting for me to work on this project. And also if this project is not in GSOC 2024 could your please suggest me the project for upcoming GSOC program.