jina-ai / jina

☁️ Build multimodal AI applications with cloud-native stack
https://docs.jina.ai
Apache License 2.0
20.56k stars 2.21k forks source link

Have Deployments Call Extend API When No Gateway #6107

Closed NarekA closed 7 months ago

NarekA commented 7 months ago

Describe your proposal/problem

If you are using standalone deployments as we are, there is no way to extend the REST API. Below I have a workaround, but it would be useful to have jina.serve.runtimes.worker.http_fastapi_app.get_fastapi_app call extend_rest_interface. We have some endpoints that do not fit into the jina model, and would like to serve them in the same deployment.

Current Workaround

For anyone looking for a way to extend deployments currently, this seems to work:


import wrapt
from pydantic import BaseModel

class MyInput(BaseModel):
    input: str

def extend_rest_function(app):

    @app.post(
        "/api/openai/v1/chat/completions",
        tags=["OpenAI Interface"],
    )
    async def post(
        input: MyInput
    ):
        return {"output": "hello world"}

    return app

def extend_fastapi_wrapper(wrapped, instance, args, kwargs):
    app = wrapped(*args, **kwargs)
    return extend_rest_function(app)

wrapt.wrap_function_wrapper(
    "jina.serve.runtimes.worker.http_fastapi_app",
    'get_fastapi_app',
    extend_fastapi_wrapper
)

from jina import Executor, requests, Deployment
from docarray import DocList
from docarray.documents import TextDoc

class MyExecutor(Executor):
    @requests(on='/foo')
    def foo(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        docs[0].text = 'hello, world!'
        docs[1].text = 'goodbye, world!'
        return docs

with Deployment(
    uses=MyExecutor
) as dep:
    dep.block()
JoanFM commented 7 months ago

Hey @NarekA,

thanks for reporting the issue.

I have a concern about this feature, and is its cloud nativeness. I am not sure how it would play with all the cloud nativeness we have. if you use one of these, then you kind of lose the support we have in our orchestration layer.

What do you think?

NarekA commented 7 months ago

@JoanFM We are using Jina as web framework. We would like to use the orchestration layer, but at the moment it is not compatible with our internal infrastructure. At times Jina is restrictive as a web framework, so being able to access FastAPI directly makes it much more feasible in the long term.