This release contains 5 new features 3 bug fixes and 8 documentation improvements.
π Features
Executor can work on single documents (#5991)
Executors no longer need to work solely on a DocList, but can expose endpoints for working on single documents.
For this, the method decorated by requests must take a doc argument and an annotation for the input and output types.
from jina import Executor, requests
from docarray import BaseDoc
class MyInputDocument(BaseDoc):
num: int
class MyOutputDocument(BaseDoc):
label: str
class MyExecutor(Executor):
@requests(on='/hello')
async def task(self, doc: MyInputDocument, **kwargs) -> MyOutputDocument:
return MyOutputDocument(label='even' if doc.num % 2 == 0 else 'odd')
This keeps Executor code clean, especially for serving models that can't benefit from working on batches of documents at the same time.
Parameters can be described as Pydantic models (#6001)
An Executor's parameters argument can now be a Pydantic model rather than a plain Python dictionary. To use a Pydantic model, the parameters argument needs to have the model as a type annotation.
Defining parameters as a Pydantic model instead of a simple Dict has two main benefits:
Validation and default values: You can get validation of the parameters that the Executor expected before the Executor may access any invalid key. You can also easily define defaults.
Descriptive OpenAPI definition when using the HTTP protocol.
Expose richer OpenAPI when serving Executor with HTTP inside a Flow (#5992)
Executors served with Deployments and Flows can now provide a descriptive OpenAPI when using HTTP. The description, examples and other relevant fields are used in the Gateway to provide a complete API.
Support streaming in single-Executor Flows (#5988)
Streaming endpoints now also support the Flow orchestration layer and it is no longer mandatory to use just a Deployment.
A Flow orchestration can accept streaming endpoints under both the gRPC and HTTP protocols.
with Flow(protocol=protocol, port=port, cors=True).add(
uses=StreamingExecutor,
):
client = Client(port=port, protocol=protocol, asyncio=True)
i = 10
async for doc in client.stream_doc(
on='/hello',
inputs=MyDocument(text='hello world', number=i),
return_type=MyDocument,
):
print(doc)
Streaming endpoints with gRPC protocol (#5921)
After adding SSE support to allow streaming documents one by one for the HTTP protocol, we added the same functionality for the gRPC protocol. A Jina server can now stream single Documents to a client, one at a time, using gRPC. This feature relies on streaming gRPC endpoints under the hood.
One typical use-case of this feature is streaming tokens from a Large Language Model. For instance, check our how-to on streaming LLM tokens.
from jina import Executor, requests, Deployment
from docarray import BaseDoc
# first define schemas
class MyDocument(BaseDoc):
text: str
# then define the Executor
class MyExecutor(Executor):
@requests(on='/hello')
async def task(self, doc: MyDocument, **kwargs) -> MyDocument:
for i in range(100):
yield MyDocument(text=f'hello world {i}')
with Deployment(
uses=MyExecutor,
port=12345,
protocol='grpc', # or 'grpc'
) as dep:
dep.block()
From the client side, you can use the new stream_doc() method to receive documents one by one:
from jina import Client, Document
client = Client(port=12345, protocol='grpc', asyncio=True)
async for doc in client.stream_doc(
on='/hello', inputs=MyDocument(text='hello world'), return_type=MyDocument
):
print(doc.text)
Fix caching models from all endpoints, inputs and outputs (#6005)
An issue was fixed that caused problems when using an Executor inside a Flow where the same document type was used as input and output in different endpoints.
Use 127.0.0.1 as local ctrl address (#6004)
The orchestration layer will use 127.0.0.1 to send health checks to Executors and Gateways when working locally. It previously used 0.0.0.0 as the default host and this caused issues in some configurations.
Ignore warnings from Google (#5968)
Warnings that used to appear in relation to the pkg_resources deprecated API are now suppressed.
π Documentation Improvements
Fix errors in getting started and preliminaries (#6008)
Add note about Flow with one Executor supported (#5990)
Add docs for secrets and jobs (#5948)
Remove include gateway arg (#5987)
Add Kubernetes hint to port ignore (#5985)
Add docs for streaming (#5980)
Add docs for jcloud horizontal pod autoscale (#5957)
Update docs to show single document serving (#6009)
π€ Contributors
We would like to thank all contributors to this release:
Release Note
This release contains 5 new features 3 bug fixes and 8 documentation improvements.
π Features
Executor can work on single documents (#5991)
Executors no longer need to work solely on a
DocList
, but can expose endpoints for working on single documents.For this, the method decorated by
requests
must take adoc
argument and an annotation for the input and output types.This keeps Executor code clean, especially for serving models that can't benefit from working on batches of documents at the same time.
Parameters can be described as Pydantic models (#6001)
An Executor's
parameters
argument can now be a Pydantic model rather than a plain Python dictionary. To use a Pydantic model, theparameters
argument needs to have the model as a type annotation.Defining
parameters
as a Pydantic model instead of a simple Dict has two main benefits:Expose richer OpenAPI when serving Executor with HTTP inside a Flow (#5992)
Executors served with Deployments and Flows can now provide a descriptive OpenAPI when using HTTP. The description, examples and other relevant fields are used in the Gateway to provide a complete API.
Support streaming in single-Executor Flows (#5988)
Streaming endpoints now also support the Flow orchestration layer and it is no longer mandatory to use just a Deployment. A Flow orchestration can accept streaming endpoints under both the gRPC and HTTP protocols.
Streaming endpoints with gRPC protocol (#5921)
After adding SSE support to allow streaming documents one by one for the HTTP protocol, we added the same functionality for the gRPC protocol. A Jina server can now stream single Documents to a client, one at a time, using gRPC. This feature relies on streaming gRPC endpoints under the hood.
One typical use-case of this feature is streaming tokens from a Large Language Model. For instance, check our how-to on streaming LLM tokens.
From the client side, you can use the new
stream_doc()
method to receive documents one by one:Read more in the docs.
π Bug Fixes
Fix caching models from all endpoints, inputs and outputs (#6005)
An issue was fixed that caused problems when using an Executor inside a Flow where the same document type was used as input and output in different endpoints.
Use
127.0.0.1
as local ctrl address (#6004)The orchestration layer will use
127.0.0.1
to send health checks to Executors and Gateways when working locally. It previously used0.0.0.0
as the default host and this caused issues in some configurations.Ignore warnings from Google (#5968)
Warnings that used to appear in relation to the
pkg_resources
deprecated API are now suppressed.π Documentation Improvements
π€ Contributors
We would like to thank all contributors to this release: