search

opea-project / GenAIInfra

Containerization and cloud native suite for OPEA
Apache License 2.0
26 stars 48 forks source link

Add gateway guardrails #363

Closed zhxie closed 3 weeks ago

zhxie commented 1 month ago

Description

Add gateway guardrails at the entry point of the pipeline which enhances the security, reduces the latency and leverages hardware accelerators.

Since GenAIInfra has integrated the Istio ingress/egress gateway as its gateway, it is easy to start with that.

* To separated PRs.

Issues

https://github.com/opea-project/docs/pull/23 https://github.com/opea-project/GenAIExamples/issues/573

Type of change

List the type of change like below. Please delete options that are not relevant.

Dependencies

Hyperscan, OpenVINO

Tests

Unit tests

zhlsunshine commented 4 weeks ago

Hi @zhxie, please add the source header for all your .h and .cpp code files. For example,

// Copyright (C) 2024 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
lianhao commented 3 weeks ago

Should we rename the directory name to something more specific, proxy is too generic and vague to me. Any comment @mkbhanda ?

mkbhanda commented 3 weeks ago

I agree, even though its proxy stuff

Gateway? Guardrails? Safety?

From: Lianhao Lu @.> Sent: Friday, September 6, 2024 2:02 AM To: opea-project/GenAIInfra @.> Cc: Bhandaru, Malini @.>; Mention @.> Subject: Re: [opea-project/GenAIInfra] Add gateway guardrails (PR #363)

Should we rename the directory name to something more specific, proxy is too generic and vague to me. Any comment @mkbhandahttps://github.com/mkbhanda ?

- Reply to this email directly, view it on GitHubhttps://github.com/opea-project/GenAIInfra/pull/363#issuecomment-2333589159, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAUTXVMEW4WXUUZUNRWXHLLZVFVR3AVCNFSM6AAAAABNIAKMB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZTGU4DSMJVHE. You are receiving this because you were mentioned.Message ID: @.**@.>>

zhxie commented 3 weeks ago

I prefer the name gateway since it is actually an Istio with AI features and sits in the gateway place as defined in the docs (https://opea-project.github.io/latest/introduction/index.html#gateways-customized-access-to-mega-and-microservices).

Other features can also be implemented including rate limiting and request transformation if we prefer Istio in the deployment.