Ocelot Load Balance using Web Socket Issue

rycho27 commented 2 years ago

Hi Ocelot Team,

Please give advice for my current issue when using Load Balance configuration on Ocelot Gateway. My websocket implementation is for Microsof SignalR on my web client

## Expected Behavior / New Feature When one of server is down, not causing error and able redirect request to connect websocket to another server

Actual Behavior / Motivation for New Feature

1 Backend Server down causing Bad Gateway request and unable to a connect websocket to another backend server

Steps to Reproduce the Problem

Server A installed with 1 Backend service (using .net core 3.1) and Gateway (Ocelot) service
Server B installed with 1 Backend service
When All service running, client able to connect websocket using gateway
When Backend service is turned off on Server A and backend in server B is running, client unable to connect websocket via gateway
When Backend service is turned off on Server B and backend in server A is running, client unable to connect websocket via gateway
Http request is running fine.

Specifications

Version: Ocelot 16.0.1
Platform: .net Core 3.1
Subsystem:

Please see my config on my ocelot.json

{
"Routes": [
    {
        "DownstreamPathTemplate": "/OPUIAPI/api/{everything}",
        "DownstreamScheme": "http",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8001
            },
            {
                "Host": "bthtsa1030.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/opui/api/{everything}",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    },
    {
        "DownstreamPathTemplate": "/eqstatoui/api/{everything}",
        "DownstreamScheme": "http",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/eqstat/api/{everything}",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    },
    {
        "DownstreamPathTemplate": "/eqstatoui/signalr",
        "DownstreamScheme": "ws",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/eqstat/signalr",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post",
            "Delete",
            "Options"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    },
    {
        "DownstreamPathTemplate": "/eqstatoui/signalr/{catchAll}",
        "DownstreamScheme": "ws",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/eqstat/signalr/{catchAll}",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post",
            "Delete",
            "Options"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    },
    {
        "DownstreamPathTemplate": "/OPUIAPI/signalr",
        "DownstreamScheme": "ws",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8001
            },
            {
                "Host": "bthtsa1030.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/opui/signalr",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post",
            "Delete",
            "Options"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    },
    {
        "DownstreamPathTemplate": "/OPUIAPI/signalr/{catchAll}",
        "DownstreamScheme": "ws",
        "DownstreamHostAndPorts": [
            {
                "Host": "bthsa1163.infineon.com",
                "Port": 8001
            },
            {
                "Host": "bthtsa1030.infineon.com",
                "Port": 8000
            }
        ],
        "UpstreamPathTemplate": "/gateway/opui/signalr/{catchAll}",
        "UpstreamHttpMethod": [
            "Get",
            "Put",
            "Post",
            "Delete",
            "Options"
        ],
        "AuthenticationOptions": null,
        "FileCacheOptions": {
            "TtlSeconds": 0
        },
        "LoadBalancerOptions": {
            "Type": "LeastConnection"
        }
    }
],
"GlobalConfiguration": {
    "BaseUrl": "https://localhost:5011"
}
}

nhochamvui commented 1 year ago

so sad bro, i'm facing this issue too

raman-m commented 7 months ago

@ggnaegi What do you think? The author wrote strange scenarios... He switched off downstream service and then he wanted to connect successfully. 😂

raman-m commented 7 months ago

Hi @rycho27 ! Most of routes are correct, but some routes are incorrectly defined. You cannot load balance if 1 service, aka DownstreamHostAndPorts object! You CAN load balance if 2+ services, or service discovery is enabled.

ggnaegi commented 7 months ago

{
            "DownstreamPathTemplate": "/OPUIAPI/signalr",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8001
                },
                {
                    "Host": "bthtsa1030.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/opui/signalr",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        }

Yes, I would expect the same behavior too, if one of the two servers is down, then the requests should be sent to the available service... We should check that.

raman-m commented 7 months ago

@ggnaegi OK I've got it. Wonder if it was not implemented in the past... I guess also that the problem is not related to ws scheme, the issue can be reproduced with http scheme too. I see Load Balancer limitation here when one service instance in the list is down, then Ocelot is not smart enough to find the next online service instance. Yeap, we have to filter offline service instances making requests to online ones, so making "redirects".

I see this is by design issue. And because Ocelot has no servicing Health Status feature we get this problem when Load Balancer service instances are not healthy. It will be great if we will think in terms of Microservices architecture which recommends to make quick short regular Health Check requests to all registered domain services. So, we need to design very simple Health feature... That will be great feature of Ocelot respecting Microservices philosophy.

raman-m commented 7 months ago

Duplicate of #1041

ThreeMammals / Ocelot