JSON-RPC DoS through Websockets

[c4-bot-4]

Lines of code

https://github.com/code-423n4/2023-11-zetachain/blob/b237708ed5e86f12c4bddabddfd42f001e81941a/repos/node/rpc/websockets.go#L50

Vulnerability details

Impact

The Websocket service accepts messages for 32MB size.

File: repos\node\rpc\websockets.go

const (
    messageSizeLimit = 32 * 1024 * 1024 // 32MB

)
(...)
conn.SetReadLimit(messageSizeLimit)

This size is 3 times bigger than what Golang accepts by default (10MB) on the HTTP service, while there is no reason that websocket payloads are bigger than HTTP payloads. In addition, there is no rate limiting within the code for the websockets.

As a consequence, an external attacker can take down the JSON RPC server by :

• Opening a websocket RPC Connection to Zetachain through Websockets.
• Sending in loop, with parallel workers, malicious Websockets messages (with an unknown method) and a total message size of 32MB.

Details

When the WebSocket server runs, it listens to the messages with readLoop() (L211).

When a message is received, the message body is put in mb variable. (L222)

This content will then be used to instance a msg variable.

File: repos\node\rpc\websockets.go

_, mb, err := wsConn.ReadMessage()
(...)
var msg map[string]interface{}
        if err = json.Unmarshal(mb, &msg); err != nil {
            s.sendErrResponse(wsConn, err.Error())
            continue
        }

Afterwards, some other variables are defined:

• method - msg["method"]
• id - msg["id"]

Depending on the "method", a switch case will determine the execution path. If the method is unknown, a call is done to tcpGetAndSendResponse():

File: repos\node\rpc\websockets.go

        default:
            // otherwise, call the usual rpc server to respond
            if err := s.tcpGetAndSendResponse(wsConn, mb); err != nil {
                s.sendErrResponse(wsConn, err.Error())
            }
        }

This tcpGetAndSendResponse function will call the RPC, locally with HTTP:

File: repos\node\rpc\websockets.go

func (s *websocketsServer) tcpGetAndSendResponse(wsConn *wsConn, mb []byte) error {
    req, err := http.NewRequestWithContext(context.Background(), "POST", "http://"+s.rpcAddr, bytes.NewBuffer(mb))
    if err != nil {
        return errors.Wrap(err, "Could not build request")
    }

However, this function does not perform any sanitation check. If an attacker sends a 32Mb message on Websockets, it will internally forward the call to the HTTP server with 32Mb payload.

Proof of Concept

A PoC was developed to exploit the vulnerability with :

• 14MB incorrect method name eth_XYZ where XYZ is a random 14MB string,

  The interest in using this function is that the server will attempt to send back the name of the method that was not found, sending therefore an extra 14MB back to the client.

• 14MB payload (random string)

• Only non-ASCII characters in the payload since it seems to cause more problems on the server upon decoding.

• 30 workers from a single machine

The PoC can be found here : https://gist.github.com/0xfadam/2846ee14d67ea95741f27e50570ac77a

The PoC can be launch with the following commands:

go mod init poc
go mod tidy
go run poc.go -ip 127.0.0.1 -ws-port 9546 -secure=false -workers 30

A video showing the crash of the server after exploitation can be found below :

https://drive.google.com/open?id=130MD8xBhNPNawRYksuETfP1P0Kr9Zxom&usp=drive_fs

Recommended Mitigation Steps

The Zetachain server needs to:

• Limit the size of each message. It should be the same what is configured for HTTP (10 MB)
• Enforce an applicative rate-limit mechanism so a single client cannot send thousands of messages within a short timeframe.

Assessed type

DoS

[c4-pre-sort]

DadeKuma marked the issue as sufficient quality report

[DadeKuma]

A lack of input sanitization and rate limit might crash the RPC server. This is possible when Conn.MaxPayloadBytes is not set to an appropriate value.

[c4-pre-sort]

DadeKuma marked the issue as primary issue

[c4-sponsor]

lumtis (sponsor) confirmed

[0xean]

Warden fails to show how this leads to a direct loss of funds. DOS is considered M per c4 docs

2 — Med: Assets not at direct risk, but the function of the protocol or its availability could be impacted, or leak value with a hypothetical attack path with stated assumptions, but external requirements.

[c4-judge]

0xean changed the severity to 2 (Med Risk)

[c4-judge]

0xean marked the issue as satisfactory

[c4-judge]

0xean marked the issue as selected for report

[iFrostizz]

I understand that DoS are classified as Medium in C4 contest because they are usually linked to hypothetical risks. However, there are scenarios that can lead to direct fund impact for example by targetting validators directly. First of all, there is this other DOS which is classified as a high: https://github.com/code-423n4/2023-11-zetachain-findings/issues/418#issuecomment-1880148123

1/ Attacking Validators Directly Causing Slashing Of The Validator.

In this scenario, let's assume that Alice is a validator (Legimate) and Eve is the attacker. An attacker can cause a slashing of a validator. Zetachain is using a PoS consensus algorithm (PBFT) in PBFT the validators has to put some value in the protocol (called Staking) (PoS) to validate the blocks.

Briefly, if a validator is not behaving correctly then the Protocol (will slash the validator for bad behavior for example not submitting blocks correctly or multiples times etc..). This will result by removing a part of the Stake of the validator (like a fine).

In our case an attacker can decide to DoS repetitively a validator, thus the validator will never submit any blocks and can be punished by the protocol in the future, also worth noting that the validator won't perceived any reward from the protocol also because the validator will be offline or in bad state (due to the Eve DoS) thus the validators won't be able to submit block correctly.

2/ Attacking Observers Lead To Funds Lost.

A/ Migration.

As we already discussed, in the finding https://github.com/code-423n4/2023-11-zetachain-findings/blob/main/data/MevSec-Q.md#low-100% -of -the -validators -must -agree -for a TSS -a-tss-migration. the migration process require 100% (all the observers) to be in consensus to accept the migration. In this case a malicious actor can DoS the Observer repetitively to block the migration to happen.

B/ CCTXs Never Processed

As the observers are known from the network. A malicious actor can decide to perform a repetitively DoS attack on all the observers validators during a CCTX is processing. Thus, the CCTX will be lost and the user transaction will never be proceed.

[0xean]

leaving as judged.