Fix event dispatcher stall

[obycode]

Description

Update event dispatcher to use standard Rust to send payloads, replacing the use of async-h1 which was causing problems because connect cannot timeout.

Applicable issues

• fixes #5097

Additional info (benefits, drawbacks, caveats)

This did require manually implementing the HTTP protocol, so does open the possibility for some new errors there. I added integration tests using a tiny-http server to help verify. I've also run a local network complete with signers and API as event observers and everything works as expected.

[jcnelson]

@obycode @jbencin I rewrote send_request() to use the Stacks HTTP parsing libraries and networking stack. I had to make a modest patch to httpcore.rs so it could be used on arbitrary HTTP requests without needing to register a dedicated request handler, but it's no biggie. It's tested via the tests @obycode wrote for event dispatcher (some of which could move to stackslib/src/net/tests if you think that'd be more appropriate).

Also, this closes #3851

[obycode]

Thanks Jude! I'm seeing this when I try to run this version. I'm not sure yet why.

2024-08-22 23:02:57 WARN [1724382177.529619] [stackslib/src/net/connection.rs:633] [main] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type"))
2024-08-22 23:02:57 WARN [1724382177.530137] [testnet/stacks-node/src/event_dispatcher.rs:584] [main] Event dispatcher: connection or request failed to stacks-api:3700 - Custom { kind: Other, error: "invalid message (malformed or bad signature): \"Failed to receive socket data\"" }

[obycode]

Thanks Jude! I'm seeing this when I try to run this version. I'm not sure yet why.

2024-08-22 23:02:57 WARN [1724382177.529619] [stackslib/src/net/connection.rs:633] [main] Invalid message preamble: Failed to decode HTTP request or HTTP response (request error: DeserializeError("Failed to parse HTTP request: Version"); response error: DeserializeError("Unsupported HTTP content type"))
2024-08-22 23:02:57 WARN [1724382177.530137] [testnet/stacks-node/src/event_dispatcher.rs:584] [main] Event dispatcher: connection or request failed to stacks-api:3700 - Custom { kind: Other, error: "invalid message (malformed or bad signature): \"Failed to receive socket data\"" }

The content type is application/json; charset=utf-8 which is not handled properly.

[obycode]

The content type is application/json; charset=utf-8 which is not handled properly.

I pushed a fix for this in 67a9d791223bf1586ba6637ee297847130145a17. It seems to be working now.

[obycode]

I tested this out a bit in a regtest network by pausing the API node, so that the event observer sends would timeout. I saw the stacks-node getting the timeout messages as expected, and it recovered when I unpaused the API!