jorian commented 1 year ago

I have this strange behaviour with a Discord bot that I'm currently making.

For a cryptocurrency project, I am using rust-jsonrpc as a client to query that cryptocurrency's blockchain. It uses the same RPC mechanism that Bitcoin uses, so I'm using this crate and it has been working fine for quite some time.

Because I have to do many RPCs, I store an instance of a rust-jsonrpc client as global data of the bot. This global data is essentially a Box::pin(async move { Data } ) where I store the client as a member of Data. (the bot is made using poise, and in this setup function is where Data gets initiated)

When I start the bot and keep it running for a while (30+ sec) and get the client from global data and do a getnewaddress RPC, I am getting a HttpResponseTooShort error: error: JsonRPC(Transport(HttpResponseTooShort { actual: 0, needed: 12 })). It happens every time I use this call after at least 30 seconds of idling and I can't figure out why this happens. A second request within 30 seconds works just fine.

I've already checked if the problem is keeping a client around for a while:

    let client = vrsc_rpc::Client::vrsc(true, Auth::ConfigFile).unwrap();

    thread::sleep(Duration::from_secs(50));

    dbg!(client.get_new_address().unwrap());

(I built a wrapper like https://github.com/rust-bitcoin/rust-bitcoincore-rpc for this cryptocurrency project)

This doesn't appear to be a problem. I can't figure out what is, though. My guess would be that I keep a connection open for too long, but I can't figure this out, as it appears to be working in a separate environment.

Any ideas?

dpc commented 1 year ago

Sounds like a connection reuse issue after timeout.

80 possibly related?

apoelstra commented 1 year ago

I agree, it sounds like we're failing to detect that the socket has died. When we do the POST request we should be detecting errors (on the second write) and retrying.

From http://www.softlab.ntua.gr/facilities/documentation/unix/unix-socket-faq/unix-socket-faq-2.html

"If the peer calls close() or exits...I would expect EPIPE, not on the next call, but the one after."

jorian commented 1 year ago

I'm glad it looks like the issue is known. What would be the timeline for a fix?

apoelstra commented 1 year ago

This week.

jorian commented 1 year ago

What is the status on this?

apoelstra commented 1 year ago

No update. I've had it as an open tab for 8 weeks. It's near the top of my TODO list. Sorry.

apoelstra commented 1 year ago

@jorian opened #84 as a fix.

jorian commented 1 year ago

Thanks! No need to say sorry :D

jorian commented 1 year ago

I checked out the branch that this PR uses and use it in my Cargo.toml:

jsonrpc = {git="https://github.com/apoelstra/rust-jsonrpc", branch = "2023-01--detect-epipe", version = "0.14.0"}

Unfortunately, I still get the error message: RPC error: transport error: HTTP response too short: length 0, needed 12.

EDIT: I just noticed it got merged to master branch, but it seems to be needing some more investigation.

apoelstra commented 1 year ago

Damn. We haven't merged it yet (but we will, since I've run into the issue that it fixes). Frustrating that it doesn't fix your issue.

Are you able to use tcpdump or something to get more information?

jorian commented 1 year ago

Well I decided to just stop reusing clients for now as there is no requirement for me to do so.

I don't have the skills to use tcpdump to find out more, sorry.

apoelstra / rust-jsonrpc

HttpResponseTooShort for first request after a while #79

80 possibly related?