sock_recv always fails with bad fd

[tiran]

Test Case

recv.zip

Steps to Reproduce

• wasmtime run --tcplisten 127.0.0.1:8080 recv.wasm
• curl http://localhost:8080

Expected Results

I expect that recv() can read from the fd that has been returned from accept.

Actual Results

recv() calls always fail with Bad file descriptor. Output from my reproducer:

accept() on fd 3
...
accept returned fd 4
recv -1
recv(4, ...) failed: Bad file descriptor (8)

Versions and Environment

Wasmtime version or commit: 0.39.1 and 418dbc15bd2a5269b338587661387e05fc77b983

Operating system: Linux

Architecture: x86_64

Extra Info

I did a bit of debugging with rust-gdb. It looks like wasi_common::snapshots::preview_1::sock_recv calls wasi_common::file::WasiFile::sock_recv which always fails with badf. sock_accept on the other hand is provided by wasi_cap_std_sync::net (crates/wasi-common/cap-std-sync/src/net.rs).

(gdb) bt
#0  wasi_common::file::WasiFile::sock_recv<wasi_cap_std_sync::net::TcpStream> (self=0x555557ab9230, _ri_data=&mut [std::io::IoSliceMut](size=1) = {...}, _ri_flags=...)
    at crates/wasi-common/src/file.rs:32
#1  0x0000555555c2d01f in wasi_common::snapshots::preview_1::{impl#19}::sock_recv::{async_block#0} () at crates/wasi-common/src/snapshots/preview_1.rs:1268
#2  0x0000555555c5e64c in core::future::from_generator::{impl#1}::poll<wasi_common::snapshots::preview_1::{impl#19}::sock_recv::{async_block_env#0}> (self=..., 
    cx=0x7fffffff5718) at /builddir/build/BUILD/rustc-1.63.0-src/library/core/src/future/mod.rs:91
#3  0x0000555555a0504f in core::future::future::{impl#1}::poll<alloc::boxed::Box<(dyn core::future::future::Future<Output=core::result::Result<(u32, wasi_common::snapshots::preview_1::types::Roflags), anyhow::Error>> + core::marker::Send), alloc::alloc::Global>> (self=..., cx=0x7fffffff5718)
    at /builddir/build/BUILD/rustc-1.63.0-src/library/core/src/future/future.rs:124
#4  0x0000555555af06d7 in wasi_common::snapshots::preview_1::wasi_snapshot_preview1::sock_recv::{async_block#0}<wasi_common::ctx::WasiCtx> ()
    at crates/wasi-common/src/snapshots/preview_1.rs:21
#5  0x0000555555a713e5 in core::future::from_generator::{impl#1}::poll<wasi_common::snapshots::preview_1::wasi_snapshot_preview1::sock_recv::{async_block_env#0}<wasi_common::ctx::WasiCtx>> (self=..., cx=0x7fffffff5718) at /builddir/build/BUILD/rustc-1.63.0-src/library/core/src/future/mod.rs:9

reproducer

#define _POSIX_C_SOURCE 200809L

#include <errno.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <time.h>
#include <unistd.h>

#define ACCEPT_FD 3

int main(void) {
    char buf[128] = {0};
    struct sockaddr_in addr;
    socklen_t addr_size = sizeof(struct sockaddr_in);
    size_t res;
    int fd = -1;
    struct timespec tv = {0, 500 * 1000 * 1000};

    printf("accept() on fd %i\n", ACCEPT_FD);
    while (fd == -1) {
        fd = accept(ACCEPT_FD, (struct sockaddr *)&addr, &addr_size);
        nanosleep(&tv, NULL); // hack
        printf(".");
        fflush(stdout);
    }
    printf("\naccept returned fd %i\n", fd);

    res = recv(fd, buf, sizeof(buf)-1, 0);
    printf("recv %zd\n", res);
    if (res == -1) {
        printf("recv(%i, ...) failed: %s (%i)\n", fd, strerror(errno), errno);
        return 2;
    } else {
        printf("buf %.*s (%zd)", (int)res, buf, res);
        return 0;
    }
}

[bjorn3]

I think you have to use read and write to read from and write to sockets.

[sunfishcode]

@bjorn3 sock_recv and sock_send correspond to POSIX recv and send, and are expected to work with sockets. It seems likely something else is going on here.

[tiran]

My example works when I replace recv() with read(). But it's kinda the wrong API call. Portable applications like Python interpreter use recv() because read() does not work with sockets on Windows. Windows treats file handles and socket handles differently.

[bjorn3]

@bjorn3 sock_recv and sock_send correspond to POSIX recv and send, and are expected to work with sockets. It seems likely something else is going on here.

Rust's libstd uses fd_read and fd_write for WASI, not sock_recv and sock_send. Wasmtime also doesn't seem to implement sock_recv and sock_send at all.

[tiran]

I'm using WASI-SDK 16 to compile a portable C application to wasm32-wasi. To be more precise I want to get CPython's socket module working under WASI. It uses recv and send to read to and write from a socket.

[sunfishcode]

Ah, I was mistaken. Wasmtime has some code for sock_send and sock_recv, but they're not fully implemented.

I've now submitted https://github.com/bytecodealliance/wasmtime/pull/4776 to implement them. With that patch, the reproducer reported above compiles and produces the expected output.

$ target/debug/wasmtime run --tcplisten 127.0.0.1:8080 recv.wasm 
accept() on fd 3
.....
accept returned fd 4
recv 78
buf GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.81.0
Accept: */*

 (78)