tinboxw
commented
2 years ago 我基于main_epoll.c例子，编写了一个基于epoll的异步tcp客户端，是可以连接成功-发送接收数据-正常关闭。
Open MincYu opened 2 years ago
tinboxw
commented
2 years ago 我基于main_epoll.c例子，编写了一个基于epoll的异步tcp客户端，是可以连接成功-发送接收数据-正常关闭。
jiangxiaosheng
commented
2 years ago 我基于main_epoll.c例子，编写了一个基于epoll的异步tcp客户端，是可以连接成功-发送接收数据-正常关闭。
any chance you could share your code and the way you establish the connection?
BaiXious
commented
2 years ago 我基于main_epoll.c例子，编写了一个基于epoll的异步tcp客户端，是可以连接成功-发送接收数据-正常关闭。
any chance you could share your code and the way you establish the connection?Please
ChiaoTeo
commented
1 year ago 可以分享一下代码吗
我基于main_epoll.c例子，编写了一个基于epoll的异步tcp客户端，是可以连接成功-发送接收数据-正常关闭。
zhubingbing
commented
1 year ago @jiangxiaosheng 能分享下例子吗
Chithesus
commented
4 months ago Hi,
To understand the performance benefits of f-stack, I started with a simple application where a client interacts with a server via TCP. Both client and server run on aws ec2 instances, each having two NICs where one for kernel driver and another for DPDK driver.
I followed the helloworld example to write a f-stack server, and it works as expected. However, the f-stack client cannot connect to any server (neither my f-stack server nor others). In particular,
ff_connectfailed and the socket gets blocked in SYN_SENT state.To figure out the reason, I tested normal client and server (based on kernel stack) as well. When running normal client/server with f-stack server/client, I can track the tcp packages on normal ones to help understand the behavior of f-stack client (while currently I can only see the byte-level traffic of the DPDK NIC on ec2 instance using f-stack tools without high-level semantics, that's why I need normal client/server). Anyway, it turns out that the server can receive the SYN package from f-stack client and send out SYN_ACK, while the latter fails to receive SYN_ACK and always re-sends SYN to the server.
I feel that the problem can either come from routing issues or the f-stack client somehow dropping the package. To exclude the routing issue, I inspected the client-side traffic of both NICs, and found the DPDK NIC DID receive packages during the tcp handshake (based on
rx packetsandrx bytesfromtrafficcommand) and there is no package coming into the kernel NIC (so the routing might be correct). I thus doubt the second reason might be more possible.Still, I'm not sure if my understanding is correct and how can I fix my problem. Here are a couple of questions. If I can inspect more package-level information (like
tcpdump) with f-stack tools, instead of only the numbers of bytes and packages received and sent? If the f-stack client does drop the package, why it might happen and how can I solve this?Any help will be greatly appreciated. Let me know in case more details are required.
I capture all the package from client and server, client send SYN to server and server reply a message, but client send SYN again with tcpdump; but bind certain port can establish tcp connection;
vewe-richard
commented
1 month ago @Chithesus Can you show me the code? I tried to bind certain port but still didn't working.
` memset(&client_addr, 0, sizeof(client_addr)); client_addr.sin_family = AF_INET; client_addr.sin_port = htons(CLIENT_PORT); client_addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (ff_bind(sockfd, (struct linux_sockaddr *)&client_addr, sizeof(client_addr)) < 0) {
printf("ff_bind failed\n");
ff_close(sockfd);
return -1;
}`
Chithesus
commented
1 month ago Thanks for your mail.邮件已收，谢谢
vewe-richard
commented
1 month ago @tinboxw Can you share the code?
vewe-richard
commented
1 month ago I fixed it, we should disable kni in config.ini
Below is the code implement a client
`#include
static struct epoll_event ev, events[MAX_EVENTS];
int epfd;
int sockfd;
//#define STANDARD
int loop(void *arg)
{
int nfds;
char send_buf[] = "Hello, Server!";
char recv_buf[BUFFER_SIZE];
nfds = epoll_wait(epfd, events, MAX_EVENTS, -1); nfds = ff_epoll_wait(epfd, events, MAX_EVENTS, -1); if (nfds < 0) {
printf("ff_epoll_wait failed\n");
} else if(nfds != 0){
printf("got events, nfds %d\n", nfds);
}
for (int i = 0; i < nfds; ++i) {
if (events[i].events & EPOLLOUT) {
printf("before write\n"); if (write(sockfd, send_buf, strlen(send_buf)) < 0) { if (ff_write(sockfd, send_buf, strlen(send_buf)) < 0) { printf("ff_write failed: %s\n", strerror(errno));
} else {
ev.events = EPOLLIN;
printf("write a buffer\n"); epoll_ctl(epfd, EPOLL_CTL_MOD, sockfd, &ev); ff_epoll_ctl(epfd, EPOLL_CTL_MOD, sockfd, &ev); }
} else if (events[i].events & EPOLLIN) { int len = read(sockfd, recv_buf, BUFFER_SIZE - 1); int len = ff_read(sockfd, recv_buf, BUFFER_SIZE - 1); if (len > 0) {
recv_buf[len] = '\0'; // 添加字符串结束符
printf("Received from server: %s\n", recv_buf);
} else {
if (len < 0) {
printf("ff_read failed: %s\n", strerror(errno));
} else {
printf("Connection closed by server\n");
} close(sockfd); ff_close(sockfd); break;
}
} else if (events[i].events & (EPOLLHUP | EPOLLERR)) {
printf("epoll error\n"); close(sockfd); ff_close(sockfd); break;
}
}}
int main(int argc, char *argv[]) { int nfds; struct sockaddr_in server_addr; int on = 1;
ff_init(argc, argv);sockfd = socket(AF_INET, SOCK_STREAM, 0);sockfd = ff_socket(AF_INET, SOCK_STREAM, 0);if (sockfd < 0) {
printf("ff_socket failed\n");
return -1;
}ioctl(sockfd, FIONBIO, &on);ff_ioctl(sockfd, FIONBIO, &on);memset(&server_addr, 0, sizeof(server_addr));
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(SERVER_PORT);
server_addr.sin_addr.s_addr = inet_addr(SERVER_IP);
printf("just before ff_connect\n");//sleep(10);if (connect(sockfd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {if (ff_connect(sockfd, (struct linux_sockaddr *)&server_addr, sizeof(server_addr)) < 0) { if (errno != EINPROGRESS) {
printf("ff_connect failed: %s\n", strerror(errno)); close(sockfd); ff_close(sockfd); return -1;
} else {
printf("EINPROGRESS\n");
}
}
printf("end ff_connect\n");epfd = epoll_create(1);epfd = ff_epoll_create(1);if (epfd < 0) {
printf("ff_epoll_create failed\n"); close(sockfd); ff_close(sockfd); return -1;
}
ev.events = EPOLLOUT | EPOLLIN;
ev.data.fd = sockfd;if (epoll_ctl(epfd, EPOLL_CTL_ADD, sockfd, &ev) < 0) {if (ff_epoll_ctl(epfd, EPOLL_CTL_ADD, sockfd, &ev) < 0) { printf("ff_epoll_ctl failed\n"); close(sockfd); ff_close(sockfd); return -1;
}while (1) {
loop(NULL);
}ff_run(loop, NULL);while (1) { nfds = epoll_wait(epfd, events, MAX_EVENTS, -1); nfds = ff_epoll_wait(epfd, events, MAX_EVENTS, -1); if (nfds < 0) {
printf("ff_epoll_wait failed\n");
if(nfds != 0) printf("got events, nfds %d\n", nfds);
for (int i = 0; i < nfds; ++i) {
if (events[i].events & EPOLLOUT) {
printf("before write\n"); if (write(sockfd, send_buf, strlen(send_buf)) < 0) { if (ff_write(sockfd, send_buf, strlen(send_buf)) < 0) { printf("ff_write failed: %s\n", strerror(errno));
} else {
ev.events = EPOLLIN;
printf("write a buffer\n"); epoll_ctl(epfd, EPOLL_CTL_MOD, sockfd, &ev); ff_epoll_ctl(epfd, EPOLL_CTL_MOD, sockfd, &ev); }
} else if (events[i].events & EPOLLIN) { int len = read(sockfd, recv_buf, BUFFER_SIZE - 1); int len = ff_read(sockfd, recv_buf, BUFFER_SIZE - 1); if (len > 0) {
recv_buf[len] = '\0'; // 添加字符串结束符
printf("Received from server: %s\n", recv_buf);
} else {
if (len < 0) {
printf("ff_read failed: %s\n", strerror(errno));
} else {
printf("Connection closed by server\n");
} close(sockfd); ff_close(sockfd); goto cleanup;
}
} else if (events[i].events & (EPOLLHUP | EPOLLERR)) {
printf("epoll error\n"); close(sockfd); ff_close(sockfd); goto cleanup;
}
}
}
}cleanup:
close(epfd);ff_close(epfd);return 0;} `
Hi,
To understand the performance benefits of f-stack, I started with a simple application where a client interacts with a server via TCP. Both client and server run on aws ec2 instances, each having two NICs where one for kernel driver and another for DPDK driver.
I followed the helloworld example to write a f-stack server, and it works as expected. However, the f-stack client cannot connect to any server (neither my f-stack server nor others). In particular,
ff_connectfailed and the socket gets blocked in SYN_SENT state.To figure out the reason, I tested normal client and server (based on kernel stack) as well. When running normal client/server with f-stack server/client, I can track the tcp packages on normal ones to help understand the behavior of f-stack client (while currently I can only see the byte-level traffic of the DPDK NIC on ec2 instance using f-stack tools without high-level semantics, that's why I need normal client/server). Anyway, it turns out that the server can receive the SYN package from f-stack client and send out SYN_ACK, while the latter fails to receive SYN_ACK and always re-sends SYN to the server.
I feel that the problem can either come from routing issues or the f-stack client somehow dropping the package. To exclude the routing issue, I inspected the client-side traffic of both NICs, and found the DPDK NIC DID receive packages during the tcp handshake (based on
rx packetsandrx bytesfromtrafficcommand) and there is no package coming into the kernel NIC (so the routing might be correct). I thus doubt the second reason might be more possible.Still, I'm not sure if my understanding is correct and how can I fix my problem. Here are a couple of questions. If I can inspect more package-level information (like
tcpdump) with f-stack tools, instead of only the numbers of bytes and packages received and sent? If the f-stack client does drop the package, why it might happen and how can I solve this?Any help will be greatly appreciated. Let me know in case more details are required.