golang It will scan in a loop, which will cause the CPU to be intermittently high

[situnan]

Problems with the use of data structures：10000 connect ，transports data is huge。and golang It will scan in a loop, which will cause the CPU to be intermittently high

transports: make(map[string]Transport),

func NewTransportLayer( dnsResolver net.Resolver, sipparser Parser, tlsConfig tls.Config, ) TransportLayer { l := &TransportLayer{ transports: make(map[string]Transport), listenPorts: make(map[string][]int),

[situnan]

GC loop make(map[string]Transport)

[emiago]

Unless you explain this better, I am closing this. Thanks

[situnan]

1) we test 100000 tcp/tls client to server（no data to server,only connect）. server memory is too high. 2)then CPU occasionally high（ top -d 1 ）,reason is golang gc to (map[string]Transport)

[emiago]

Can you send your test command, or what tool are you using

[situnan]

server：go build ./example/register/server/main.go ./main -ip=192.168.121.107:5061 -t=tls -tlskey=xxx -tlscrt=xxx

client（code see）：ulimit -n 10240 && ./test own 192.168.121.107 5061 10000

include

include

include

include <sys/socket.h>

include

include

include <netinet/in.h>

include <arpa/inet.h>

include

include <sys/types.h>

include <sys/stat.h>

include

include <openssl/ssl.h>

include <openssl/err.h>

include

define MAXBUF 1024

define TEST_MAX_COUNT 102400

int test(char server_name, char server_ip, int server_port, int count) { char sending_buf[MAXBUF+1]; struct sockaddr_in dest; char buffer[MAXBUF+1]; SSL_CTX ctx=NULL; SSL ssl[TEST_MAX_COUNT]={NULL}; int sock[TEST_MAX_COUNT]={-1}; int res; int i;

if (count > TEST_MAX_COUNT) count = TEST_MAX_COUNT;
bzero(&dest, sizeof(dest));
dest.sin_family=AF_INET;
dest.sin_port=htons(server_port);
if (inet_aton(server_ip,(struct in_addr *)&dest.sin_addr.s_addr) == 0) {
    perror(server_ip);
    return -1;
}
ctx=SSL_CTX_new(TLSv1_2_client_method());
if (ctx == NULL) {
    ERR_print_errors_fp(stdout);
    return -1;
}
// establish the tls connections
for ( i = 0; i < count; i ++ ) {
    if ((sock[i]=socket(AF_INET, SOCK_STREAM, 0)) < 0) {
        perror("Socket");
        continue;
    }
    fprintf(stdout,"socket[%d] created\n", i);
    if (connect(sock[i],(struct sockaddr *)&dest,sizeof(dest)) != 0) {
        perror("connect");
        close(sock[i]);
        continue;
    }
    fprintf(stdout,"server[%d] connected\n\n", i);
    ssl[i]=SSL_new(ctx);
    SSL_set_fd(ssl[i],sock[i]);
    SSL_set_tlsext_host_name(ssl[i], server_name);
    if (SSL_connect(ssl[i]) == -1) {
        ERR_print_errors_fp(stderr);
        SSL_shutdown(ssl[i]);
        SSL_free(ssl[i]);
        ssl[i] = NULL;
        close(sock[i]);
        sock[i] = -1;
        continue;
    }
    else {
        fprintf(stdout,"Connected with %s encryption\r\n", SSL_get_cipher(ssl[i]));
    }
}

fprintf(stdout,"wait for 3600s ...\r\n");
sleep(3600);
for ( i = 0; i < count; i ++ ) {
    if (ssl[i]) {
        SSL_shutdown(ssl[i]);
        SSL_free(ssl[i]);
    }
    if (sock[i] >= 0) {
        close(sock[i]);
    }
}
if (ctx) {
    SSL_CTX_free(ctx);
}
return 0;

}

int main(int argc, char *argv[]) { int ssl_version = 0; if (argc != 5) { fprintf(stdout,"%s [server name] [server ip] [server port] [count]\r\n",argv[0]); fprintf(stdout,"current openssl version %x\r\n", OPENSSL_VERSION_NUMBER); return -1; } SSL_library_init(); OpenSSL_add_all_algorithms(); SSL_load_error_strings(); test(argv[1], argv[2], atoi(argv[3]),atoi(argv[4])); fprintf(stdout,"bye!!!\r\n"); return 0; }

[situnan]

The test also found an issue： the main service process memory did not drop and did not fall back to the original value when all connections had been dropped

[emiago]

Hi @situnan thank for reporting. There are no timeouts currently, I will improve this. Still opening connections I fail to see what has todo with transport layer. I will try to replicate.

[emiago]

@situnan I see nothing wrong here, at least no gc loop can triggered (Specially on that map). I hope this is not some LLM code scanning! Dropping connection, returns memory and connection pool is empty. GC must be triggered for cleanup! As for more control over net Listener, it can be passed directly, for safer use cases. Example used here is using basic listener. Still this does not cause any issue here mentioned.

Unless some profiling or demo of this can be provided, there is no way forward. Feel free to send PR for fixing this.