bogdanfinn / tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.
BSD 4-Clause "Original" or "Old" License
670 stars 133 forks source link

[Bug]: Memory issues with nodejs ffi-napi #60

Closed 3ldar closed 11 months ago

3ldar commented 11 months ago

TLS client version

v.1.3.2

System information

node:current-bullseye image vCPUs based on AMD

Issue description

I run a node js application in a docker environment using current:bullseye image. The application creates multiple forks of itself. Every fork has one TLSClient (TLS client based on the typescript client) active and has some proxy rotation as I read in the comments every proxy change causes a new TLSClient to be created so I make sure to call destorySessionasync and destroyAllAsync before moving to a new proxy URL but the memory consumption always increases until we get out of memory.

Steps to reproduce / Code Sample

This is part of my proxy rotation and session destruction code :

        await this.proxyManager.initProxies(this.proxyPools);
        this.options = this.proxyManager.randomizeOptions(); // this generates the new sessionId
        this.options.tlsPayload = this.proxyManager.getTlsClientPayload(); // this will assign the new sessionId
        if (this.currentSession) { // as the proxy URL change destroy the previous session
            await this.tlsClient.destroySession({sessionId: this.currentSession});
        }
        await this.tlsClient.destroyAllAsync(); // destroy everyting

        this.currentSession = this.options.sessionId; // keep the sessionId for the next destruction
bogdanfinn commented 11 months ago

hey @3ldar what you are doing with the session / destroying the session is in general correct. The point you are missing is freeing the memory allocated for passing the data from the shared lib to your node application.

Let me try to explain:

When you call something from the DLL you receive a response (as string) which can be parsed to json in the happy path. You might have noticed that every Response received from the DLL has a "id" property with some uuid. Even the error responses. This Response is send from the DLL to your application as a C.CString.

// Go string to C string
// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CString(string) *C.char 

So after you received your response you need to let the DLL know when to free the allocated memory. Similar to the methods for destroying a session or all sessions we have a method called freeMemory() which takes just the response id as a string as parameter. See example in js:

// call the library with the requestPayload as string
const response = tlsClientLibrary.request(JSON.stringify(requestPayload));

// convert response string to json
const responseObject = JSON.parse(response)

// free the allocated memory for the response above by passing the response id to the freeMemory Call
tlsClientLibrary.freeMemory(responseObject.id)

I know it is a bit ugly but until now we do not have a better solution.

3ldar commented 11 months ago

// free the allocated memory for the response above by passing the response id to the freeMemory Call
tlsClientLibrary.freeMemory(responseObject.id)

Darn, I remember I saw something similar bu I didn't pay that much attention. Thank you for your time for answering this.