Open xiamoplus opened 1 year ago
Update lib pip install --force-reinstall tls-client
Also, having a memory leak issue. Can't seem to fix it :/
The memory leak is still present. The issue can be reproduced with the following code. I have hosted a local httpbin server and sent some large random data as JSON to the library. The memory in the first iteration is around 36 MB which gradually increases to more than 250MB.
import tls_client
from memory_profiler import profile
@profile
def send_requests():
session = tls_client.Session(
client_identifier="chrome112",
random_tls_extension_order=True
)
url = 'http://0.0.0.0:8080/anything'
random_data = ["a"] * 10 ** 4
res = session.post(url=url, json=random_data)
json = res.json()
print(len(json))
for i in range(100000):
send_requests()
@sumeshmurali Haven't checked how tls_client
handles session closing, but locally I was able to reproduce the issue with your code. Memory for me increased from 55MB -> 283MB (10000) requests. However, after adding session close the memory seems to stabilize. Could you try it on your side and confirm or deny if this works?
Edit: After testing it on a real and relatively heavy page (15MB, used GET
instead of POST
) the leak seems to still be present even with session.close()
.
import tls_client
from memory_profiler import profile
@profile
def send_requests():
session = tls_client.Session(
client_identifier="chrome112",
random_tls_extension_order=True
)
url = 'http://localhost:8000/anything'
random_data = ["a"] * 10 ** 4
try:
res = session.post(url=url, json=random_data)
print(res.status_code)
finally:
session.close()
for i in range(10000):
send_requests()
Try add force_http1 tls_client.Session(client_identifier="firefox119",force_http1=True)
In golang version it help me solv this issue
@OxynDev the issue being talked about is a memory leak. forcing the client to use http/1.1 would work. just not smart if you're trying to replicate a browser which this client is made for. depending on what backend you're sending these requests to. most will support this proto, but likely stuff wont work as intended. as servers will expect a browser to do http requests over newer protos
Hello, I saw that the memory leak was fixed by a recent pull request, but I still get an error from it. Any idea on how to avoid it? thx