search

apache / jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services
https://jmeter.apache.org/
Apache License 2.0
8.3k stars 2.09k forks source link

An OOM occurred when uploading the file after executing tens of millions #5816

Closed wangchy0927 closed 1 year ago

wangchy0927 commented 1 year ago

Expected behavior

No response

Actual behavior

No response

Steps to reproduce the problem

1、BeanShell PreProcessor

String zipPath = JmeterMain.createNewFile();
vars.put("zip",zipPath);

this file about 1kb

2、HTTP Request image

3、After executing tens of millions，OOM image

JMeter Version

5.2.1

Java Version

java version "1.8.0_191"

OS Version

No response

milamberspace commented 1 year ago

How many virtual user have you in your script? What is the value for the heap Java (-Xmx param) in jmeter.bat, default value?

FSchumacher commented 1 year ago

Please tell us, what you do in JmeterMain#createNewFile. If you are registering files via FileServer, you have to make sure to release them, once they are not needed anymore. Otherwise you might explode your heap, as your memory dump shows.

wangchy0927 commented 1 year ago

Please tell us, what you do in JmeterMain#createNewFile. If you are registering files via FileServer, you have to make sure to release them, once they are not needed anymore. Otherwise you might explode your heap, as your memory dump shows.

set HEAP=-Xms4g -Xmx4g -XX:MaxMetaspaceSize=256m

I registering files viajava.io.FileInputStream， Not using any Jmeter methods 。Last night i use fileServer.closeFiles(); to release is ok now, I guess there was a problem with Jmeter uploading the byte stream. The file in the store before the upload started, but there was no automatic release after the upload ended

image

wangchy0927 commented 1 year ago

How many virtual user have you in your script? What is the value for the heap Java (-Xmx param) in jmeter.bat, default value?

set HEAP=-Xms4g -Xmx4g -XX:MaxMetaspaceSize=256m

FSchumacher commented 1 year ago

Please tell us, what you do in JmeterMain#createNewFile. If you are registering files via FileServer, you have to make sure to release them, once they are not needed anymore. Otherwise you might explode your heap, as your memory dump shows.

set HEAP=-Xms4g -Xmx4g -XX:MaxMetaspaceSize=256m

I registering files viajava.io.FileInputStream， Not using any Jmeter methods 。Last night i use fileServer.closeFiles(); to release is ok now, I guess there was a problem with Jmeter uploading the byte stream. The file in the store before the upload started, but there was no automatic release after the upload ended

image

FileServer is meant for sharing files between threads and was designed to handle a small number of files. At the time, it was created a few thousand threads was the maximum you could dream of. Multiply that by a few files per thread (it was mainly used for CSV dataset configs, which there are normally not that many per test) and you get a low 10.000 number of files. Your use case with millions of files is not a use case, it was thought for. So either clean up regularly the files and keep the numbers low (in FileServer), or use other means to handle your files. Know, that each open filehandle will consume memory. If you don't need it open, close it.

All in all, I read your comment, as you have solved your problems with heap consumption, so I close this issue. Feel free to reopen it, if you still have questions about this.