apache / jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services
https://jmeter.apache.org/
Apache License 2.0
8.3k stars 2.09k forks source link

Save Responses to File generates bad paths and names #4393

Open asfimport opened 7 years ago

asfimport commented 7 years ago

toogoodtopassup (Bug 61118): Using the Save Responses to File is really inconsistent to the point that it is unusable.

Using all the default options except where noted and making http requests, here are the outcomes:

Completely blank options - nothing filled in

Files are placed in the JMeter bin directory of the form: 1.html 2.html etc.

The problem: The JMeter bin directory should never be the current directory for any file interaction - It should always default to the same directory as the current jmx file.

Use '~/' as the filename prefix

Files are written to the directory above the directory containing the .jmx file (its parent-parent directory). These files are named like this: [parent-parent directory name]1.html [parent-parent directory name]2.html etc.

The Problem: Using ~/ is the (somewhat) well advertised way to refer to the directory containing the jmx file, however, it doesn't work and instead targets the directory above that. Really I believe this should target the current user's home directory, but that bridge seems to have been crossed.

Additionally, the file name is messed up in this case, pulling in the name of the JMX file's parent-parent directory as part of the generated filename.

Use '~/output/' as the filename prefix

Files are written to the directory containing the .jmx file (not any sort of /output subdirectory). These files are named like this: output1.html output2.html etc.

The problem: While this does now target the directory containing the JMX file, it incorrectly adds 'output' to the file name rather than attempting to place the files in an /output directory despite the trailing slash.

Use '~/output/output/' as the filename prefix

Files are written to a /output subdirectory next to the JMX file. These files are named like this: output1.html output2.html etc.

The problem: 'output' is still added to the file name. Additionally, this will fail is a /output subdirectory does not exist - This may be by design but should be documented. It think it would be preferable to create the directory by default.

Note: Using the './' path results in the same behavior as using no filename prefix. Note: The single quotes in the filename prefix were not actually used.

The propose fix

In the case of the Save Responses to a File component specifically, there should be a separate directory property to configure where files should be written to so that users don't have to try to cram path into a file name.

Severity: normal OS: All

asfimport commented 7 years ago

@pmouawad (migrated from Bugzilla): Hello, Thanks for analysis which I share. Having used this component, it is true it's not clear at all where files are output.

Adding an output directory would make it much clearer.

asfimport commented 7 years ago

@pmouawad (migrated from Bugzilla): (In reply to toogoodtopassup from comment 0)

Using the Save Responses to File is really inconsistent to the point that it is unusable.

Using all the default options except where noted and making http requests, here are the outcomes:

Completely blank options - nothing filled in

Files are placed in the JMeter bin directory of the form: 1.html 2.html etc.

The problem: The JMeter bin directory should never be the current directory for any file interaction - It should always default to the same directory as the current jmx file.

As per docs, Filename Prefix is required so this test case may not be significant

Use '~/' as the filename prefix

Files are written to the directory above the directory containing the .jmx file (its parent-parent directory). These files are named like this: [parent-parent directory name]1.html [parent-parent directory name]2.html etc.

The Problem: Using ~/ is the (somewhat) well advertised way to refer to the directory containing the jmx file, however, it doesn't work and instead targets the directory above that. Really I believe this should target the current user's home directory, but that bridge seems to have been crossed.

Additionally, the file name is messed up in this case, pulling in the name of the JMX file's parent-parent directory as part of the generated filename.

The issue here is that your filename prefix is a folder prefix. So this seems to be an edgy case also.

Use '~/output/' as the filename prefix

Files are written to the directory containing the .jmx file (not any sort of /output subdirectory). These files are named like this: output1.html output2.html etc.

The problem: While this does now target the directory containing the JMX file, it incorrectly adds 'output' to the file name rather than attempting to place the files in an /output directory despite the trailing slash.

The issue here is that your filename prefix is a folder prefix. So this seems to be an edgy case also.

Use '~/output/output/' as the filename prefix

Files are written to a /output subdirectory next to the JMX file.
This looks ok as per dcos;

These files are named like this: output1.html output2.html etc.

The problem: 'output' is still added to the file name. Additionally, this will fail is a /output subdirectory does not exist - This may be by design but should be documented. It think it would be preferable to create the directory by default.

You're again in an edgy case as instead of providing a Filename prefix, you're providing a Folder prefix.

Note: Using the './' path results in the same behavior as using no filename prefix. Note: The single quotes in the filename prefix were not actually used.

The propose fix

  • Centralize all path resolution so that behaviors are consistent throughout JMeter (similar problems existing in other components like the CSV DataSet config).

  • Use ~/ to refer to the user's home directory This has a different meaning today shared with other components by the way. It means "the path is assumed to be relative to the JMX file location"

  • Use ./ to refer to the current directory, which would always be the same directory as the current JMX file.

In the case of the Save Responses to a File component specifically, there should be a separate directory property to configure where files should be written to so that users don't have to try to cram path into a file name.

I agree with this proposal but managing backward compatibility is a big annoying.