search

apache / jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services
https://jmeter.apache.org/
Apache License 2.0
8.24k stars 2.08k forks source link

POST multipart/form-data cURL code generated from Postman is not imported correctly #5463

Open asfimport opened 3 years ago

asfimport commented 3 years ago

eR@SeR (Bug 65013): Hello,

When POST multipart/form-data request is opened in Code/cURL mode in Postman, then entire lines copied and pasted into "Import from cURL" in JMeter, generated HTTP Sampler is not completely correct:

  1. Values in Parameters tab have redundant quotes
  2. Values in File upload tab/File path have redundant quotes
  3. MIME type value in File upload tab is application/octet-stream

If request with real URL and path is executed, the response is:

"java.io.FileNotFoundException: D:\jmxFileLocation"\D:\fileLocation\Desktop.jpg" (The filename, directory name, or volume label syntax is incorrect) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:213) at java.base/java.io.FileInputStream.<init>(FileInputStream.java:155) at org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:116) at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$ViewableFileBody.writeTo(HTTPHC4Impl.java:1513) at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134) at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157) at org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:113) at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152) at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$2.doSendRequest(HTTPHC4Impl.java:458) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:935) at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:646) at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:66) at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1296) at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1285) at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638) at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) at java.base/java.lang.Thread.run(Thread.java:830)"

If the same request is executed when quotes from all values in parameters and file path in file upload tab are removed, and changing application/octet-stream to image/jpeg, response is then OK.

Maybe for some other APIs application/octet-stream might work, but for mine, it doesn't. Is it possible to check the extension and generate the MIME type according to it?

Check attachment for cURL example and screenshots.

Postman 7.36.1 Jmeter 5.4.1 54e820e Microsoft Windows 10 Pro 64-bit java version "1.8.0_251"

Created attachment cURL and screenshots.zip: cURL and screenshots

Version: Nightly Severity: normal OS: All

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): The used curl command from the zip file is:

curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'

The filenames are placed together with their quotes into the fields in JMeter (as shown in the images in the zip).

The quotes from around the values of the form parameters are kept, too. (Shown again in the images inside the zip).

So there are two problems reported here:

1) quotes around the values are kept, but curl would drop them 2) the mime type is set to a very generel mime-type (probably curl is doing a better job at guessing the mime-type)

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Created attachment unquote-curl.diff: unquote form parameters

unquote-curl.diff ````diff diff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java index 8d95da29b7..8a937c16ab 100644 --- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java +++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java @@ -710,9 +710,9 @@ public class BasicCurlParser { String key = nameAndValue.substring(0, indexOfEqual).trim(); String value = nameAndValue.substring(indexOfEqual + 1).trim(); if ("form-string".equals(option.getDescriptor().getName())) { - request.addFormStringData(key, value); + request.addFormStringData(key, unquote(value)); } else { - request.addFormData(key, value); + request.addFormData(key, unquote(value)); } request.setMethod("POST"); } else if (option.getDescriptor().getId() == USER_AGENT_OPT) { @@ -955,7 +955,7 @@ public class BasicCurlParser { postdata = encodePostdata(postdata); } else { if (postdata.charAt(0) == '@' && !"data-raw".equals(dataOptionName)) { - postdata = postdata.substring(1, postdata.length()); + postdata = unquote(postdata.substring(1, postdata.length())); postdata = readFromFile(postdata); if (!"data-binary".equals(dataOptionName)) { postdata = deleteLineBreak(postdata); @@ -965,6 +965,15 @@ public class BasicCurlParser { return postdata; } + private String unquote(String value) { + LoggerFactory.getLogger(this.getClass()).info("Unquote {}", value, new RuntimeException("")); + if (value.charAt(0) == '"') { + String result = value.substring(1, value.length() - 1); + return result.replaceAll("\\\\(.)", "\\1"); + } + return value; + } + /** * Encode the post data * @@ -975,7 +984,7 @@ public class BasicCurlParser { if (postdata.contains("@")) { String contentFile = null; String[] arr = postdata.split("@", 2); - String dataToEncode = readFromFile(arr[1]); + String dataToEncode = readFromFile(unquote(arr[1])); try { contentFile = URLEncoder.encode(dataToEncode, StandardCharsets.UTF_8.name()); } catch (UnsupportedEncodingException e) { diff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java index 49ca3d8d68..43d6d44a9f 100644 --- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java +++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java @@ -526,7 +526,7 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato boolean isContainsFile = "@".equals(formValue.substring(0, 1)); boolean isContainsContentType = formValue.toLowerCase().contains(TYPE_FORM); if (isContainsFile) { - formValue = formValue.substring(1, formValue.length()); + formValue = unquote(formValue.substring(1, formValue.length())); String contentType; if (isContainsContentType) { String[] formValueWithType = formValue.split(TYPE_FORM); @@ -552,6 +552,13 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato } } + private String unquote(String substring) { + if (substring.charAt(0) == '"') { + return substring.substring(1, substring.length() - 1).replaceAll("\\\\(.)", "\\1"); + } + return substring; + } + private void createProxyServer(Request request, HTTPSamplerProxy httpSampler) { Map proxyServer = request.getProxyServer(); for (Map.Entry proxyPara : proxyServer.entrySet()) { diff --git a/src/protocol/http/src/test/java/org/apache/jmeter/curl/BasicCurlParserTest.java b/src/protocol/http/src/test/java/org/apache/jmeter/curl/BasicCurlParserTest.java index c9f92286e9..7bc9fd8eef 100644 --- a/src/protocol/http/src/test/java/org/apache/jmeter/curl/BasicCurlParserTest.java +++ b/src/protocol/http/src/test/java/org/apache/jmeter/curl/BasicCurlParserTest.java @@ -422,6 +422,39 @@ public class BasicCurlParserTest { "With method 'parser', we should post form data"); } + @Test + public void testFormWithQuotedValue() { + String cmdLine = "curl 'https://www.exaple.invalid/' " + + "--form 'test=\"something quoted\"'"; + BasicCurlParser basicCurlParser = new BasicCurlParser(); + BasicCurlParser.Request request = basicCurlParser.parse(cmdLine); + Map res = request.getFormData(); + assertEquals("something quoted", res.get("test"), + "With method 'form', we should post form data"); + } + + @Test + public void testFormWithQuotedValueWithQuotes() { + String cmdLine = "curl 'https://www.exaple.invalid/' " + + "--form 'test=\"something \\\"quoted\\\"\"'"; + BasicCurlParser basicCurlParser = new BasicCurlParser(); + BasicCurlParser.Request request = basicCurlParser.parse(cmdLine); + Map res = request.getFormData(); + assertEquals("something \"quoted\"", res.get("test"), + "With method 'form', we should post form data"); + } + + @Test + public void testFormWithQuotedFilename() { + String cmdLine = "curl 'https://www.exaple.invalid/' " + + "--form 'image=@\"/some/file.jpg\"'"; + BasicCurlParser basicCurlParser = new BasicCurlParser(); + BasicCurlParser.Request request = basicCurlParser.parse(cmdLine); + Map res = request.getFormData(); + assertEquals("/some/file.jpg", res.get("image"), + "With method 'form', we should post form data"); + } + @Test public void testFormString() { String cmdLine = "curl 'https://www.w3schools.com/html/tryit.asp?filename=tryhtml_form_submit/action_page.php' " ````
asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Can you test the next build from trunk or nightly and report back, whether it fixes your issue?

commit bb0c4557e10278f334844f8c1af873b708981b08 AuthorDate: Tue Dec 22 11:19:21 2020 +0100

POST multipart/form-data cURL code with quoted arguments is not imported correctly

https://github.com/apache/jmeter/issues/5463

.../jmeter/protocol/http/curl/BasicCurlParser.java | 17 ++++++++--- .../http/gui/action/ParseCurlCommandAction.java | 9 +++++- .../apache/jmeter/curl/BasicCurlParserTest.java | 34 ++++++++++++++++++++++ .../gui/action/ParseCurlCommandActionTest.java | 23 +++++++++++++++ xdocs/changes.xml | 1 + 5 files changed, 79 insertions(+), 5 deletions(-)

asfimport commented 3 years ago

eR@SeR (migrated from Bugzilla): Hi,

Quotes around parameter values are not generated anymore and if file extension is .png .jpg .jpeg .gif .avi (might be more) MIME type is properly recognized.

However, some commonly used extensions like .bmp .pdf .doc .docx .xls .xlsx .rar .zip etc. are still not recognized and generated MIME type is still application/octet-stream. You can check "Common MIME types" list on https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types and add all mentioned MIME types.

Besides this, I've found a few more findings that might be bugs:

  1. If parameter starts with @ like "@somevalue", form-data parameter is put inside Files upload tab instead to remain in Parameters tab.
  2. If same header or parameter name with same or different value is added, only last header or parameter is generated. All headers or parameters should be added even if they are duplicated.

cURL example:

curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.2' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpeg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="@ID_CARD"' \ --form 'BacksideImage=@"/C:/Users/n.aleksic/Desktop/BTT flow .xlsx"' \ --form 'ApplicationTypeId="11"'

DocumentType parameter moved to Files upload. Only one HTTP_X_FORWARDED_FOR header and ApplicationTypeId, BacksideImage parameters are imported.

And definitely the bug:

  1. If URL, header or parameter value is blank, cURL is not imported.

cURL example - URL:

curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'

Error:

ERROR o.a.j.p.h.g.a.ParseCurlCommandAction: Error creating test plan from cURL command:curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"', error:Unexpected format for command line:curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"', error:Unknown option -h java.lang.IllegalArgumentException: Unexpected format for command line:curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"', error:Unknown option -h at org.apache.jmeter.protocol.http.curl.BasicCurlParser.parse(BasicCurlParser.java:797) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.parseCommands(ParseCurlCommandAction.java:670) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.actionPerformed(ParseCurlCommandAction.java:619) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) ~[?:?] at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) ~[?:?] at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) ~[?:?] at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) ~[?:?] at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279) ~[?:?] at java.awt.Component.processMouseEvent(Component.java:6636) ~[?:?] at javax.swing.JComponent.processMouseEvent(JComponent.java:3342) ~[?:?] at java.awt.Component.processEvent(Component.java:6401) ~[?:?] at java.awt.Container.processEvent(Container.java:2263) ~[?:?] at java.awt.Component.dispatchEventImpl(Component.java:5012) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2321) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918) ~[?:?] at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547) ~[?:?] at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2307) ~[?:?] at java.awt.Window.dispatchEventImpl(Window.java:2762) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) ~[?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:745) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:743) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?] at java.awt.EventQueue.dispatchEvent(EventQueue.java:742) [?:?] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?] at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?] ERROR o.a.j.p.h.g.a.ParseCurlCommandAction: Error creating test plan from cURL command list:[curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'] java.lang.IllegalArgumentException: Unexpected format for command line:curl --location --request POST '' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"', error:Unknown option -h at org.apache.jmeter.protocol.http.curl.BasicCurlParser.parse(BasicCurlParser.java:797) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.parseCommands(ParseCurlCommandAction.java:670) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.actionPerformed(ParseCurlCommandAction.java:619) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) ~[?:?] at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) ~[?:?] at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) ~[?:?] at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) ~[?:?] at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279) ~[?:?] at java.awt.Component.processMouseEvent(Component.java:6636) ~[?:?] at javax.swing.JComponent.processMouseEvent(JComponent.java:3342) ~[?:?] at java.awt.Component.processEvent(Component.java:6401) ~[?:?] at java.awt.Container.processEvent(Container.java:2263) ~[?:?] at java.awt.Component.dispatchEventImpl(Component.java:5012) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2321) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918) ~[?:?] at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547) ~[?:?] at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2307) ~[?:?] at java.awt.Window.dispatchEventImpl(Window.java:2762) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) ~[?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:745) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:743) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?] at java.awt.EventQueue.dispatchEvent(EventQueue.java:742) [?:?] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?] at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]

cURL example - blank parameter:

curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId=""' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'

Error:

ERROR o.a.j.p.h.g.a.ParseCurlCommandAction: Error creating test plan from cURL command:curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId=""' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"', error:begin 0, end 1, length 0 java.lang.StringIndexOutOfBoundsException: begin 0, end 1, length 0 at java.lang.String.checkBoundsBeginEnd(String.java:3720) ~[?:?] at java.lang.String.substring(String.java:1909) ~[?:?] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.setFormData(ParseCurlCommandAction.java:526) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.createSampler(ParseCurlCommandAction.java:313) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.addToTestPlan(ParseCurlCommandAction.java:691) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.actionPerformed(ParseCurlCommandAction.java:636) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) ~[?:?] at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) ~[?:?] at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) ~[?:?] at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) ~[?:?] at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279) ~[?:?] at java.awt.Component.processMouseEvent(Component.java:6636) ~[?:?] at javax.swing.JComponent.processMouseEvent(JComponent.java:3342) ~[?:?] at java.awt.Component.processEvent(Component.java:6401) ~[?:?] at java.awt.Container.processEvent(Container.java:2263) ~[?:?] at java.awt.Component.dispatchEventImpl(Component.java:5012) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2321) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918) ~[?:?] at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547) ~[?:?] at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2307) ~[?:?] at java.awt.Window.dispatchEventImpl(Window.java:2762) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) ~[?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:745) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:743) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?] at java.awt.EventQueue.dispatchEvent(EventQueue.java:742) [?:?] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?] at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]

cURL example - blank header:

curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR;' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'

Error:

2020-12-23 02:24:10,285 ERROR o.a.j.p.h.g.a.ParseCurlCommandAction: Error creating test plan from cURL command list:[curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR;' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.avi"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId="13"' \ --form 'Country="ITA"' \ --form 'DocumentType="ID_CARD"'] java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 21 at java.lang.String.checkBoundsBeginEnd(String.java:3720) ~[?:?] at java.lang.String.substring(String.java:1909) ~[?:?] at org.apache.jmeter.protocol.http.curl.BasicCurlParser.parse(BasicCurlParser.java:693) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.parseCommands(ParseCurlCommandAction.java:670) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at org.apache.jmeter.protocol.http.gui.action.ParseCurlCommandAction.actionPerformed(ParseCurlCommandAction.java:619) ~[ApacheJMeter_http.jar:5.4.1-SNAPSHOT] at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) ~[?:?] at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) ~[?:?] at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) ~[?:?] at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) ~[?:?] at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279) ~[?:?] at java.awt.Component.processMouseEvent(Component.java:6636) ~[?:?] at javax.swing.JComponent.processMouseEvent(JComponent.java:3342) ~[?:?] at java.awt.Component.processEvent(Component.java:6401) ~[?:?] at java.awt.Container.processEvent(Container.java:2263) ~[?:?] at java.awt.Component.dispatchEventImpl(Component.java:5012) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2321) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918) ~[?:?] at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547) ~[?:?] at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488) ~[?:?] at java.awt.Container.dispatchEventImpl(Container.java:2307) ~[?:?] at java.awt.Window.dispatchEventImpl(Window.java:2762) ~[?:?] at java.awt.Component.dispatchEvent(Component.java:4844) ~[?:?] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) ~[?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:745) ~[?:?] at java.awt.EventQueue$5.run(EventQueue.java:743) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:391) [?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?] at java.awt.EventQueue.dispatchEvent(EventQueue.java:742) [?:?] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?] at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]

Jmeter 5.4.1 4ed732d Microsoft Windows 10 Pro 64-bit java version "1.8.0_251"

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): For the third bug report part, you could look at https://github.com/apache/jmeter/issues/5372 which is probably the same as your report.

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): For the second part (parameters which values start with @ landing in files upload tab), I believe, JMeter is doing the correct thing here.

The curl man page states in the '--form' section: "... To force the 'content' part to be a file, prefix the file name with an @ sign. ..."

If you want to send a value starting with an @ sign, you probably have to use --form-string as parameter. The curl man page for '-form-string' says: "... Similar to -F, --form except that the value string for the named parameter is used literally. Leading '@' and '<' characters, and the ';type=' string in the value have no special meaning. ..."

Maybe postman tries the value in the parameter, before it decides, whether this is a file or not. JMeter can't really decide this correctly, as the file has not to be present on creation of the test plan. It has to be present when the test plan is being run.

I am not sure, what to do about this, but it might be a good idea to discuss this on the mailing list and/or put this into its own bug report, as it will change the behaviour of detection.

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): With this path multiple headers/parameters with the same name will get added to the generated request.

It will also change the detection method for mimetypes to be hopefully a bit more precise. It uses Apache Tika and tries to look at the content of the file (if it can), as well as the extension of the filename.

Created attachment 0001-bug65024-curl-stuff-part-two.patch: Fixes for multi-header and -parameter and mime detection

0001-bug65024-curl-stuff-part-two.patch ````diff From 21a90ec85e3b2091de7b980cc7ad626a0b0ee05e Mon Sep 17 00:00:00 2001 From: Felix Schumacher Date: Wed, 23 Dec 2020 10:45:44 +0100 Subject: [PATCH] bug65024-curl-stuff-part-two --- src/protocol/build.gradle.kts | 8 ++++++ .../protocol/http/curl/BasicCurlParser.java | 25 ++++++++++--------- .../gui/action/ParseCurlCommandAction.java | 19 +++++++++----- 3 files changed, 34 insertions(+), 18 deletions(-) diff --git a/src/protocol/build.gradle.kts b/src/protocol/build.gradle.kts index d0855cff4a..263eec8df1 100644 --- a/src/protocol/build.gradle.kts +++ b/src/protocol/build.gradle.kts @@ -89,6 +89,14 @@ project("http") { implementation("com.fasterxml.jackson.core:jackson-databind") testImplementation(testFixtures(project(":src:testkit-wiremock"))) testImplementation("com.github.tomakehurst:wiremock-jre8") + // For some reason JMeter bundles just tika-core and tika-parsers without transitive + // dependencies. So we exclude those + implementation("org.apache.tika:tika-core") { + isTransitive = false + } + runtimeOnly("org.apache.tika:tika-parsers") { + isTransitive = false + } } } diff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java index 9471130243..4a78a5d29b 100644 --- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java +++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java @@ -43,6 +43,7 @@ import org.apache.commons.cli.avalon.CLArgsParser; import org.apache.commons.cli.avalon.CLOption; import org.apache.commons.cli.avalon.CLOptionDescriptor; import org.apache.commons.io.FileUtils; +import org.apache.commons.lang3.tuple.Pair; import org.apache.jmeter.protocol.http.control.AuthManager.Mechanism; import org.apache.jmeter.protocol.http.control.Authorization; import org.apache.jmeter.protocol.http.control.Cookie; @@ -123,7 +124,7 @@ public class BasicCurlParser { public static final class Request { private boolean compressed; private String url; - private Map headers = new LinkedHashMap<>(); + private List> headers = new ArrayList<>(); private String method = "GET"; private String postData; private String interfaceName; @@ -133,8 +134,8 @@ public class BasicCurlParser { private String filepathCookie=""; private Authorization authorization = new Authorization(); private String caCert = ""; - private Map formData = new LinkedHashMap<>(); - private Map formStringData = new LinkedHashMap<>(); + private List> formData = new ArrayList<>(); + private List> formStringData = new ArrayList<>(); private Set dnsServers = new HashSet<>(); private boolean isKeepAlive = true; private double maxTime = -1; @@ -201,7 +202,7 @@ public class BasicCurlParser { if ("COOKIE".equalsIgnoreCase(name)) { this.cookieInHeaders = value; } else if (!HEADERS_TO_IGNORE.contains(name)) { - headers.put(name, value); + headers.add(Pair.of(name, value)); } } @@ -239,8 +240,8 @@ public class BasicCurlParser { /** * @return the headers */ - public Map getHeaders() { - return Collections.unmodifiableMap(this.headers); + public List> getHeaders() { + return Collections.unmodifiableList(this.headers); } /** @@ -406,8 +407,8 @@ public class BasicCurlParser { /** * @return the map of form data */ - public Map getFormStringData() { - return Collections.unmodifiableMap(this.formStringData); + public List> getFormStringData() { + return Collections.unmodifiableList(this.formStringData); } /** @@ -415,14 +416,14 @@ public class BasicCurlParser { * @param value the value of form data */ public void addFormStringData(String key, String value) { - formStringData.put(key, value); + formStringData.add(Pair.of(key, value)); } /** * @return the map of form data */ - public Map getFormData() { - return Collections.unmodifiableMap(this.formData); + public List> getFormData() { + return Collections.unmodifiableList(this.formData); } /** @@ -430,7 +431,7 @@ public class BasicCurlParser { * @param value the value of form data */ public void addFormData(String key, String value) { - formData.put(key, value); + formData.add(Pair.of(key, value)); } /** diff --git a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java index e03ae17066..2a310cfb70 100644 --- a/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java +++ b/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/gui/action/ParseCurlCommandAction.java @@ -38,7 +38,6 @@ import java.util.List; import java.util.Map; import java.util.Set; -import javax.activation.MimetypesFileTypeMap; import javax.swing.JButton; import javax.swing.JCheckBox; import javax.swing.JLabel; @@ -52,6 +51,7 @@ import javax.swing.tree.TreePath; import org.apache.commons.io.FileUtils; import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.tuple.Pair; import org.apache.jmeter.config.Arguments; import org.apache.jmeter.config.KeystoreConfig; import org.apache.jmeter.control.Controller; @@ -101,6 +101,7 @@ import org.apache.jmeter.visualizers.ViewResultsFullVisualizer; import org.apache.jorphan.collections.HashTree; import org.apache.jorphan.gui.ComponentUtil; import org.apache.jorphan.gui.JMeterUIDefaults; +import org.apache.tika.Tika; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -118,6 +119,7 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato private static final String CREATE_REQUEST = "CREATE_REQUEST"; private static final String TYPE_FORM = ";type="; private static final String CERT = "cert"; + private Logger log = LoggerFactory.getLogger(getClass()); /** A panel allowing results to be saved. */ private FilePanel filePanel = null; static { @@ -126,6 +128,7 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato private JSyntaxTextArea cURLCommandTA; private JLabel statusText; private JCheckBox uploadCookiesCheckBox; + private final Tika tika = new Tika(); public ParseCurlCommandAction() { super(); } @@ -344,9 +347,8 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato headerManager.setProperty(TestElement.GUI_CLASS, HeaderPanel.class.getName()); headerManager.setProperty(TestElement.NAME, "HTTP HeaderManager"); headerManager.setProperty(TestElement.COMMENTS, getDefaultComment()); - Map map = request.getHeaders(); boolean hasAcceptEncoding = false; - for (Map.Entry header : map.entrySet()) { + for (Pair header : request.getHeaders()) { String key = header.getKey(); hasAcceptEncoding = hasAcceptEncoding || key.equalsIgnoreCase(ACCEPT_ENCODING); headerManager.getHeaders().addItem(new Header(key, header.getValue())); @@ -515,12 +517,12 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato throw new IllegalArgumentException("--form and --data can't appear in the same command"); } List httpFileArgs = new ArrayList<>(); - for (Map.Entry entry : request.getFormStringData().entrySet()) { + for (Pair entry : request.getFormStringData()) { String formName = entry.getKey(); String formValue = entry.getValue(); httpSampler.addNonEncodedArgument(formName, formValue, ""); } - for (Map.Entry entry : request.getFormData().entrySet()) { + for (Pair entry : request.getFormData()) { String formName = entry.getKey(); String formValue = entry.getValue(); boolean isContainsFile = "@".equals(formValue.substring(0, 1)); @@ -533,7 +535,12 @@ public class ParseCurlCommandAction extends AbstractAction implements MenuCreato formValue = formValueWithType[0]; contentType = formValueWithType[1]; } else { - contentType = new MimetypesFileTypeMap().getContentType(formValue); + try { + contentType = tika.detect(new File(formValue)); + } catch (IOException e) { + log.info("Could not detect contentType for file {} by content, so falling back to detection by filename"); + contentType = tika.detect(formValue); + } } httpFileArgs.add(new HTTPFileArg(formValue, formName, contentType)); } else { -- 2.25.1 ````
asfimport commented 3 years ago

eR@SeR (migrated from Bugzilla): (In reply to Felix Schumacher from comment 5)

For the third bug report part, you could look at https://github.com/apache/jmeter/issues/5372 which is probably the same as your report.

cURL is not the same as I reported, but if https://github.com/apache/jmeter/issues/5372 gets fixed, might be fixed for reported one. Please note that same happens if parameter or header values are blank.

(In reply to Felix Schumacher from comment 6)

For the second part (parameters which values start with @ landing in files upload tab), I believe, JMeter is doing the correct thing here.

The curl man page states in the '--form' section: "... To force the 'content' part to be a file, prefix the file name with an @ sign. ..."

If you want to send a value starting with an @ sign, you probably have to use --form-string as parameter. The curl man page for '-form-string' says: "... Similar to -F, --form except that the value string for the named parameter is used literally. Leading '@' and '<' characters, and the ';type=' string in the value have no special meaning. ..."

Maybe postman tries the value in the parameter, before it decides, whether this is a file or not. JMeter can't really decide this correctly, as the file has not to be present on creation of the test plan. It has to be present when the test plan is being run.

I am not sure, what to do about this, but it might be a good idea to discuss this on the mailing list and/or put this into its own bug report, as it will change the behaviour of detection.

You might missed difference between:

--form 'DocumentType="@ID_CARD"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \

@ in DocumentType parameter is inside quotes, but in BacksideImage is before quotes. Does this change anything?

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Now with test cases

Created attachment 0001-bug65024-curl-stuff-part-two.patch: Fixes for multi-header and -parameter and mime detection

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): (In reply to eR@SeR from comment 8)

(In reply to Felix Schumacher from comment 5) > For the third bug report part, you could look at > https://github.com/apache/jmeter/issues/5372 which is probably the > same as your report.

cURL is not the same as I reported, but if https://github.com/apache/jmeter/issues/5372 gets fixed, might be fixed for reported one. Please note that same happens if parameter or header values are blank.

The currently used parser for CLI options can not cope with empty parameters. So, I think it is the same report.

(In reply to Felix Schumacher from comment 6) > For the second part (parameters which values start with @ landing in files > upload tab), I believe, JMeter is doing the correct thing here. > > The curl man page states in the '--form' section: "... To force the > 'content' part to be a file, prefix the file name with an @ sign. ..." > > If you want to send a value starting with an @ sign, you probably have to > use --form-string as parameter. The curl man page for '-form-string' says: > "... Similar to -F, --form except that the value string for the named > parameter is used literally. Leading '@' and '<' characters, and the > ';type=' string in > the value have no special meaning. ..." > > Maybe postman tries the value in the parameter, before it decides, whether > this is a file or not. JMeter can't really decide this correctly, as the > file has not to be present on creation of the test plan. It has to be > present when the test plan is being run. > > I am not sure, what to do about this, but it might be a good idea to discuss > this on the mailing list and/or put this into its own bug report, as it will > change the behaviour of detection.

You might missed difference between:

--form 'DocumentType="@ID_CARD"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \

@ in DocumentType parameter is inside quotes, but in BacksideImage is before quotes. Does this change anything?

That changes the story :) I missed that one. curl does indeed differentiate between name="@..." and name=@"...".

asfimport commented 3 years ago

eR@SeR (migrated from Bugzilla): (In reply to Felix Schumacher from comment 10)

The currently used parser for CLI options can not cope with empty parameters. So, I think it is the same report. All right.

That changes the story :) I missed that one. curl does indeed differentiate between name="@..." and name=@"...". Nice :)

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Now, it tries to differentiate between "@value" and @"value".

Created attachment 0001-bug65024-curl-stuff-part-two.patch: Fixes for multi-header and -parameter and mime detection

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Another iteration on the remaining problems.

curl can do a lot of stuff when specifying parameters, like 'file=@filename;type=text/html' (which has been partly supported already) or 'param="something with spaces";headers=\"X-something: one\"'

The latter example is not possible right now, but a small step has been taken in that direction. Missing is also support for 'file=@"some/file";type=text/xml' that is a filename which was quoted.

Created attachment 0001-bug65024-curl-stuff-part-two.patch: Fixes for multi-header and -parameter and mime detection

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): Could you try next nightly or build from trunk and report back, whether your issues are solved?

Remember completely empty parameters like in "curl ''" are still not supported. But I think a lot of your reported ones are fixed.

commit f045cf5e1604c68b7d43986fe9bd82a102fa2b76 Author: Felix Schumacher <felix.schumacher@internetallee.de> AuthorDate: Wed Dec 23 10:45:44 2020 +0100

Sending mime type with parameter throws IllegalArgumentException

More stuff from that bugzilla entry:

* Better distinguishing between a quoted filename and a quoted name, that contains an @ char
* Allow more than one header and parameter with the same name to be added
* Simplify the code in ParseCurlCommandAction by moving some of the logic in extra classes

What is still missing are curl specialities like

 --form file=@"file with; semicolon or space";type=text/something
 --form param="red green blue";headers="X-Something: \"yeah; something\""

https://github.com/apache/jmeter/issues/5470

src/protocol/build.gradle.kts | 8 +++ .../jmeter/protocol/http/curl/ArgumentHolder.java | 55 +++++++++++++++ .../jmeter/protocol/http/curl/BasicCurlParser.java | 51 +++++++++----- .../protocol/http/curl/FileArgumentHolder.java | 80 ++++++++++++++++++++++ .../protocol/http/curl/StringArgumentHolder.java | 74 ++++++++++++++++++++ .../http/gui/action/ParseCurlCommandAction.java | 57 +++++++-------- .../apache/jmeter/curl/BasicCurlParserTest.java | 72 ++++++++++++++----- 7 files changed, 334 insertions(+), 63 deletions(-)

commit 9d2ba68c16a079c4e7156d847ed9acb7a6d4bb88 Author: Felix Schumacher <felix.schumacher@internetallee.de> AuthorDate: Wed Dec 23 17:12:47 2020 +0100

Sending mime type with parameter throws IllegalArgumentException

More stuff from that bugzilla entry:

* Allow empty headers to be added

https://github.com/apache/jmeter/issues/5470

.../apache/jmeter/protocol/http/curl/BasicCurlParser.java | 14 ++++++++++---- .../java/org/apache/jmeter/curl/BasicCurlParserTest.java | 10 ++++++++++ 2 files changed, 20 insertions(+), 4 deletions(-)

asfimport commented 3 years ago

eR@SeR (migrated from Bugzilla):

However, some commonly used extensions like .bmp .pdf .doc .docx .xls .xlsx .rar .zip etc. are still not recognized and generated MIME type is still application/octet-stream. You can check "Common MIME types" list on https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types and add all mentioned MIME types. I checked all extensions from the MIME types list and almost all were properly detected. Some are not, some differ a little bit, but I guess that is fine. Nice improvement though :)

  1. If parameter starts with @ like "@somevalue", form-data parameter is put inside Files upload tab instead to remain in Parameters tab.
  2. If same header or parameter name with same or different value is added, only last header or parameter is generated. All headers or parameters should be added even if they are duplicated.
  3. If URL, header or parameter value is blank, cURL is not imported. Those three are fixed except if URL is blank. As you mentioned, that is already reported in https://github.com/apache/jmeter/issues/5372

(In reply to Felix Schumacher from comment 13)

Created attachment 37646 [details] Fixes for multi-header and -parameter and mime detection

Another iteration on the remaining problems.

curl can do a lot of stuff when specifying parameters, like 'file=@filename;type=text/html' (which has been partly supported already) or 'param="something with spaces";headers=\"X-something: one\"'

The latter example is not possible right now, but a small step has been taken in that direction. Missing is also support for 'file=@"some/file";type=text/xml' that is a filename which was quoted. Probably, the following parameter and file parameter example that I've found might be related, so please confirm (or fix if not):

curl --location --request POST 'http://example.com/api/somepath' \ --header 'Accept-Language: it-IT' \ --header 'HTTP_X_FORWARDED_FOR: 127.0.0.1' \ --form 'FrontsideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'BacksideImage=@"/C:/Users/someuser/Desktop/Desktop.jpg"' \ --form 'ApplicationTypeId=""' \ --form 'Country="ITA"' \ --form 'DocumentType="IDCARD"' \ --form 'parameterWithAllSymbols="testTEST ~`!@#$%^&*()+1234567890[]{};'\''\:\"|,./<>?"' \ --form 'fileNameWithAllAllowedSymbolsByWin=@"/C:/Users/someuser/Desktop/Desktop 0123456789 `~!@#$&()_+{}[];'\''..jpg"'

Actual chars typed in Postman:

parameterWithAllSymbols = testTEST ~!@#$%^&*()_+1234567890[]{};'\:"|,./&lt;&gt;? fileNameWithAllAllowedSymbolsByWin = Desktop 0123456789~!@#$&()_+{}[];'..jpg

It looks like that ' and ; are problematic ones. If they are removed, cURL is properly imported.

Interesting part was if ; is put at the end of string, but without ' character in parameterWithAllSymbols value, cURL is properly imported, but the same for ' is not working.

You said 'missing support'. Is there a workaround to make exceptions by yourself? Maybe simple find and replace or regex can do the trick?

I noticed that ' is generated in Postman as '\'' Can find and replace fix it?

asfimport commented 3 years ago

@FSchumacher (migrated from Bugzilla): commit 66e492113bde973ce5d0ed163efb5aa61efe3145 AuthorDate: Fri Jan 15 18:54:12 2021 +0100

Remove leftover log message from debug session

Instead of logging at info level, log it with debug, only and remove the
unnecessary creation of a RuntimeException.

Cleanup after commit bb0c4557e10278f334844f8c1af873b708981b08
https://github.com/apache/jmeter/issues/5463

.../main/java/org/apache/jmeter/protocol/http/curl/BasicCurlParser.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)